2.1 Microvascular Visualization

Vascular visualization in biomedicine relies on traditional volumetric and isosurface rendering. These methods have generally been sufficient because either (1) available images do not obtain sufficient resolution to detect microvessels [22],[25] or (2) microscopy images have a small field-of-view (FOV) that minimizes volume and network size [7],[9]. When visualization of large, prominent arteries and veins is necessary for surgery preparation and assistance, flattening approaches have shown been used for robust visualization [12],[38]. Recent advances in imaging provide both sufficient resolution and volume extent, making traditional techniques impractical [1],[24],[33]. While recent visualization efforts support teravoxel-scale volumetric images [2], these methods don’t address the underlying model complexity and the resulting occlusion that makes microvascular visualization challenging. One promising approach is selective capillary visualization by culling a majority voxels based on graph connectivity [23]. While this method addresses the issue of occlusion, it provides very little insight into vascular structure across large volumes.

Figure 1:

(a) maximum intensity projection over 100 slices, with a much smaller region showing an extracted sub-graph (b). An edge represents a 3D curve between two vertices (blue), where a vertex is the 3D location of a micro-vessel. Where a 3D centerline is an approximation of a tubular structure with a varying radius (green) (c). The same graph can be summarized with an arc glyph (d). This summary shows the color of the cluster, the fraction of some global property represented by the cluster and some user selected local property.

Show All

Aggregation techniques enable exploring the statistical properties of microvascular networks. For example, the complex geometry can be simplified using a tensor or glyph-based visualization to summarize important features [13]. While glyph techniques have an advantage in preserving local and global anisotropic characteristics [34],[41], they remove structural details necessary for comparison between volumes. An area of few microvessels surrounded by healthy tissue would not be clearly visible using an aggregation technique.

2.2 Graph Visualization

Previous surveys on graph visualization identify the key issues of clarity and viewability [16], which are generally controlled by layout selection. The most prominent issue affecting viewability is a large number of crossing edges [31]. Radial and tree layouts are advantageous when there are only a few crossing edges [18]. Direct visualization of a pre-processed adjacency matrix is ideal for hierarchical graphs [15]. Otherwise, the most common graph layouts use iterative force-directed (FD) algorithms [20], which have O(V²E) complexity. The Fruchterman-Reingold algorithm is one of the earliest layout generation algorithms [10],[20], but suffers from common FD problems, such as getting caught in local minima [17].

Microvasculature can be represented as a highly-cyclical but sparse graph, with a density [8] on the order of 10⁻⁴ to 10⁻³ (in the mouse cortex) with large Cheeger constants [26]. This suggests that there are very few "bottlenecks" and a large number of redundant edges. The visual quality of these graphs is dependent on how planar they are. A common strategy for generating planar layouts is node-subdivision and combinatorics, where nodes and edges are merged or split to generate a planar graph. While this is often not an issue in information visualization, it would break down our direct mapping to physical structures, such as capillaries and branches. We generate an effective layout by maintaining the embedded correlation with physical structures and focus on leveraging connectivity for visualization and exploration.

3.1 Encoding a Microvascular Network as a Graph

We first encode the three-dimensional microvascular network using a graph. Network segmentation is performed using a GPU-based predictor-corrector algorithm [13] that extracts the vascular centerline while preserving connectivity and topology. The extracted medial axis is encoded into a graph G = {V,E} (Figure 1b and c). The vertex set V = [v₀,v₁,v₂…v_n] represents all capillary interconnections (branches). A property vector is specified for each vertex to store metrics such as degree, vertex connectivity, and closeness centrality. Each edge in E = {e₀,e₁,e₂...e_n} represents the medial axis using a variable number of points (Figure 1). Similar to the vertices, these points, their associated radii along with singular metrics including length, volume, and eccentricity are stored as edge property vector describing the changes between two connected vertices in V. We use the edge properties as candidates for color, width, and shape, as well as generating graph layouts.

3.2 Graph Cuts and Clustering

Our multi-modal approach leverages graph-based methods to allow annotation of the volumetric view. We generate rudimentary clusters using a full weighted affinity matrix A with edge properties, where a_ij is defined by the step function: aij={1,G(∑wij,σ),if i=jif i≠j(1)View Source\begin{equation*}{a_{ij}} = \begin{cases} {1,}&{{\text{if }}i = j} \\ {G\left( {\sum {{w_{ij}},\sigma } } \right),}&{{\text{if }}i \ne j} \end{cases}\tag{1}\end{equation*} where G(x,σ) is a normalized Gaussian function and w_ij is a user-selected property (e.g. microvessel length). When i ≠ j, we treat the value as the sum of weights over the shortest path between vertices i and j using the property w_ij as the edge length. Since capillary length is traditionally used to characterize microvascular networks in biomedical studies [7], most visualizations presented here make use of this property. A Gaussian kernel is applied to all weights w_ij to generate compact clusters. The standard deviation σ is chosen by fitting a one-sided distribution to the histogram of edge weights W_j.

An optimal cluster number is commonly selected using the eigen-gap heuristic [40], however data with poorly separated eigenvalue spectra does not have a singular best solution. This is the case for microvascular data, resulting in a small number of clusters (k ≈ 1). To mitigate this problem, we select the third-smallest eigengap, resulting in a value of k ≈ 10 to 20 for smaller networks and k ≈ 20 to 30 for larger networks.

Figure 2:

An illustrated exploration of a small graph network. The user first loads a microvascular network and is presented with a default layout (a). The user then looks at the overview glyphs and tweaks the layout to better facilitate their exploration (b). The user then selects a cluster highlighted in (b) for a detailed exploration. The user positions the camera in order to view the selected cluster in the context of the surrounding microvessels (c), and after being satisfied with the visualization, the user fades out the microvasculature surrounding the cluster (d) and begins exploring the metrics within the vertices of their chosen cluster using the camera in the 3D (e) to build a correlation between the graph topology and the 3D rendering.

Show All

3.3 Arc Glyph

We introduce an arc glyph to summarize cluster properties as an annotation to the graph. Vertices and edges within a cluster are summarized by user-specified properties to provide an overview of inter- and intra-cluster relationships (Figure 1d). The glyph center designates cluster (by color), the middle arc shows globally separable properties, and the outer arc shows properties within the cluster. Since the middle arc depends on the global property fraction, it allows the viewer to compare regions based on the clustering results. The outer arc supports the comparison of some multi-variate inter-cluster property. For example, this arc can display the distribution of node degrees within the cluster. From the visualization in Figure 1d, we can deduce that this cluster is central to the network because the size of the blue arc (Degree < 3) is much smaller than the size of the green arc (Degree ≥ 3), signaling a lower number of terminal edges. Summary glyphs are connected using line segments with thickness based on the number of connections between clusters. The potential use of each metric is demonstrated in Figures 2 and 3.

Finally, users can select clusters of interest via their glyphs that are then emphasized by making all other clusters translucent. This effect is duplicated in the three-dimensional view for the corresponding capillaries.

3.4 Mapping Between Graph and Volumetric Views

As discussed in Section 3, a set of points representing the centerline of a microvessel j: P^j and their corresponding radii R^j are stored as edge properties. This information used to build a three-dimensional mesh representing the network.

We introduce a novel projection strategy that allows a user to intuitively link between vertices in the 2D graph and microvasculature in 3D. This mapping uses transfer functions that calculate the distance between the camera and mesh components corresponding to graph features (branch points and capillaries). Equation 2 shows how this transfer function is calculated with respect to the vertices v_i.

dvi=∥pcamera−vi∥(2)

View Source\begin{equation*}{d_{{v_i}}} = \left\| {{{\mathbf{p}}_{camera}} - {{\mathbf{v}}_i}} \right\|\tag{2}\end{equation*}

These distance properties are normalized to the range [0.0,1.0]. The set of distance properties D_v is binned into five bins and used to assign an alpha value and scale-factor for corresponding edges and rendering graph nodes. This reduces the size and increases the translucency of vertices associated with branches further from the camera, providing context between the two visualizations and allowing the user to readily select components from either view port.

5.1 Feature Selection and Disease Characterization

We demonstrate one scenario for our workflow (Figure 2) to facilitate microvascular exploration. In this example, we study the relationships within a small microvascular network. The user uses the microvascular graph to first establish an intuitive mapping between the graph and volumetric visualization by manipulating the camera. Figures 2 (d-e) demonstrate how the camera, color, and distance metric isolate features across view ports. By positioning the camera closer to the smaller clusters of interest, the user can understand the relationship between capillaries and edges in the graph topology.

Cortical microvasculature exhibits fragmentation during the progression of Alzheimer’s disease, where approximately 5 to 10% of the microvascular network degrades. When exploring large volumes, occlusion limits visibility to fragmentation occurring near the volume boundary. Our method exposes fragmentation within the volume through the graph view using the summary arc glyphs (Figure 3). In this example we extract a fully connected network from a section of cortex. This produces a topology consisting entirely of D ≥ 3 vertices. We simulate progression of Alzheimer’s disease by removing 10% and 5% of the microvessels to produce fragmentation, resulting in D = 1 vertices (Figure 3b and c). Our method presents a clear advantage over tradition visualization by effectively mitigating the occlusion and visual homogeneity of the three-dimensional volume visualization.

Figure 3:

A simulated example that allows for drawing conclusions that are inaccessible using traditional visualization methods. We simulate the progression of Alzheimer’s disease by taking a fully connected, healthy microvascular network (a) and removing 5% during early stages (b) and 10% during the late stages (c) of the disease. In order to highlight the resulting broken edges, we overlay the healthy network over the culled network in the 3D visualization in (b) and highlight some areas where the difference is visible (A). Our application allows us to quickly identify the areas most affected using the outer arc of the overview glyphs which shows the degree distribution of each cluster.

Show All

Additionally we establish a visual relationship between the proportion of the size of the cluster silhouette in the graph view and the metric used to scale the vertices. Silhouette that appears visually larger intuitively communicates to the user that the cluster contains larger proportions of that metric since the size of the vertices is proportional to the value of the metric used in graph view. This, combined with the ability to dim all the capillaries belonging to clusters that hold little interest to the user and the summary arc glyphs, facilitates exploration of complex microvasculature without information overload.

We have found that clustering tends to cut larger vessels that serve as connections across clusters, which we annotate as the black microvessels (Figure 2a-c). Our experts believe that this is biologically relevant since those larger vessels are venules and arterioles used to carry blood to and from the capillary network. Therefore, this could potentially be used in segmentation of microvascular volumes to annotate vascular trees. Spectral clustering therefore improves separation between clusters while generating clusters that divide microvasculature into subsystems linked by arterioles and venules.

5.2 Expert Evaluation

We provided our interface to domain experts studying relationships between microvascular networks in Alzheimer’s disease. We asked general questions related to their ability to form new conclusions with our interface. Additionally, we also recorded their feedback to the interface design.

Our visualizations centered on regions of the mouse brain collected using knife-edge scanning microscopy (KESM) [24]. We visualized areas of the brain exhibiting high fragmentation (Figure 3). The experts were interest in the application potential, mentioning that "the ability to visualize large structural changes would be novel in the study of Alzheimer’s and cancer." This type of analysis is currently limited to averaging vascular volume over regions of interest, so the ability to visualize connectivity across large volumes could provide significant benefit in quantifying disease progression. In cases where both degeneration and growth were observed, previous methods fail to capture a significant difference between volumes. Experts also mentioned that the arc glyphs allowed them to "easily focus on the areas of significant degeneration based on the outer ring of the arc glyph." This provides potential for studying tumorogenesis and cancer treatment, since angiogenesis during tumor growth also exhibits fragmentation that can be partially repaired through a variety of anti-angiogenic treatments.

We received feedback that the ability to use the zoomed in view was "less useful due to the distraction caused by the overlapping intra-cluster edges." However, in the zoomed out view using the camera to explore specific section would "improve the quality of the visualization in our publications." Focus on a single cluster by applying dimming to external microvessels compensated for some of these inadequacies. While the experts mentioned that this framework would allow " "novel paradigms", they did state that further refinement of the system may be required.