Network-based statistic: Identifying differences in brain networks

Introduction

It has become fashionable in the field of neuroimaging to model and analyze the brain in terms of a network. With the popularization of complex systems theory and the emergence of network science, graph models have become especially popular for they open up the door to a systems-theoretic description of the brain. That is, a description where brain complexity can be reduced to an account of the interactions between basic neuronal elements.

The graph model of the brain is an abstract structure used to represent pairwise relations between interregional ensembles of neuronal elements, referred to as nodes. These pairwise relations, or links, can either be of a functional origin and represent coherent physiological activity between neural ensembles, or they can be of a structural origin and represent anatomical connections formed by white-matter axonal fiber tracts.

The graph model has been used in neuroimaging research as a framework to test the structure-function hypothesis (Honey et al., 2009, Honey et al., 2010, Ramani et al., 2004, Skudlarski et al., 2008, Sporns et al., 2000) and as a methodological tool to examine brain network organization, topology and complex dynamics (Bullmore et al., 2009, Hagmann et al., 2008, Sporns et al., 2004). It has been found that the human brain exhibits various nontrivial organizational and topological properties, such as: assortativity, centrality, clustering, efficiency, hierarchy, hubs, modularity, robustness, small-worldness, synchronizability, etc. (see Bullmore & Sporns, 2009, for a review). Differences in one or more of these properties have been found in people with Alzheimer's disease (He et al., 2008, Stam et al., 2007), attention-deficit disorder (Wang et al., 2009), multiple sclerosis (He et al., 2009) and schizophrenia (Bassett et al., 2008, Liu et al., 2008, Rubinov et al., 2009) as well as associations with age and gender (Gong et al., 2009, Wang et al., 2010), and intelligence (van den Heuvel et al., 2009, Li et al., 2009).

The graph model also provides an ideal framework to identify functional or structural connections associated with a particular effect or contrast of interest; for example, a group difference in a case-control comparison, a difference due to changing task conditions in a functional paradigm, or a correlation with some clinical measure. To this end, mass-univariate testing of the hypothesis is undertaken, after which the family-wise error rate (FWE) is controlled with a generic procedure, such as the false discovery rate (FDR) (Genovese et al., 2002). Specifically, a test statistic and corresponding p-value is independently computed for each link based on the strength of the pairwise association the link represents. The strength of a pairwise association between nodes is usually measured as the value of temporal correlation (functional) or the total number of interconnecting streamlines (structural).

The advantage of this approach is that it does not require interpretation of any abstract organizational or topological properties. Its main disadvantage though is the inherent massive number of multiple comparisons that must be performed. To appreciate the massiveness of the multiple comparisons problem, consider the following: performing a mass-univariate analysis on a statistical parametric map involves in the order of thousands of multiple comparisons, say 1000 voxels for concreteness; however, if each of these 1000 voxels delineates a unique node, the number of multiple comparisons increases to a staggering $\left.\begin{array}{c}\hfill 1000\hfill \\ \hfill 2\hfill \end{array}\right)=499,500$ connections. With such a large number of multiple comparisons, together with a potentially low contrast-to-noise ratio, mass-univariate testing may not offer sufficient power.

Note that when the hypothesis of interest is associated with a global network measure, the issue of multiple comparisons does not arise. This paper exclusively focuses on multiple hypothesis testing, where one instead tests the hypothesis of interest at each network connection, thereby introducing more localizing power at the cost of a massive number of multiple comparisons.

The main contribution of this paper is to present a potentially more powerful method to control the FWE when performing this kind of analysis. The new method is called the network-based statistic (NBS) and can be thought of as a translation of conventional cluster statistics (Bullmore et al., 1999, Nichols and Holmes, 2001) to a graph. In brief, the NBS operates as follows: foremost, the test statistic computed for each link is thresholded to construct a set of suprathreshold links. Any connected structures, or components in graph parlance, that may be present in the set of suprathreshold links are then identified. A p-value is assigned to each identified component by indexing its size with the null distribution of maximal component size.

In this way, the NBS attempts to utilize the presence of any structure exhibited by the connections comprising the effect or contrast of interest to yield greater power than what is possible by independently correcting the p-values computed for each link using a generic procedure to control the FWE. In this paper, any procedure for controlling the FWE that treats each link independently will be referred to as a link-based controlling procedure, or simply link-based FWE control.

The NBS is not intended as a replacement for link-based FWE control. In particular, the NBS offers no power if the links associated with the contrast or effect of interest are in isolation of each other and do not form any connected structures. Therefore, in addition to introducing and demonstrating the utility of the NBS, one of the main purposes of this paper is to undertake a quantitative evaluation of the gain (and potential loss) in statistical power offered by our new approach. To this end, the paper comprises three parts: Section 2: introduction of the NBS by way of an illustrative example; Section 3: evaluation of the power of the NBS with use of receiver operating characteristic (ROC) curves; and, Section 4: demonstration of the NBS in the context of a real case-control study involving resting-state functional MRI data acquired in 12 people with schizophrenia and 15 controls.

Methods

To use the NBS, one must first generate a connectivity matrix for each subject. The connectivity matrix is intrinsic to the graph model and its computation has been described in (Bullmore & Sporns, 2009) and many references therein. As such, the main focus of this section is the implementation of the NBS after the connectivity matrix stage has been reached, though a brief overview of getting to this stage is provided for completeness.

Network-based statistic

For ease of exposition, the NBS is described here within the framework of a generic case-control study. However, it is important to note that the NBS is easily generalizable to multifactorial designs stemming from, for example, task-based functional MRI studies, as we have recently demonstrated in Fornito et al., 2010.

The purpose of the NBS in the context of a case-control study is to identify any pairwise associations that are significantly different between groups. In the graph model, a pairwise association manifests as a “connection,” or link, between a distinct pair of nodes. An N × N connectivity matrix stores the estimates of a total of N(N − 1)/2 unique pairwise associations. For each of these associations, the test statistic of interest is calculated independently using the values stored in each subject's connectivity matrix. In some circumstances, it may be appropriate to first transform the raw measure of association; for example, Fisher's r-to-z transform can be applied to a correlation based measure of association to ensure normality. This is not a specific requirement of the NBS, which is a nonparametric technique, but yields a sensible measure of variation to which a threshold can be applied.

The NBS seeks to identify any potential connected structures formed by an appropriately chosen set of suprathreshold links. The topological extent of any such structure is then used to determine its significance. This process is performed as follows: the test statistic computed for each pairwise association is thresholded to construct a set of suprathreshold links. Any connected components that may be present in the set of suprathreshold links are then identified using a breadth first search (Ahuja et al., 1993) and the number of links they comprise, or their size, is stored.

Permutation testing is used to ascribe a p-value controlled for the FWE to each connected component based on its size. A total of M random permutations are generated independently, where for each permutation, the group to which each subject belongs is randomly exchanged. For each permutation, the test statistic of interest is recalculated, after which the same threshold is applied to define a set of suprathreshold links. The maximal component size in the set of suprathreshold links derived from each of the M permutations is then determined and stored, thereby yielding an empirical estimate of the null distribution of maximal component size. Finally, the p-value of an observed component of size k is estimated by finding the total number of permutations for which the maximal component size is greater than k and normalizing by M. This kind of permutation testing is not new and is synonymous with conventional cluster-based thresholding of statistical parametric maps (Bullmore et al., 1999, Hayasaka and Nichols, 2004, Nichols and Holmes, 2001). The novelty here is in the translation and application of this approach to the graph model (see Fig. 1). The approach can also be straightforwardly generalized to multifactorial designs (Suckling & Bullmore, 2004).

1. Download : Download high-res image (55KB)
2. Download : Download full-size image

Fig. 1. The equivalence between a cluster in a statistical parametric map (left) and a component in a graph (right). Both are of extent five: the cluster comprises five voxels, while the component comprises five links. Note that a cluster is defined by spatial contiguity of suprathreshold voxels in physical image space, whereas a graph component is defined by interconnectedness of suprathreshold links in topological space.

To implement the NBS, an algorithm is required to compute the set of all connected components in a graph. This can be achieved in a runtime of $\mathcal{O}$(N + L) using a breadth-first search (Hopcroft & Tarjan, 1973), where N is the number of nodes and L is the number of suprathreshold links. This is repeated for M permutations to estimate the null distribution, thereby yielding a total runtime of $\mathcal{O}$(M(N + L)). In practice, the majority of runtime is devoted to computing the test statistic of interest at each link.

Illustrative example

To conclude this section, an illustrative example is presented in Fig. 2 to demonstrate the gain in power offered by the NBS relative to link-based FWE control, in a situation that is suited to the NBS; namely, a relatively low contrast-to-noise ratio of unity and a contrast that is defined by a set of links that form a single connected component.

1. Download : Download high-res image (1MB)
2. Download : Download full-size image

Fig. 2. The network-based statistic (NBS) as well as link-based FWE control provided by the false discovery rate (FDR) were used to detect a contrast that was simulated between two groups: (i) a connected component, referred to as the contrast, was disrupted in one of the groups to yield a contrast-to-noise ratio of unity between the two groups. The red blocks of the adjacency matrix indicate links comprising the contrast, while the white blocks indicate the other links that were tested but were not part of the contrast. (ii) The FDR was used to identify the component using false discovery rate thresholds of q = 5, 10 and 20%. (iii) The NBS was then used with primary (t-statistic) thresholds of t = 1, 1.5 and 2. True positives, colored orange, correspond to connections that were part of the contrast and correctly identified as such, while false positives, colored red, correspond to connections that were not part of the contrast but incorrectly identified as such. Each component identified by the NBS satisfied p < 0.01. With link-based FWE control, the full extent of the contrast only became evident for a liberal false discovery rate threshold. The true and [false] positive rates for each threshold were: FDR: q = 5%: 0.3[0]; q = 10%: 0.5[0]; q = 20%: 0.7[0.006]; and NBS: t = 1: 1[0.08]; t = 1.5: 0.9[0.01]; t = 2: 0.9[0.006]. Nodes are depicted at their two-dimensional center of mass. The two components evident for t = 2 were each of sufficient size to be declared significant in their own right.

For the purpose of the example, a between group difference was simulated in which a connected component, referred to as the contrast, was “disrupted” in one of the two groups. In one group, the measure of association assigned to each link was sampled from a Gaussian distribution of zero mean and unity variance. In the other group, sampling was also from a Gaussian distribution of unity variance, but with unity mean if the link comprised the contrast, otherwise with a mean of zero. This yielded a contrast-to-noise ratio of unity. A t-statistic and corresponding p-value was then calculated for each link, after which the NBS and the FDR were used to identify the contrast (see caption of Fig. 2 for details). The graph model comprised a total of 82 nodes, each of which was extracted from the AAL atlas.

In Fig. 2, it can be seen that for even a very liberal FDR threshold of q = 20% (Genovese et al., 2002), link-based FWE control yielded a true positive rate of 0.7, with a false positive rate of 0.006. However, with the NBS, for p = 0.01 corrected, the true positive rate was 0.9, with a false positive rate of 0.006 (t = 3). It is crucial to remember though that this gain in power is at the cost of offering only weak control of the FWE.

Performance evaluation

Receiver operating characteristic (ROC) curves are presented in this section to evaluate the specificity and sensitivity of the NBS as well as link-based FWE control under a range of different network topologies, contrast extents and contrast-to-noise ratios. The FDR served as the link-based controlling procedure, and thus any reference to FDR in this section is synonymous with link-based FWE control.

An ROC curve is a plot of the true positive rate (TPR) against the false positive rate (FPR), which is generated by varying the discrimination threshold over a continuous range. In the case of the NBS, the discrimination threshold is the minimum size of a component for it to be declared significant, while for the FDR, the q-value (Genovese et al., 2002) serves as the discrimination threshold. Ideal performance is indicated by a point in the upper left corner of the ROC curve, which represents 100% sensitivity (no false negatives) and 100% specificity (no false positives).

To construct an ROC curve, a contrast was simulated, corrupted with noise and then the NBS and the FDR were separately used to identify the contrast for a range of discrimination thresholds. The TPR was computed as |$\mathcal{P}$ ∩ ĥ|/|$\mathcal{P}$| and the FPR was computed as |$\mathcal{N}$ ∩ ĥ|/|$\mathcal{N}$|, where $\mathcal{P}$ was the set of links comprising the contrast, $\mathcal{N}$ was the set of links comprising the rest of the network and ĥ was the set of links declared significant. This was repeated independently for 10³ trials and the average TPR and FPR was computed across the trials for each threshold. In this way, the TPR and FPR were parameterized in terms of the discrimination threshold. The TPR was then plotted as a function of the FPR in the range 0.001 to 0.05. An FPR exceeding 0.05 is most likely of no interest in a typical analysis and was therefore not considered, while an FPR below 0.001 was far too stringent to detect the contrast given the contrast-to-noise ratios evaluated.

The Barabási–Albert (BA) model (Albert & Barabási, 2002) was used to generate a random, scale-free network for each trial using a preferential attachment mechanism. A scale-free model was chosen because some studies have shown that brain networks are scale-free (Eguiluz et al., 2005, van den Heuvel et al., 2008), although it is important to remark that other studies have presented evidence for different degree distributions (Achard et al., 2006). The BA model is attractive because it is fully connected by construction, and it can be parameterized with only two variables, the number of nodes comprising the network, N, and a variable, m, which denotes the number of existing nodes to which a newly added node is attached to during each iteration of the preferential attachment mechanism. Hence, the number of links in the BA model is approximately Nm.

The next step was to create a contrast for each network. The contrast comprised a set of links that necessarily formed a connected component. The contrast was created by initiating a breadth first search (Ahuja et al., 1993) from a randomly chosen node in the network. Each link traversed during the search was included in the contrast and the search was terminated as soon as the contrast grew to reach its desired size.

Finally, a between group difference was created with respect to the contrast. In one group, the measure of association assigned to each link was sampled from a Gaussian distribution of zero mean and unity variance. In the other group, sampling was also from a Gaussian distribution of unity variance, but with a mean of μ if the link comprised the contrast, otherwise with a mean of zero. This yielded a contrast-to-noise ratio of μ.

The steps undertaken to construct an ROC curve can be summarized as follows:

• (i)

  Generate a network with N nodes using the BA preferential attachment mechanism. Let $\mathcal{L}$ be the set of links comprising the network.

• (ii)

  Initiate a breadth first search from a randomly chosen node to define a set of links, $\mathcal{P}$ ⊂ $\mathcal{L}$, to serve as a contrast. Terminate the search as soon the contrast is of the desired size.

• (iii)

  Let a_l be the measure of association at link l ∈ $\mathcal{L}$. In one group, sample such that a_l ∼ N(0, 1) for all l ∈ $\mathcal{L}$, which yields a noise-only network. In the other group, a_l ∼ N(μ, 1) if l ∈ $\mathcal{P}$, otherwise a_l ∼ N(0, 1), which yields a noise + contrast network.

• (iv)

  Perform mass-univariate testing for a between group difference; in particular, compute a t-statistic and corresponding p-value for each link.

• (v)

  Use the NBS as well as link-based FWE control to yield an estimate, ĥ, of the set of links comprising the contrast for a range of discrimination thresholds.

• (vi)

  For each discrimination threshold, compute TPR = |$\mathcal{P}$ ∩ ĥ|/|$\mathcal{P}$| and FPR = |$\mathcal{N}$ ∩ ĥ|/|$\mathcal{N}$|.

• (vii)

  Repeat the above for 10³ trials to yield an average FPR and TPR.

Performance evaluation results

The ROC curves are presented in Fig. 3, Fig. 4, Fig. 5. Fig. 3 considers a contrast of 10 links embedded in a network comprising 100 nodes and 200 links (m = 2) for contrast-to-noise ratios of: (i) 0.5, (ii) 1 and (iii) 1.5. A crossover point is evident for each ROC curve in Fig. 3. To the right of the crossover point, the NBS curve is higher than the FDR curve, signifying the NBS offers greater power over the range to the right of the crossover point. To the left of the crossover point, the opposite is true, and thus the FDR offers greater power over this range. The NBS clearly offers the greatest power over the TPR range probably of most interest, 0.5 to 1, although the advantage of the NBS declines as the contrast-to-noise ratio increases, until both approaches perform equally.

1. Download : Download high-res image (327KB)
2. Download : Download full-size image

Fig. 3. Receiver–operator characteristic (ROC) curves were plotted to objectively compare the specificity and sensitivity of: the network-based statistic (NBS); and, link-based FWE control provided by the FDR. A separate ROC was plotted for three different contrast-to-noise ratios (CNR). To generate each ROC curve, the NBS and the FDR were used to detect a contrast that was simulated between two groups. In particular, a random, scale-free network of 100 nodes was generated and the connectivity in a component comprising 10 links was altered in one group to yield a CNR between the two groups of: (i) 0.5, (ii) 1 and (iii) 1.5. This was repeated 1000 times, each time generating a new random network and a new contrast. The average true positive rate (TPR) over the 1000 realizations was then plotted as function of the average false positive rate (FPR). Each point along the ROC curve for the FDR (colored blue) corresponds to a distinct q-value, some of which have been indicated as a percentage. For the NBS (colored red), each point represents a distinct component size threshold (i.e. components of size exceeding this threshold are declared significant). Observe that the NBS curve abruptly terminates and does not extend to an FPR of 0.1. This termination point represents the minimum possible component size threshold of unity.

1. Download : Download high-res image (298KB)
2. Download : Download full-size image

Fig. 4. The effect of a change in network size. In Fig. 3, the network comprised 100 nodes, 200 links and a contrast comprising 10 links. Here, the experiment was repeated for a larger network comprising 500 nodes and 1000 links, also with a contrast of 10 links. Therefore, the ratio of the total number of links to the number of links comprising the contrast was increased 5-fold from 200/10 = 20 to 1000/10 = 100. This figure demonstrates that the gain in power offered by the NBS increases with network size. See Fig. 3 caption for further details.

1. Download : Download high-res image (341KB)
2. Download : Download full-size image

Fig. 5. The effect of a change in network sparsity. In Fig. 3, the network sparsity was 4% (100 nodes and 200 links). Here, the experiment was repeated for networks of sparsity: (i) 2%, (ii) 8% and (iii) 16%. Sparsity was altered by increasing the number of connections and maintaining the total number of nodes fixed at 100. A unity contrast-to-noise (CNR) ratio was used. This figure shows that the gain in power offered by the NBS is relatively unchanged to variations in sparsity. See Fig. 3 caption for further details.

In Fig. 3, q-values have been indicated at strategic points along the FDR curve. The q-value is essentially the FDR analogue of the p-value, and is the minimum FDR at which significance is declared. The disadvantage of the FDR here is that a very high q-value (q = 20–50%) is required to achieve a reasonable FPR. For example, Fig. 3(ii) shows that a q-value threshold exceeding 20% is required to enforce a rather strict FPR of 0.01. This suggests that the FDR may be overly conservative in cases of low contrast-to-noise ratio.

Fig. 4 considers a substantially larger network than Fig. 3 to evaluate the effects of a change in network size. Specifically, Fig. 4 considers a contrast of 10 links embedded in a network comprising 500 nodes and 1000 links (m = 2) for the same set of contrast-to-noise ratios. Therefore, relative to Fig. 3, the ratio of the number of links comprising the contrast to the total number of links is reduced 100-fold from 10/100 to 10/1000. The same features are evident in Fig. 4; however, the crossover point is shifted further to the left and now occurs at an FPR well below 0.01. This indicates that the FPR range over which the NBS is favorable increases with network size.

Finally, Fig. 5 considers a change in network sparsity. In Fig. 3, the network sparsity was 4%. In Fig. 5, the sparsity is increased to (i) 2%, (ii) 8% and (iii) 16%. Sparsity was altered by including more links and maintaining the total number of nodes fixed at 100. In particular, the preferential attachment mechanism underlying the BA model was implemented with an increased value of m. Fig. 5 suggests that variations in sparsity impart a negligible effect.

The area under each ROC curve is shown in Supplementary Table S1.

While it is important to keep in mind that these conclusions do not necessarily generalize to problems of an arbitrary nature, it may be useful to summarize the key features evident in Fig. 3, Fig. 4, Fig. 5:

• −

  The NBS is of no use if the contrast does not form a connected component. The following remarks pertain only to scenarios where the contrast does form a component.

• −

  A single crossover point exists at which the NBS and the FDR offer equal power. The NBS offers greater power to the right of the crossover point. The right of the crossover point encompasses the TPR region of most interest, 0.5 to 1.

• −

  As the contrast-to-noise ratio is increased, both the FDR and the NBS perform equally well.

• −

  The gain in power offered by the NBS increases as the number of nodes and links is increased, with the contrast extent held constant, but is insensitive to variations in sparsity.

• −

  A very high q-value (q = 20–50%) is required to operate over the range considered, indicating that the FDR may be overly conservative in cases of low contrast-to-noise ratio.

Finally, it is important to remark that the abscissa of each ROC curve is the expected proportion of false positives per trial. An alternative would have been to quantify the FPR as the proportion of trials for which there exists at least one false positive. Both definitions have distinct pros and cons. The former definition is the more conventional of the two and enables a better characterization of accuracy as well as requiring simulation of fewer trials. The main advantage of the latter definition is that it represents the strictest possible control of the FWE. The former definition may be considered more appropriate here given that the NBS is not intended to offer any control of the FWE at the level of individual links.

Application

The purpose of this section is to present new results derived from application of the NBS to a real case-control study involving a group of people with schizophrenia. In this section, a pair of nodes showing a weaker association in the group with schizophrenia is referred to as a dysconnection and the set of all such dysconnections is referred to as the dysconnected subnetwork.

Dysconnected subnetwork

The NBS identified a single dysconnected subnetwork (p = 0.037 ± 0.005, corrected) in the group with schizophrenia comprising 40 functional dysconnections. The confidence interval for the p-value was estimated parametrically as $2\sqrt{(p(1-p)/M)}$. Fig. 8 shows an axial schematic of the subnetwork, while Fig. 9 shows a three-dimensional visualization with both an oblique and axial perspective. Table 1 lists the nodes comprising the dysconnected subnetwork as well as their degree in this subnetwork, anatomical name and abbreviation.

Table 1. Node abbreviations and degree.

Abbreviations	Node	Degree
PreC	Precentral L	2
	Precentral R	1
Rola	Rolandic Oper L	3
	Rolandic Oper R	4
SMA	Supp Motor Area L	7
	Supp Motor Area R	3
Cgl2	Cingulum Mid L	2
	Cingulum Mid R	1
Calc	Calcarine L	2
	Calcarine R	1
PstC	Postcentral L	6
	Postcentral R	2
Hesc	Heschl L	13
	Heschl R	1
Tem1	Temporal Sup L	1
	Temporal Sup R	1
Fro1	Frontal Sup L	1
Fro8	Frontal Inf Orb L	2
Fro9	Frontal Sup Medial L	1
Fro0	Frontal Med Orb L	6
Insu	Insula L	1
Cgl1	Cingulum Ant L	1
Hipp	Hippocampus L	3
Amy	Amygdala L	1
Cune	Cuneus L	3
Ling	Lingual R	3
Occ1	Occipital Sup L	5
Par1	Parietal Sup L	1
PCun	Precuneus L	2

When considering Fig. 8, Fig. 9, it is important to remember that no individual dysconnection can be declared significant alone, only the dysconnected subnetwork as a whole can be declared significant. With respect to Fig. 8, it should additionally be noted that each node is positioned at its two-dimensional center of mass (left-right and anterior-posterior), resulting in no depth in the axial plane. This gives the illusion that the SMA nodes (supplementary motor area), for example, are located within the ventricles, which occurs simply because the axial slice chosen as the underlay is far more inferior than the SMA.

The dysconnected subnetwork identified with the NBS predominantly comprises fronto-temporal and occipito-temporal dysconnections, as well as SMA-temporal, SMA-occipital and the cingulum. While the dysconnected subnetwork is relatively bilateral, the left hemisphere is clearly affected to a greater extent. Indeed, the node from which the greatest number of dysconnections originate is the left Heschl's gyrus (Hesc), followed by the left post central gyrus (PstC) and the left SMA (see Table 1). Furthermore, the frontal regions implicated are exclusive to the left hemisphere, and thus the set of fronto-temporal dysconnections are asymmetric in the sense that the right temporal nodes show dysconnection with the left frontal nodes, but the converse is not evident.

The empirically computed null distribution of maximal component size is plotted in Fig. 6.

1. Download : Download high-res image (128KB)
2. Download : Download full-size image

Fig. 6. Empirically computed null distribution of maximal component size. The component identified in the real data comprised 40 connections.

Comparison to link-based FWE control

For a rather liberal minimum FDR threshold of q = 10%, link-based FWE control only identified a single dysconnection between the left post-central gyrus (PstC) and left rolandic operculum (Rola). This is in fact one of the dysconnections constituting the dysconnected subnetwork identified by the NBS. No significant findings were evident for the more commonly used threshold of q = 5%. Fig. 7 depicts the dysconnections identified by the NBS and the FDR in the form of separate adjacency matrices. A dysconnection is shown as a white block, while all other connections are colored black.

1. Download : Download high-res image (55KB)
2. Download : Download full-size image

Fig. 7. Adjacency matrices showing dysconnections (white blocks) in the group with schizophrenia identified with: (a) link-based FWE control provided by the FDR; and, (b) the NBS. The FDR identified a single dysconnection between the left post-central gyrus (PstC) and left rolandic operculum (Rola), while the NBS identified a single dysconnected subnetwork primarily comprising fronto-temporal and occipito-temporal dysconnections (see Fig. 8).

Discussion

With continuing popularization of the graph model in all kinds of neuroimaging research, we believe there is a need for statistical approaches to identify connections in this model that may be associated with an effect or contrast of interest; for example, diagnostic status in a case-control comparison, a difference due to changing task conditions in a functional paradigm, pharmacological modulation, or correlation with some external behavioral measure. While mass-univariate testing is an option, the enormous number of multiple comparisons involved, together with a potentially low contrast-to-noise ratio, means that this approach may be underpowered. The NBS is a new option that can offer substantially greater power if the connections associated with the contrast or effect of interest are interconnected to form a structure; or more precisely, a connected component. The underlying principle behind the NBS is to utilize the presence of any such components to potentially yield greater power than what is possible by independently correcting the p-values computed for each connection comprising the graph with a generic procedure such as the FDR (Figs. 8 and 9).

1. Download : Download high-res image (1MB)
2. Download : Download full-size image

Fig. 8. The NBS identified a dysconnected functional subnetwork (p = 0.037 ± 0.005, corrected) in a group of 12 people with schizophrenia. The dysconnections comprising this subnetwork correspond to pairs of nodes between which the resting-state times series (wavelet decomposed) was more weakly correlated in the group with schizophrenia than in the control group. It is important to emphasize that no individual dysconnection can be declared significant alone, only the dysconnected subnetwork as a whole can be declared significant. Each node is depicted as a circle positioned at its center of mass. See Table 1 for node abbreviations.

1. Download : Download high-res image (440KB)
2. Download : Download full-size image

Fig. 9. Three-dimensional visualization of the dysconnected functional subnetwork identified with the NBS in a group of 12 people with schizophrenia. See caption of Fig. 8 for details. Red: occipital/parietal nodes. Green: temporal nodes. Purple: pre and post central gyrus. Light blue: frontal nodes. Blue: cingulum. Magenta: Supp. Motor Area.

But is the NBS really necessary? Indeed, if the contrast-to-noise ratio is sufficiently high, generic procedures that operate on the link-based p-values alone can presumably provide enough power to declare the effect of interest significant. Moreover, generic procedures such as the FDR offer three distinct advantages over the NBS. Foremost, the NBS must be used under the assumption that the connections comprising the contrast of interest form components. If they do not form components, or if the extent of the components formed are too small, the NBS is ineffective, whereas generic procedures providing link-based FWE control do not require any such assumption. Secondly, the NBS only offers weak control of the FWE; that is, with the NBS, only the omnibus null hypothesis for a component can be rejected, but the individual connections comprising a component can never be declared significant. In this way, the NBS provides coarser localizing power. And thirdly, when using the NBS, a rather arbitrary choice must be made to select the value of the threshold used to define the set of suprathreshold links.

So why use the NBS, given that its localizing power is coarser and it necessitates an assumption on the interconnectedness of the contrast? Firstly, we believe that the assumptions on which the NBS is based are implicit in a typical network analysis of neuroimaging data. In particular, the hypothesis tested usually involves examination of effects distributed across various components of information-processing pathways. While it is possible for highly focal effects to exist in a given network, most networks, being interconnected by definition, are likely to show secondary consequences of any focal effects that can propagate along interconnected pathways. Thus, most effects are likely to influence interconnected subnetworks that the NBS is well suited to detecting.

The second reason is simply a matter of power—the NBS can offer substantially greater power in the right circumstances, which is advantageous in the context of the graph model due to the massive number of multiple comparisons that arise when the hypothesis of interest is tested at every connection. To appreciate the massiveness of the multiple comparisons problem, an example was given where it was shown that the number of multiple comparisons scaled polynomially with the number of voxels of interest, namely $\mathcal{O}$(N²), whereas the number of multiple comparisons performed in a statistical parametric map is linear in the number of voxels of interest. Therefore, due to this massive increase, the contrast-to-noise ratio ought to be very high if a generic procedure is to have any chance of declaring the contrast significant. Unfortunately, the range of measures available to quantify pairwise associations between nodes is derived from MRI data or other experimental imaging modalities, which are invariably noisy. It is thus inevitable for this noise to propagate to the measure of association, potentially resulting in a low contrast-to-noise ratio. For example, the number of interconnecting streamlines, a measure of anatomical connectivity derived from diffusion MRI, is notoriously noisy due to artifacts stemming from tractography (Zalesky & Fornito, 2009).

As such, a conventional mass-univariate analysis within the framework of the graph model may not be able to provide sufficient power. This lack of power motivated the development of the NBS. The NBS is founded on the same principle underlying traditional cluster-based thresholding of statistical parametric maps. The main difference is that graph components play the role of voxel clusters.

With the NBS, the link-based p-values corresponding to the contrast or effect of interest need not be so significant as to survive in a sea of potentially hundreds of thousands of multiple comparisons (as would be required by the FDR). Instead, they only need to be significant enough to be admitted into the set of suprathreshold links, and assuming they are interconnected to form a sufficiently large component (or components), they can be declared significant. In a mass-univariate analysis controlled with the FDR though, to survive in a sea of hundreds of thousands of multiple comparisons, the contrast-to-noise ratio would need to be high in order to yield very significant link-based p-values.

An advantage of the NBS is that its power may scale with the size of the network. Suppose the cortex is coarsely subdivided into a total of 10 nodes, and the NBS as well as conventional mass-univariate testing identify a significant effect involving say three connections giving rise to a component of extent three. For a coarse subdivision such as this, a node is likely to encompass several functionally distinct regions, and thus any dysconnections between regions that are lumped together within a common node cannot be identified because they are simply not modeled. Now suppose the cortex is subdivided more finely into a total of 1000 nodes. The number of multiple comparisons increases from 45 to 499,500, and thus a mass-univariate analysis may not have sufficient power to detect the effect unless the contrast-to-noise ratio is high. The NBS is also likely to have difficulty in identifying a component of extent three in a network comprising 499,500 connections, for the possibility of a component of larger or equal extent arising as a matter of chance increases dramatically relative to the 10 node case. However, the effect is likely to be more extensive now and therefore easier to detect because dysconnections between nodes that were previously lumped together within a common node now arise. In other words, although the multiple comparisons problem is exacerbated as the size of the network is increased, the extent of the contrast or effect of interest is also likely to scale up with network size.

The NBS attempts to exploit the interconnected nature of link-based effects, as revealed through mass univariate testing. This makes the approach relatively simple to implement and interpret. However, brain connectivity data are inherently multivariate in nature. As such, multivariate methods may provide added sensitivity for detecting alterations of connectivity under various conditions. In a multivariate version of the NBS, functional as well as anatomical measures of connectivity can be combined, for example, by computing Hotelling's T² statistic at each link.

Finally, it is important to make clear that it is difficult to provide definitive rules guiding how to choose the set of suprathreshold links. If the threshold is chosen too low, large components can arise in the permuted data as a matter of chance and thereby reduce power. In contrast, if the threshold is set too high, connections comprising the effect of interest may not be admitted to the set of suprathreshold links. In Section 3, the thresholds used for the NBS were chosen rather arbitrarily; namely, t = 1.5 if the contrast-to-noise ratio was below unity and t = 2 otherwise. It is possible that the performance of the NBS could have been improved in this study by ‘optimizing’ the threshold selection with trial and error. An approach to avoid the problem of threshold selection was recently proposed by Smith & Nichols (2009) in the context of cluster-based statistics.

Conclusion

This paper presented a new approach, called the network-based statistic (NBS), to identify functional or structural connectivity differences in neuroimaging data that is modeled as a network. With the use of receiver operating characteristic (ROC) curves, the NBS was shown to yield substantially greater statistical power than generic procedures for controlling the FWE, as long as any connectivity differences were structured in such a way that they formed connected components. The fact that the NBS can only provide weak control of the family-wise error rate was recognized as its main disadvantage. The utility of the NBS was then demonstrated in the context of a real case-control study involving 12 people with chronic schizophrenia and 15 controls for which resting-state functional MRI data was acquired. The NBS identified a single dysconnected subnetwork in the group with schizophrenia that predominantly comprised fronto-temporal and occipito-temporal dysconnections. We believe the NBS will play an important role in the network analysis of neuroimaging data.