Technical NoteFalse discovery rate revisited: FDR and topological inference using Gaussian random fields
Introduction
In this note, we consider the detection of distributed signals in image data, using statistical parametric mapping (SPM). The notion of a distributed signal is critical, in that it forces us to consider signal (as well as noise) as spatially continuous without compact support. Examples of distributed signals include induced responses in EEG studies that are distributed over frequency and peristimulus time or hemodynamic responses in fMRI that are mediated by molecules that diffuse rapidly over space. When signal is distributed, one might intuitively define the signal at a spatial location (e.g., voxel) as the value of the signal process at that location. However, voxel-wise approach this leads to several problems. A priori, all points have signal (see Table 1), so it is illogical to examine a null hypothesis of no signal. This compels us to define treatment effect or activation as signal above some ad hoc threshold. Second, the multiple-comparison problem becomes severe (with thousands of voxels contributing to family-wise error). These considerations lead to the notion of an activation that is defined in terms of the signal's topological features (e.g., maxima, spatial extent etc). This converts a continuous signal into a discrete set of features, whose statistics can be examined in the usual way. The notion of a topological response finesses the interpretation of inference and allows for rigorous control of a smaller multiple comparison problem (Friston et al., 1991, Worsley et al., 1992). Under the topological perspective, a response or activation is an attribute of the signal profile over voxels; it is therefore a category error1 to call a voxel activated. For example, we refer to a peak in a SPM as “an activation” not a collection of activations at voxels subtending the peak. The implications of this category error can be quite profound, because it permits images to be treated collections of discrete voxels or statistical tests that do not consider the continuity constraints under which the data were generated. In this work, we look at FDR procedures from the topological perspective. In particular, we show that, in the context of smooth distributed signals, conventional FDR procedures do not control the FDR of either voxels or topological features. The purpose of this note is to promote discussion of current public-software implementations of voxel-wise FDR and their usefulness.
False discovery rate procedures were introduced to neuroimaging by Genovese et al. (2002). Since their introduction, they have enjoyed considerable use. Controlling false discovery rate (FDR) provides a more sensitive analysis than the conventional control of family-wise error. This is particularly important for neuroimaging, which faces a severe and rather complicated multiple comparison problem. However, the problem faced by conventional FDR procedures is that they regard SPMs as a collection of discrete tests. This is in contrast to random field theory approaches, which consider an SPM to be a lattice approximation to an underlying continuous process. This distinction is not trivial. Inference, using random field theory, is about topological features of the SPM, such as the number of maxima or regions, their spatial extent or their peak height. On the other hand, inference using false discovery rate treats each voxel as a separate feature. This can lead to the following problem, which is best illustrated with an example:
- ●
Imagine that we declare a hundred voxels significant using an FDR criterion. 95 of these voxels constitute a single region that is truly active. The remaining five voxels are false discoveries and are dispersed randomly over the search space. In this example, the false discovery rate of voxels conforms to its expectation of 5%. However, the false discovery rate in terms of regional activations is over 80%. This is because we have discovered six activations but only one is a true activation. This is a contrived example but illustrates nicely the problem we want to address.
In brief, when we make an inference using SPM it is about a topological feature e.g. inflection points, or clusters above a threshold. It is not about each voxel in that cluster (or more formally the excursion set). This is why one only reports the cluster, usually in terms of its maximum value and location. Conventional family-wise procedures (e.g., Bonferroni correction) cannot support this sort of inference because they have no notion of topology. In other words, the fact that two voxels are part of the same cluster is incidental to both inference and the way the results are reported. This limits the usefulness of procedures like the Bonferroni correction and FDR in imaging and was the motivation for random field theory approaches to topological inference based on differential topology (Friston et al., 1991, Worsley et al., 1992). In short, the Bonferroni correction controls the false positive rate of voxels, whereas SPM controls the false positive rate of features. Conventional FDR procedures control the false discovery rate of voxels, whereas they should be controlling the false discovery rate of features.
This problem with FDR is articulated nicely by Heller et al. (2006); “Recognizing that the fundamental units of interest are the spatially contiguous clusters of voxels that are activated together, we set out to approximate these cluster units from the data by a clustering algorithm especially tailored for fMRI data” (see also Pacifico et al., 2004, Benjamini and Heller, 2007). We pursue the same theme but using a simple approach and standard results from random field theory.
This paper comprises two sections. The first presents the theoretical background to conventional inference in neuroimaging, false discovery rate and a quantitative illustration of the problem introduced above. We then consider alternative formulations of FDR based on the topology of excursion sets. The second section provides some worked examples and evaluates the procedures using simulated and real data.
Section snippets
Theory
This section examines a commonplace procedure: voxel-wise FDR on smooth data. Our purpose is to show that the implicit assumptions about signal and noise may be untenable. Regarding the former, voxel-wise FDR on data with continuous signal is strictly illogical2
FDR based on spatial volume
One simple solution to the problem highlighted above is to control the false discovery rate of topological features as opposed to voxels. In other words, apply the FDR procedure to the null distribution of features such as cluster-volume or peak height. In this work, we focus on FDR for cluster-volume. As stated in Genovese et al. (2002) any valid statistical test with a known null distribution can be subject to FDR control. In this section, we use this procedure with simulated and real data to
Discussion
In this work we have revisited the use of FDR for topological inference on neuroimages. We have shown that there are fundamental problems with the interpretation of voxel-wise FDR. Firstly, because it contains no inherent representation of the spatial structure of signal, voxel-wise FDR cannot control the false discovery rate of regional effects. One consequence is that the regional FDR arising from a voxel-wise FDR analysis may be intolerably large (see Fig. 2). Second, in practice, the use of
Acknowledgments
This work was funded by CoMPLEX, UCL and the Wellcome trust. We thank Emily for her help in preparing this manuscript and our reviewers for their help and guidance is addressing these issues.
References (28)
- et al.
Voxel-based morphometry—the methods
NeuroImage
(2000) - et al.
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
NeuroImage
(2002) - et al.
Diffusion-based spatial priors for imaging
NeuroImage
(2007) - et al.
Cluster-based analysis of FMRI data
NeuroImage
(2006) - et al.
A derivation of a macroscopic field theory of the brain from the quasi-microscopic neural dynamics
Physica, D
(1997) - et al.
Robust smoothness estimation in statistical parametric maps using standardized residuals from the General Linear Model
NeuroImage
(1999) - et al.
Applications of random field theory to electrophysiology
Neurosci. Lett.
(2005) The brain wave equation: a model for the EEG
Math. Biosci.
(1974)- et al.
MEG source localization under multiple constraints: an extended Bayesian framework
NeuroImage
(2006) An improved theoretical P value for SPMs based on discrete local maxima
NeuroImage
(2005)
Unified univariate and multivariate random field theory
NeuroImage
A Bayesian approach to introducing anatomo-functional priors in the EEG/MEG inverse problem
IEEE Trans. Biomed. Eng.
Controlling the false discovery rate—a practical and powerful approach to multiple testing
J. R. Stat. Soc., B. Methodol.
False discovery rates for spatial signals
J. Am. Stat. Assoc.
Cited by (378)
Differential gene expression analysis pipelines and bioinformatic tools for the identification of specific biomarkers: A review
2024, Computational and Structural Biotechnology JournalThe brain correlates of hostile attribution bias and their relation to the displaced aggression
2022, Journal of Affective DisordersAssociations between sex, body mass index and the individual microglial response in Alzheimer’s disease
2024, Journal of NeuroinflammationSpatiotemporal Correlation between Amyloid and Tau Accumulations Underlies Cognitive Changes in Aging
2024, Journal of Neuroscience