Elsevier

NeuroImage

Volume 44, Issue 1, 1 January 2009, Pages 62-70
NeuroImage

Technical Note
False discovery rate revisited: FDR and topological inference using Gaussian random fields

https://doi.org/10.1016/j.neuroimage.2008.05.021Get rights and content

Abstract

In this note, we revisit earlier work on false discovery rate (FDR) and evaluate it in relation to topological inference in statistical parametric mapping. We note that controlling the false discovery rate of voxels is not equivalent to controlling the false discovery rate of activations. This is a problem that is unique to inference on images, in which the underlying signal is continuous (i.e., signal which does not have a compact support). In brief, inference based on conventional voxel-wise FDR procedures is not appropriate for inferences on the topological features of a statistical parametric map (SPM), such as peaks or regions of activation. We describe the nature of the problem, illustrate it with some examples and suggest a simple solution based on controlling the false discovery rate of connected excursion sets within an SPM, characterised by their volume.

Introduction

In this note, we consider the detection of distributed signals in image data, using statistical parametric mapping (SPM). The notion of a distributed signal is critical, in that it forces us to consider signal (as well as noise) as spatially continuous without compact support. Examples of distributed signals include induced responses in EEG studies that are distributed over frequency and peristimulus time or hemodynamic responses in fMRI that are mediated by molecules that diffuse rapidly over space. When signal is distributed, one might intuitively define the signal at a spatial location (e.g., voxel) as the value of the signal process at that location. However, voxel-wise approach this leads to several problems. A priori, all points have signal (see Table 1), so it is illogical to examine a null hypothesis of no signal. This compels us to define treatment effect or activation as signal above some ad hoc threshold. Second, the multiple-comparison problem becomes severe (with thousands of voxels contributing to family-wise error). These considerations lead to the notion of an activation that is defined in terms of the signal's topological features (e.g., maxima, spatial extent etc). This converts a continuous signal into a discrete set of features, whose statistics can be examined in the usual way. The notion of a topological response finesses the interpretation of inference and allows for rigorous control of a smaller multiple comparison problem (Friston et al., 1991, Worsley et al., 1992). Under the topological perspective, a response or activation is an attribute of the signal profile over voxels; it is therefore a category error1 to call a voxel activated. For example, we refer to a peak in a SPM as “an activation” not a collection of activations at voxels subtending the peak. The implications of this category error can be quite profound, because it permits images to be treated collections of discrete voxels or statistical tests that do not consider the continuity constraints under which the data were generated. In this work, we look at FDR procedures from the topological perspective. In particular, we show that, in the context of smooth distributed signals, conventional FDR procedures do not control the FDR of either voxels or topological features. The purpose of this note is to promote discussion of current public-software implementations of voxel-wise FDR and their usefulness.

False discovery rate procedures were introduced to neuroimaging by Genovese et al. (2002). Since their introduction, they have enjoyed considerable use. Controlling false discovery rate (FDR) provides a more sensitive analysis than the conventional control of family-wise error. This is particularly important for neuroimaging, which faces a severe and rather complicated multiple comparison problem. However, the problem faced by conventional FDR procedures is that they regard SPMs as a collection of discrete tests. This is in contrast to random field theory approaches, which consider an SPM to be a lattice approximation to an underlying continuous process. This distinction is not trivial. Inference, using random field theory, is about topological features of the SPM, such as the number of maxima or regions, their spatial extent or their peak height. On the other hand, inference using false discovery rate treats each voxel as a separate feature. This can lead to the following problem, which is best illustrated with an example:

  • Imagine that we declare a hundred voxels significant using an FDR criterion. 95 of these voxels constitute a single region that is truly active. The remaining five voxels are false discoveries and are dispersed randomly over the search space. In this example, the false discovery rate of voxels conforms to its expectation of 5%. However, the false discovery rate in terms of regional activations is over 80%. This is because we have discovered six activations but only one is a true activation. This is a contrived example but illustrates nicely the problem we want to address.

In brief, when we make an inference using SPM it is about a topological feature e.g. inflection points, or clusters above a threshold. It is not about each voxel in that cluster (or more formally the excursion set). This is why one only reports the cluster, usually in terms of its maximum value and location. Conventional family-wise procedures (e.g., Bonferroni correction) cannot support this sort of inference because they have no notion of topology. In other words, the fact that two voxels are part of the same cluster is incidental to both inference and the way the results are reported. This limits the usefulness of procedures like the Bonferroni correction and FDR in imaging and was the motivation for random field theory approaches to topological inference based on differential topology (Friston et al., 1991, Worsley et al., 1992). In short, the Bonferroni correction controls the false positive rate of voxels, whereas SPM controls the false positive rate of features. Conventional FDR procedures control the false discovery rate of voxels, whereas they should be controlling the false discovery rate of features.

This problem with FDR is articulated nicely by Heller et al. (2006); “Recognizing that the fundamental units of interest are the spatially contiguous clusters of voxels that are activated together, we set out to approximate these cluster units from the data by a clustering algorithm especially tailored for fMRI data” (see also Pacifico et al., 2004, Benjamini and Heller, 2007). We pursue the same theme but using a simple approach and standard results from random field theory.

This paper comprises two sections. The first presents the theoretical background to conventional inference in neuroimaging, false discovery rate and a quantitative illustration of the problem introduced above. We then consider alternative formulations of FDR based on the topology of excursion sets. The second section provides some worked examples and evaluates the procedures using simulated and real data.

Section snippets

Theory

This section examines a commonplace procedure: voxel-wise FDR on smooth data. Our purpose is to show that the implicit assumptions about signal and noise may be untenable. Regarding the former, voxel-wise FDR on data with continuous signal is strictly illogical2

FDR based on spatial volume

One simple solution to the problem highlighted above is to control the false discovery rate of topological features as opposed to voxels. In other words, apply the FDR procedure to the null distribution of features such as cluster-volume or peak height. In this work, we focus on FDR for cluster-volume. As stated in Genovese et al. (2002) any valid statistical test with a known null distribution can be subject to FDR control. In this section, we use this procedure with simulated and real data to

Discussion

In this work we have revisited the use of FDR for topological inference on neuroimages. We have shown that there are fundamental problems with the interpretation of voxel-wise FDR. Firstly, because it contains no inherent representation of the spatial structure of signal, voxel-wise FDR cannot control the false discovery rate of regional effects. One consequence is that the regional FDR arising from a voxel-wise FDR analysis may be intolerably large (see Fig. 2). Second, in practice, the use of

Acknowledgments

This work was funded by CoMPLEX, UCL and the Wellcome trust. We thank Emily for her help in preparing this manuscript and our reviewers for their help and guidance is addressing these issues.

References (28)

  • WorsleyK.J. et al.

    Unified univariate and multivariate random field theory

    NeuroImage

    (2004)
  • BailletS. et al.

    A Bayesian approach to introducing anatomo-functional priors in the EEG/MEG inverse problem

    IEEE Trans. Biomed. Eng.

    (1997)
  • BenjaminiY. et al.

    Controlling the false discovery rate—a practical and powerful approach to multiple testing

    J. R. Stat. Soc., B. Methodol.

    (1995)
  • BenjaminiY. et al.

    False discovery rates for spatial signals

    J. Am. Stat. Assoc.

    (2007)
  • Cited by (378)

    View all citing articles on Scopus
    View full text