Automatic identification of atypical clinical fMRI results

Functional MRI (“fMRI”) is one of the most popular and widely used brain activation measurement tools in cognitive neuroscience. fMRI is a non-invasive imaging modality with a spatial resolution that is high compared with other non-invasive functional imaging methods, such as EEG. It is also relatively easily accessible due to the wide availability of MRI scanners, particularly in clinical centers. These characteristics are important reasons why the development of fMRI has had a strong impact on neuroscience.

It is also considered to carry a strong potential for clinical applications. Clinical applications of fMRI have mostly centered on presurgical use for patients with a brain tumor and patients with epilepsy. In both fields, several reviews have concluded that fMRI can provide important information for clinical care [1‐5]. For instance, several studies have indicated that for language dominance, fMRI shows good agreement with invasive clinical measures such as the Wada test [6‐9]. It has also been shown to be helpful in providing information about brain function topography prior to surgery for brain tumor patients [3, 10‐16] and epilepsy patients [17], although some limitations have also been described [18].

One important issue limiting the use of fMRI in a clinical setting is the quality of the results which can vary considerably due to the complex process of fMRI acquisition. It is far from straightforward to determine, even for experts, whether the quality of an fMRI scan has been compromised by movement, task non-compliance, or other disruptions during acquisition. Additionally, brain activation patterns from neurological patients can deviate strongly from what is expected in location, extent, and magnitude of activated brain areas due to the neurological disorder [3, 19]. This could further complicate the problem of detecting invalid scans in a clinical setting. While certain measures indicating the quality of a scan are currently available, there is no general expert agreement on their value for accepting or rejecting a clinical fMRI activity map. The effect of some other factors, such as task compliance, can currently only be determined by expert evaluation. Thus, the determination of the quality of an fMRI scan currently requires expertise that is not readily available in clinical settings. A fast, automated, and objective first selection method for identifying atypical fMRI scans would greatly facilitate clinical use of fMRI.

In this report, we introduce a method that can provide a fast, automatic, and objective determination of the atypicality of an activation pattern. The approach is based on the notion that the performance of an fMRI task affects the whole brain, rather than only those parts that exceed a statistical threshold. We exploit this factor in our proposed method to determine an objective value describing the atypicality of an activation pattern. This value is based on a whole brain voxel by voxel correlation of each individual activity map with an independent template activity (“template matching analysis” or “TMA”; see Fig. 1).

In order to test the validity of the method, we analyzed fMRI from healthy controls collected at seven clinical centers in Europe with two standardized fMRI tasks to map hand motor and language function. Participating centers were all members of the European Low Grade Glioma Network (www.​braintumours.​eu). The tasks have previously been described for presurgical localization of motor [11, 20, 21] and language function [22, 23]. Additionally, we used this dataset to artificially create atypical activity patterns due to incorrect analysis, incorrect orientation, or incorrect task execution. This set was used to test the feasibility of our method to automatically identify scans with atypical activation patterns.

We also applied our method on a series of 32 clinical brain tumor verb generation scans from consecutive patients considered for surgery at the UMC Utrecht. The main goal of including the patient data in the current manuscript was to provide proof of principle for our presented method, specifically to test if the presence of a tumor would not affect the full brain activation patterns in such a severe manner that all patterns would become atypical and thus invalidate our approach.

The main goal of this study was to evaluate a novel method for fast, objective, and automatic detection of atypical fMRI scans based on the activation patterns. The method is based on a template matching analysis (TMA). Activation patterns were collected from seven European clinical centers and two commonly used clinical fMRI tasks for localization of motor and language, and from patients with a brain tumor from the UMC Utrecht.

Results indicated that fast and automatic detection of atypical activation patterns appears to be a reachable goal in healthy subjects. For localization of motor activity, a detection rate of 93.5% of compromised scans could be achieved using the TMA approach, while for location of language activation, the VG protocol, a detection rate of 96.4% was achieved. The analysis of patient VG data suggests the feasibility of TMA in a clinical setting. Patients with a brain tumor did show slightly lower template matching values on average, which is likely due to the inclusion of poor-quality data due to movement and cognitive impairments, and anatomical and functional abnormalities related to the tumor. None of these confounders was considered for the purpose of this study, in order to obtain a conservative indication of the robustness of the TMA method.

Automatic detection of atypical activation patterns may prove helpful in clinical fMRI settings. While scans with a high TMA value can be forwarded for clinical use without need for expert quality assessment, experts can be consulted in cases where the scan has a low TMA value in order to determine the cause, such as movement, failure to comply with task rules by the patient or suboptimal analysis, versus abnormal activity due to severe anatomical deformation or functional plasticity. Thus, TMA may improve the quality of care by providing an objective and reproducible assessment of the fMRI activation pattern, and it can increase the cost efficiency by reducing the need for fMRI experts to analyze and interpret each fMRI dataset.

An alternative use of our method can be found in applications for large databases: TMA can automatically, at low cost and minimal manpower, identify datasets that are potentially compromised, thereby rapidly indicating potential site-specific problems and improving the quality of the database. This could facilitate studies that aim at gathering large databases over longer time or studies that need to pool fMRI data from various centers with potential systematic differences in quality.

It can be expected that the presence of a brain tumor changes the anatomy of a brain due to its effects on tissue integrity and displacement, as well as the functional anatomy, as it may disrupt functional networks. While the patients did display lower TMA values than healthy controls, the values were significantly higher than those of the atypical healthy subject scans (Fig. 5). A lower mean TMA value is to be expected since the set is likely to include poor-quality data (due to for instance movement, non-compliance, compromised cognitive function) as well as functional and large anatomical changes caused by the tumor.

One possible explanation for the limited tumor effects on TMA is that the protocols that were used evoked a pattern of activations and deactivations that are widespread over the brain. Thus, although the anatomy as well as functional activity pattern may change in the vicinity of the tumor, the activity pattern across the whole scan volume appears to be more stable and not severely affected by the tumor, resulting in only a somewhat lower than normal value for atypicality for most patients. Of note, task compliance and movement were not taken into account in order to obtain a conservative indication of robustness of TMA.

An efficient and reliable language protocol is a particularly important tool for presurgical planning as well as research [17, 28‐31]. Clinical fMRI can locate the presence of crucial language activation in the immediate vicinity of a tumor. This would mandate a cautionary surgical approach and can be a reason to perform awake surgery or even advise against resective surgery. Importantly, fMRI may fail at the edge or within the tumor, due to abnormal vascular properties and brain tissue which can reduce the fMRI signal [32], and as such can only be used in conjunction with other clinical diagnostic modalities.

One of the important and possibly counterintuitive findings of this study is that the language protocol performed comparable with the motor protocol and showed robust results, despite the more complex nature of language production compared with motor performance and execution in four languages. This result suggests that the pattern of activity evoked by language production is as consistent over subjects as that of motor production.

Our study also demonstrated that fMRI patterns do not differ strongly between centers. Previous multisite studies, applying simple motor protocols [33‐38], visual protocols [39, 40], and sometimes cognitive protocols [41‐43], have consistently reported promising results for the possibility to combine fMRI data across centers. Site variation typically appears to be much smaller than subject variation (see for instance Costafreda [44] for a review). However, previous multisite studies have predominantly evaluated location or strength of activation at a specific predefined area that was expected to be activated by the protocol. TMA indicates that whole-brain activation patterns associated with language as well as motor function are consistent over centers with a considerable range in hardware and scan protocols (Table 1).

The fMRI tasks were chosen based on extensive use across centers and validation against direct electrical stimulation in brain surgery [45, 46] and electrocortical mapping [47, 48]. The tasks are easy to understand and perform for most surgery patients and yield robust levels of activation. Apart from simplicity to promote patient compliance, characteristics that may be important in robustness include a block design (as opposed to a less sensitive event-related design [49]) and non-ambiguity of the task to avoid use of different cognitive strategies. Yet, patients should be tested before an fMRI scan to ascertain cognitive capability and ability to sustain attention.

The robustness of a standardized protocol can be applied to improve reliability of clinical fMRI, facilitate analysis, and interpretation of results. The quality of a fMRI scan is vulnerable and can be affected by many factors, such as subject movement, task compliance, and incorrect analysis steps. In many cases, it is difficult to objectively judge the quality of an fMRI scan, other than through visual inspection of the activity pattern, and as of yet, there is no commonly accepted metric for quality. The current results indicate that TMA provides a simple but effective metric that can be evaluated in future studies against expert quality assessment.

Robust protocols can also facilitate studies that are plagued by low power because of difficulties in recruitment due to for example low incidence rates of specific pathologies (e.g., low-grade glioma at a particular location) or surgical procedures [50]. It allows for the possibility to pool data from various centers for clinical fMRI studies, without substantial loss in statistical power. This approach can for instance be used for patient follow-up studies that aim at examining the association between surgery and patient outcome, especially in terms of the presence of functional plasticity, or for cohort studies, where large populations are required.

There are several limitations that have to be considered in interpreting the results of this study. Importantly, while the patient results do successfully indicate that presence of a tumor does not necessarily have an invalidating impact on TMA, it will take more patient data and further expert evaluation in order to determine a firm TMA threshold for clinical use.

Our study did not evaluate the relevance of the information acquired by the task protocols but was limited to the reproducibility of the pattern across individuals. Our study did not evaluate differences in the level of the activity between centers, but instead focused on agreement between patterns. Possibly, there may have been differences in levels of activity between centers due to hardware and scan sequence differences. However, for clinical use, we do believe that the pattern of activity may be more informative than absolute levels.

Another limitation is that patients were not evaluated for quality of the scans, leaving the question about an appropriate TMA threshold for clinical use open. In the patient dataset used here, it is not known how many were of poor quality or how many exhibited lack of compliance due to cognitive or attentional deficits. This requires an extensive study comparing TMA in patient data with multiple quality measures including subjective evaluations by experts which did not fall within the scope of this proof-of-principle study. Moreover, we did not apply correction for abnormalities in anatomical structure caused by the tumor. But based on the current results, techniques that are able to correct for those, for instance, by non-linear “warping” techniques for normalizing a patient anatomical scan to a standard brain [51], are likely to further improve robustness of TMA. Finally, future studies as well as applications need to indicate if TMA can also be used to detect more subtle forms of atypical activation, for instance, if they are associated with poor task performance due to cognitive deficits.

As a final note, one could argue that head motion could increase the TMA value by generating false activations. We argue that motion artifacts can only elevate TMA if motion causes increases and decreases in various regions in such a way that they match the whole-brain template. Given the nature of data acquisition, this is highly unlikely since motion affects whole slices (and generally multiple) and is most often manifested along intensity edges (typically gray matter vs CSF) across the brain. Neither is likely to generate the template pattern. In effect, even though task-correlated motion may generate spurious significant activity along intensity edges, motion will reduce the TMA that is computed across the brain because it adds the image noise resulting from motion to the data.

In conclusion, we present a straightforward method for assessing typicality of fMRI activation patterns and validate it with correctly and incorrectly analyzed data from 42 healthy subjects from 7 centers. The results indicate that TMA is robust and detects the atypical activity patterns derived from the incorrectly analyzed quite well. The application of TMA to a series of patient data suggests that the presence of localized anatomic abnormalities does not invalidate the method (although TMA values were lower than that of healthy subjects). Of note, it is of importance for clinical fMRI to optimize the scan procedure in order to avoid artifacts, preferably by applying some sort of quality assurance.