Validation of the Dutch version of the quick mild cognitive impairment screen (Qmci-D)

The prevalence of mild cognitive impairment (MCI) [1] and dementia [2] is increasing worldwide. Differentiating MCI from dementia is important, because treatment options differ. In particular, pharmaceutical therapy, indicated for treatment of dementia, is inappropriate and potentially even harmful, in those with MCI [3]. Differentiating MCI from normal cognition (NC) is also important, because people with MCI are at increased risk of developing dementia, compared to aged-matched individuals in the population [4], and early identification may allow prompt intervention [5, 6]. Few short (administration time of approximately 5 min) cognitive screening instruments are useful in discriminating between MCI and NC or dementia. Most are limited by their sensitivity and specificity [7]. Likewise, few are available in the Dutch language. One of the most widely used tools is the Mini-Mental State Examination (MMSE) [8]. The MMSE was standardized to improve reliability, producing the Standardised Mini-Mental State Examination (SMMSE) [9, 10], which is available in a wide variety of languages including Dutch [11]. However, there is limited evidence that either the MMSE or SMMSE are sufficiently accurate in identifying MCI [12], particularly in those with high educational attainment [13].

To address these challenges, the Quick Mild Cognitive Impairment (Qmci) screen was developed. Based upon the ABCS 135 [14], it was modified to increase its sensitivity to differentiate NC from MCI, by the addition of logical memory. The Qmci is more sensitive in differentiating MCI from NC than the SMMSE and ABCS 135 [15]. The Qmci has six subtests, orientation, registration, clock drawing, delayed recall, verbal fluency and logical memory. It is scored out of 100 points, can be administered and scored in less than 5 min and has excellent test-retest reliability [16]. The Qmci correlates highly with the Standardised ADAS-cog, Clinical Dementia Rating scale and the Lawton-Brody activities of daily living scale [17].

The goal of the present study was to adapt the Qmci for use in Dutch-language countries, to validate the Dutch version of the Qmci (Qmci-D) and to compare its sensitivity and specificity in differentiating MCI from NC and dementia to the most widely used short cognitive screen in the Netherlands, the Dutch version of the SMMSE (SMMSE-D).

In total, 90 participants, 41 males (46 %) and 49 females (54 %), were included in this study. In this sample, 35 (39 %) had NC, 30 (33 %) had MCI and 25 (28 %) were diagnosed with dementia. The overall mean age of the sample was 72.9, standard deviation (SD) of 9.1 years. The NC group (mean age 68.7) was younger than the MCI (mean age 79.1) and dementia (mean age 79.2) groups (p < 0.001). The NC group had a mean Qmci-D score of 64 (SD 10.5) and a mean SMMSE-D score of 28 (SD 1.8). The mean Qmci-D score for the MCI group was 46 (SD 11.8) compared with 24 (SD 2.9) for the SMMSE-D, while the dementia group scored 34 (SD 15.8) for the Qmci-D and 22 (SD 4.4) for the SMMSE-D. These scores and demographic data are summarized in Table 1.

One-way ANCOVA, used to test for differences in SMMSE-D scores between the three groups (NC, MCI and dementia), showed that the mean scores differed significantly across the three groups (F = 20.55, df = 3,86, p < 0.001). Post hoc pair-wise comparisons using the Tukey’s HSD test showed significant mean differences between groups (p < 0.05). The mean scores for the Qmci-D also differed significantly between the three groups; (F = 33.24, df = 3,86, p < 0.001). The differences between groups are presented as boxplots in Fig. 1. ANOVA post-hoc testing for multiple comparisons, showed a significant difference in mean scores between the dementia and NC groups, for both tests (see Table 2).

Comparisons of the AUC of ROC curves and the point estimates (cut-off scores), providing the optimal sensitivity and specificity, are presented in Fig. 2. These show that the Qmci-D was more accurate than the SMMSE-D in discriminating between dementia and NC, with an AUC of 0.95 (95 % CI 0.90–0.99), compared to 0.89 (95 % CI 0.80–0.96) for the SMMSE-D (see Fig. 2a, b). The difference was significant (p = 0.006). Both the Qmci-D and the SMMSE-D discriminated MCI from NC, with ANOVA post-hoc tests showing a significant mean difference between the MCI and NC groups (p < 0.001). The AUC of the Qmci-D, in discriminating MCI from NC, was marginally greater at 0.86 (95 % CI 0.77–0.95), compared to 0.84 (95 % CI 0.74–0.94) for the SMMSE-D (see Fig. 2c, d). This difference was non-significant (p = 0.335). At the point estimate the Qmci-D had a sensitivity of 70 % and specificity of 94 % compared to a sensitivity of only 60 % and a similar specificity of 94 % for the SMMSE-D. As for the discrimination of MCI from dementia, ANOVA post-hoc tests, showed a significant mean difference (p < 0.001) between these participants with the Qmci-D. The difference between scores for the MCI and dementia groups, for the SMMSE-D (p = 1.02) was not significant. The score test for homogeneity of variances across groups in ANCOVA indicated that homogeneity for the Qmci is rejected. Heteroscedasticity-corrected SEs and Tests after ANCOVA [31] indicated that the difference in Qmci-D mean scores after correction for age, between MCI and NC as well as between MCI and dementia, were significantly different from zero. The ability of the Qmci-D, to discriminate MCI from dementia, was significantly greater (p = 0.024) than the SMMSE-D, AUC of 0.73 (95 % CI 0.59–0.85) compared to 0.60 (95 % CI 0.45–0.75), respectively (See Fig. 2e, f). At the point estimate the Qmci-D had a modest sensitivity of 64 % to differentiate MCI from dementia, although this compared favorably to the SMMSE-D with a sensitivity of only 28 %.

When moderate and severe dementia cases were removed from analysis, the AUC of the Qmci-D and SMMSE-D for differentiating MCI from mild dementia cases alone was unchanged at 0.62 and 0.54, p = 0.03, respectively.

The goals of this study were to adapt the Qmci for use in Dutch-language countries and to explore its concurrent validity against the most commonly used short cognitive screen in Dutch, the SMMSE-D. The results show that the Qmci-D is more accurate than the SMMSE-D in differentiating dementia from NC. It had only fair accuracy (AUC 0.73) at differentiating MCI from dementia, although it was significantly more accurate than the SMMSE (AUC 0.60). In this study however, the SMMSE-D wasn’t able to discriminate MCI from dementia with a very poor sensitivity of 24 %, particularly when moderate to severe cases were excluded. Based upon this data it would appear that both instruments have limited ability to separate MCI from dementia. This is markedly different from the initial validation of the English language version of the Qmci against the SMMSE [15], which in a much larger sample of almost 1000 Canadians, suggested that both had excellent accuracy (AUC >0.90), although it showed no significant difference between the two instruments in their ability to distinguish MCI from dementia. There was also no significant difference between the tests’ ability to discriminate between MCI and NC in this sample, although the accuracy of both tests was good, suggesting that both were able to separate MCI from NC. This again differs from the initial validation, where the Qmci showed significantly greater accuracy over the SMMSE. This discrepancy may relate to the small sample size, suggesting that this study was underpowered to show superiority of one instrument over the other. This said, the goal of this study was not to show superiority of a Dutch language version of the Qmci, the Qmci-D, rather it was to show the concurrent validity of the translation against a widely used screening instrument.

The strength of this study is the robust analysis with bootstrapped ROC curves and 95 % confidence intervals, to identify the discriminatory characteristics of both screening tools. This method provides more accurate results than non-bootstrapped methods, especially when analyzing smaller sample sizes. The 95 % confidence intervals obtained from the bootstrap and the asymptotic approach [30], were in all cases virtually equal. This indicates that the intervals are valid.

The study has limitations. First, the diagnosis of MCI was based on clinical criteria, which may have increased the heterogeneity of this group and led to some bias. However, no single gold standard criterion for MCI exists and there is still a lack of uniformity in the clinical diagnosis of MCI between studies [32]. This said, in this study an objective history of cognitive decline over time was obtained from each patient’s collateral (family member or caregiver) and assessed by neuropsychological testing, independent of the results of the short cognitive screens, in keeping with the National Institute on Aging Alzheimer’s Association diagnostic guidelines [21]. Second, the NC group consisted of participants recruited by convenience sampling from healthy relatives or caregivers attending with patients. These participants were significantly younger than patients with MCI or dementia. This could have increased heterogeneity and created bias, explaining why there was no significant difference in the ability of both instruments to distinguish MCI from NC, unlike that seen in the initial validation of the Qmci [15]. Patients with MCI and dementia were, however, well matched for age and gender. ANCOVA testing and post-hoc analysis confirmed that differences in mean test score were not attributable to age. Furthermore, the educational status of patients was not recorded routinely, which may also have created some bias. Third, the sample size was small and did not exceed the desired 76-paired observations, calculated as the sample size to detect differences in accuracy between participants with MCI and NC for the screening tools. Fourth, the study excluded those with active depression and less prevalent dementia subtypes as described above. Active depression was excluded as these patients may have slower reaction times and processing speeds [33]. Frontotemporal, Parkinson’s disease and Lewy body dementia often present with exaggerated functional deficits potentially clouding the diagnosis of MCI, the focus of this study. This may have caused the sample to be less representative and created some spectrum bias, limiting the generalizability of the results. Finally, the study compared the Qmci-D only with the SMMSE-D. This was because the SMMSE is the most widely used short cognitive screen [34] and in the initial validation of the English language version of the Qmci the comparator was the SMMSE, allowing direct comparison with the results of that study [15]. The authors acknowledge the importance of future validation against other short, albeit longer, screens including the Dutch version of the Montreal cognitive assessment (MoCA) [35, 36], the Addenbrooke’s Cognitive Examination-Revised [37] and shorter instruments like the Mini-Addenbrooke’s Cognitive Examination (M-ACE) [38]. The authors also caution that screening for cognitive impairment continues to have challenges and in clinical practice [39] it remains only one part of a comprehensive assessment of cognition, and should not be relied upon exclusively.

In conclusion, this study shows the concurrent validity of the Qmci-D against the SMMSE-D. The data suggests that the Qmci-D, although statistically significantly more accurate than the SMMSE-D, is limited in its ability to differentiate MCI from dementia. The results also suggest that the accuracy of both instruments at distinguishing MCI from NC was good although the Qmci was more accurate than the SMMSE in separating dementia from NC in a Dutch speaking population. Given this, albeit limited analysis in a small sample, as well as its brevity and ease of administration [15‐17, 40], the Dutch version of the Qmci, the Qmci-D, appears useful as a short cognitive screen. Further research is now required to confirm these findings with a larger sample including other dementia subtypes and to compare the test to other cognitive screens including the MoCA and M-ACE.