Improving the precision of depression diagnosis in general practice: a cluster-randomized trial

The primary aim was to determine if case-finding with the mandatory use of MDI on clinical suspicion was more accurate than usual clinical assessment in identifying depression in Danish general practice. The hypothesis was that case-finding would be more accurate than usual clinical assessment. Accuracy was estimated based on the positive predictive value (PPV) in the two groups using the MINI International Neuropsychiatric Interview (MINI) [17] as a reference standard. A secondary aim was to evaluate the accordance between depression severity found via MINI and mean MDI sum-scores.

The study was set up as a two-armed, cluster-randomized clinical superiority trial with an intervention group (case-finding) and a control group (usual clinical assessment). It was nested within the Danish Collabri trials investigating collaborative care for depression and anxiety disorders in primary care [18‐20] (see Fig. 1). The Collabri trials refer to the Collabri studies and Collabri Flex studies. These studies investigated a complex intervention with a multi-professional approach to management and treatment of depression and anxiety. The intervention consisted of multiple components such as supervision by a psychiatrist and the introduction of a care manager to collaborate with GPs in providing evidence-based care. The control group in the Collabri studies received treatment as usual (TAU), whereas the GPs providing care for the control group participants in the Collabri Flex studies could also consult a team of mental health specialists.

GPs in the Capital Region of Denmark (except the Island of Bornholm) were invited to participate in the study. GPs were recruited from May 2014 to July 2015, and patients were recruited from November 2014 to July 2018. Cluster-randomization was conducted using a centralized random computer-generated allocation sequence, carried out externally by the Research Centre for Prevention and Health. Each cluster corresponded to a GP provider number. A provider number could include one or more GPs. First, cluster-randomization for collaborative care intervention was performed. This randomization was performed in three rounds. The allocation ratio was 1:1 in the first two rounds and 3:1 (control:intervention) in the third, including four GPs. Clusters in each group were randomized (allocation ratio 1:1) according to the depression detection method, either case-finding or usual clinical assessment. Many GPs from the Collabri studies also participated in the following Collabri Flex studies, where they kept their detection allocation. We planned only to include participants from the Collabri studies; however, due to limited participant intake, the recruitment strategy was changed to also include participants from the Collabri Flex studies.

The GPs who agreed to participate identified potential participants with depression in their practice and referred them to the Collabri trials. This nested study used data from the referral process and the eligibility interviews. Patients were included if the GP had diagnosed depression, if they were 18 years or older, Danish-speaking, had given their written consent and did not meet any exclusion criteria. Only some of the exclusion criteria from the Collabri and Collabri Flex studies [18, 20] pertained to this nested study. Exclusion criteria included pregnancy, a dementia diagnosis, and having an unstable somatic condition as determined by the GP. Additionally, patients of GPs allocated to collaborative care could not participate if they preferred treatment through the publicly subsidized psychologist program. For participants referred from the Collabri studies current/past (within six months) medical/psychological treatment for anxiety or depression and having a pending disability pension application were also exclusion criteria. The in-and exclusion criteria are updated from earlier descriptions [18].

In the usual clinical assessment group, GPs were instructed to diagnose depression as they would normally do [18]. In accordance with National guidelines, GPs are recommended to explore a patient’s condition further using the MDI or ICD-10 criteria if at least two core symptoms of depression, according to ICD-10, are present [21]. In the case-finding group, GPs were instructed to systematically apply the MDI every time they suspected depression and to let the MDI result guide the diagnostics process. Usually, within a week from GP’s referral, a research assistant conducted the MINI interview for DSM IV diagnoses [17] and asked ICD-10 specific questions to validate the ICD-10 diagnosis of depression. Depression severity was assessed using ICD-10. Assessors were blinded for GP allocation and, in the Collabri studies, for the participant’s referral diagnosis. As the study was nested within the Collabri trials, GPs could also refer patients with anxiety disorders who were not eligible for this nested study. Assessors were trained in the MINI interview and received ongoing support and supervision within the research team. If the MINI diagnosis differed from the GP diagnosis, a psychiatrist within the project group would be consulted, and the GP was contacted to determine the result. In case of discrepancy, the GP took the final decision.

The MDI consists of 12 items, each item scored on a six-point Likert scale. Used as a diagnostic tool, two out of three of the top three items corresponding to ICD-I0 depression diagnosis’s core symptoms must be present most of the time or all the time for two weeks. At least two of the remaining seven items (two items have subitems where only the item with the highest score counts) corresponding to the accompanying symptoms of the ICD-I0 depression diagnosis must be present more than half of the time the same period to suggest depression. Hereafter the ICD-10 algorithms establish whether the depression is mild, moderate, or severe [21]. The MDI has been validated in an outpatient sample of 43 participants showing an acceptable sensitivity ranging from 0.86 to 0.92 and a specificity ranging from 0.82 to 0.86 [22], and the tool has been used to determine the presence of depression in clinical settings [23, 24].

The MINI is a short and structured diagnostic interview [17]. It has been validated in relation to the Structured Clinical Interview for DSM-III-Revised Patients (SCID-P) with a kappa-value for major depression of 0.83, a sensitivity of 0.96, a specificity of 0.88, a PPV of 0.87, and a negative predictive value (NPV) of 0.97 [17]; and in regards to the Composite International Diagnostic Interview (CIDI) for International Statistical Classification of Disease (lCD) with a kappa-value for major depression of 0.73, a sensitivity of 0.94, a specificity of 0.79, a PPV of 0.82, and an NPV of 0.93 [25].

The MINI was chosen as a reference standard in the present study. Initially, we investigated whether it could be managed over the telephone. Details and results of this validation are shown in Box 1 in Additional file 1.

Initial sample size calculations showed that a minimum of 480 participants should be included [18]. Based on the study by Mitchell et al., the PPV for usual clinical assessment can be set at 45% [5]. A clinical possible and meaningful increase is assessed to be 60%. Thus, we wanted to detect a difference in PPV of 15%, with an alpha of 0.05, a power of 0.8, a cluster-size of 10, and an ICC of 0.04.

In the primary analyses, the PPV of GP depression diagnosis in the two detection groups was calculated by constructing two-times-two tables. True positives were defined as the number referred by GPs with depression and found with depression via MINI interview. A true positive depression could be accompanied by another psychopathology found via MINI. However, depression, as part of a bipolar disorder, was considered a false positive. We used STATA version Stata/SE 15.1 to calculate confidence intervals (command “bootstrap”) in the primary and secondary analyses and to perform summary statistics in the secondary analyses.

Strengths of this trial were the relatively large sample of participants, the centralized computer-based cluster-randomization, and the use of blinded assessors regarding the allocation. We consider it as a strength of the study that the MDI was used as an assessment tool. The MDI is a measure already used in general practice and recommended in Danish guidelines [21]. In a study by Nielsen et al., the validity of the MDI, compared against the Munich-Composite International Diagnostic Interview (M-CIDI), was investigated in a set up comparable to usual clinical practice in Danish general practice [27]. Authors found that the MDI was a conservative measure of depression compared to the M-CIDI and a valid tool for diagnosing depression when applied to persons who were suspected to have depression [27]. It was a limitation that we used the MINI as standard reference and not SCID or CIDI. However, we used MINI because it is less time-consuming and a widely used tool for validating symptoms and diagnoses. Performed MINI interviews might not have diagnosed all patients with depression, but if the research assistant’s diagnosis was inconsistent with the GP’s diagnosis, the research assistant consulted a psychiatrist in the Collabri group, who contacted the GP to agree on a result. This procedure would, however, not identify false-positive cases if the MINI would diagnose patients with a depression that did not have depression. The identification of participants relied on the GPs’ judgment, and other recruitment strategies such as identification through records or waiting room screenings could have identified a different sample of participants on which the MDI would have been applied. However, the present strategy of GPs identifying participants is closer to everyday clinical practice. We also cannot rule out the possibility that time between tests may have had an impact, as the symptoms may have fluctuated, or medical treatment could have been initiated.

We planned to estimate the sensitivity, specificity, and NPV of the GP’s depression diagnosis in the case-finding group. However, as GPs only referred few persons believed not to have depression after first suspecting one, we consider these estimates as unreliable. Thus, less information about the case-finding method than first anticipated was gained.

It can impact the external validity that some exclusion criteria from the Collabri trials also applied to this nested study. Moreover, the low number of general practices participating (51 of 713 invited) is also a threat to the external validity, as participating GPs could represent those especially interested in collaborative care or depression detection. Further, if both detection groups perform well, it would be more difficult to detect a difference between groups. It is a limitation that we have no baseline information about GPs on factors such as sex, age, and years of practice, as these could have had an impact on their clinical performance. Additionally, there is a risk that the GP sample size is too small to avoid significant baseline differences between GPs, which could affect the detection practice and thereby, the outcome.

The results did not show large problems with false positives in either of the studied groups. In comparison, Mitchell et al. [5] found a PPV of 42.0% (39.6%-44.3%) for the GP depression diagnosis across a sample of 19 studies. In this meta-analysis, depression checklists or questionnaires for GPs were, for example, used to assess the diagnosis. Golden standards were often Structural clinical interview for DSM (SCID), Composite International diagnostic interview (CIDI), and the Diagnostic interview scale (DIS). Christensen et al. found a true positive rate of 60.5% for a group of GPs using case-finding [14]. Still, in our study, 7% of patients in the usual clinical assessment group and 17% of patients in the case-finding group were diagnosed with a depression that was not verified using MINI.

Our findings suggest that usual clinical assessment is more precise than case-finding in this setting. However, more research is needed to support this result. Identification of depression might be improved by integrating the case-finding approach into a stepped care model; however, this should be studied further. Optimally, the MDI should be used for people with at least two of three core symptoms of depression. Thus, it would be relevant to examine the precision of the first three questions in the MDI used as a screening tool before testing with the full MDI.