Psychometric properties of the PDS
The diagnosis based on the clinical interview concurred with the PDS result in 69 cases (
N = 90, 77%) as Table
2 illustrates. As shown in Table
2, sensitivity of the PDS was 85%, with 80% accurate positive screening outcomes (
positive predictive value). Specificity was 63%. The resulting Youden-Index was
J = .48. There were 16% false-negative and 38% false-positive results. Of the patients with a negative PDS, 69% did in fact not have a diagnosis of PDD (
negative predictive value). The resulting positive likelihood ratio was 2.3, meaning it was 2.3 times more likely that a subject with PDD had a positive PDS than subjects without PDD having a positive PDS. The negative likelihood ratio was .24, meaning it was 4.2 times more likely that a subject without PDD had a negative PDS compared to subjects with PDD having a negative PDS. Jaeschke et al. (1994) regard values of these magnitudes as
small, but sometimes important [
40].
Table 2
Agreement between the diagnoses from the interview and the PDS
likely PDD diagnosis based on PDS result | present | 49 (85%) | 12 (38%) | 61 (68%) |
not present | 9 (16%) | 20 (63%) | 29 (32%) |
total | 58 | 32 | 90 |
Cohen’s κ was .48 ([95%-CI .28, .68],
p < .001,
SE = 0.10). The strength of agreement can hence be considered
moderate with a range from
fair to
substantial [
41]. Bias between PDS results and clinical interviews was negligible for the value of κ (
BI = .03). The prevalence effect was moderate (
PI = .32). This moderate prevalence effect implies that
Cohen’s κ might be an underestimation of the agreement between the PDS and the clinical interview. We therefore calculated the prevalence-adjusted bias-adjusted Kappa (
PABAK), which was .53. Accordingly, this can be interpreted as a
moderate agreement between the PDS results and the outcomes of the clinical interviews [
41]. When the answers to the PDS were not dichotomized but treated as an ordinal variable for agreement with the interview results, a significant and strong relation of Cramér’s
V = .59,
p < .001 was determined. In this sample 98% (
n = 88) of participants suffered from a depressive disorder, the remaining two patients who were not diagnosed with depression were correctly categorized by the PDS as
not likely suffering from PDD.
When a patient had been suffering from depressive symptoms for more than 2 years, the PDS categorized the patient as having a
likely diagnosis of PDD (answers [c] to [e]). The threshold for a PDD diagnosis can be shifted to examine its accuracy. Table
3 shows that when examining the Youden-Index there are two possible thresholds in the answers to the PDS – the original one at
more than 2 years (answers [c] to [e]) and the threshold at
more than 5 years (answers [d] and [e]). The latter provided a larger Youden-Index of
J = .56 compared to
J = .48 of the original threshold. However, it could only offer a sensitivity of 59%, which does not meet the requirements of how a screening instrument for depression should perform [
42]. It can be concluded that the original threshold at more than 2 years (answers [c] to [e]) showed the highest agreement coefficient in combination with a high sensitivity and a reasonable specificity. It offered the most accurate and valuable information.
Table 3
Diagnostical thresholds of the PDS
More than 1 year ago (a vs. b-e) | .30 | .002 | 90% | 38% | .28 |
More than 2 years ago (a-b vs. c-e) | .48 | < .001 | 85% | 63% | .48 |
More than 5 years ago (a-c vs. d-e) | .48 | < .001 | 59% | 97% | .56 |
More than 10 years ago (a-d vs. e) | .18 | .007 | 26% | 97% | .23 |
To examine the understanding of the PDS, we tested for differences in the agreement between outcomes of the interview and outcomes of the PDS by controlling for level of education. We found a slightly better value for
Cohen’s κ for patients with a higher level of education (κ = .51 ([95%-CI .20, .82],
p < .005,
SE = 0.16,
n = 30) compared to patients with lower education (κ = .46 ([95%-CI .22, .70],
p < .001,
SE = 0.12,
n = 59). Both values can be interpreted as
moderate [
41]. Sensitivity and specificity of the PDS were 83 and 67% for patients with higher education, whereas sensitivity for patients with lower education was 85% and specificity was 60%.
Retest reliability of the PDS
Data was collected again from 69 participants after an interval of 2 weeks to determine retest reliability (77% of main analysis sample). Agreement between the first result of the PDS and its repetition was 80% (55 of 69 cases). The agreement rate with
Cohen’s κ = .52 ([95%-CI .3, .74],
p < .001,
SE = 0.11) can be interpreted as
moderate [
41]. After adjusting for a small bias (
BI = −.06) and a moderate prevalence effect (
PI = .39), the agreement rate was supported by
PABAK = .59. When the answers of the PDS are not dichotomized, but examined as an ordinal variable for agreement, a moderate to substantial relation was detected, Spearman’s ρ = .49,
p < .01.