Validity of the patient health questionnaire 9-item in autistic youths: a pilot study

Depression is a common comorbid psychopathology in autism spectrum disorders (ASD) [1, 2]. A prospective study over a period of 1–5 years investigating psychiatric comorbidities in autistic youths aged 6 to 20 years reported that 42% had experienced an increase in expression of suicidal behaviors from childhood to adolescence, and of the 30% that suffered from major depressive disorder (MDD), 20% exhibited severe suicidal behavior or had attempted suicide [3].

Screening for depression is essential for early detection and treatment. Families of autistic patients are more likely to make frequent primary care visits [4], thus, primary care physicians play an important role in diagnosis, and prompting early intervention for depression—especially for patients and families who may not seek psychiatric care. Current tools used to assess depression in autistic youths may be inappropriate for outpatient healthcare settings as they may be costly, time-consuming, involve extensive scoring processes and/or an additional psychiatric specialist assessment [5‐8].

The Patient Health Questionnaire 9-item (PHQ-9) is a widely used depression screen [9, 10] in primary care, outpatient, and research settings with good reliability and validity. It is brief, user-friendly, easy to administer, and available in parent- and self-reported formats. Furthermore, the PHQ-9 promptly categorizes the severity level of depressive symptoms and is available without cost, making it effective for busy healthcare settings. The PHQ-9 has shown good validity in neuro-typical adolescents [11], in a multi-ethnic Asian sample such as those in Singapore [12] and in autistic adults [13]. However, no study has investigated its utility and optimal cutoff in autistic youths as yet.

This study aims to determine the optimal cutoffs for the self- and parent-reported PHQ-9 screen for MDD (as verified by the Mini-International Neuropsychiatric Interview, Kid version [MINI-Kid]) in autistic youths. We hypothesize that different cutoffs (compared to neuro-typical adolescents) may be needed to exclude MDD in autistic youths, while maintaining an acceptable false positive/negative rate, since clinical symptoms of ASD may overlap with core symptoms of depression (e.g., sleep disturbances, abnormal eating habits, psychomotor retardation and irritability) [14]. Secondary aims include determining internal consistency and convergent validity between the self- and parent-rated PHQ-9 and investigating possible correlations between participant characteristics and severity of functional impairment and depressive symptoms in autistic youths.

The final sample consisted of 101 youth and parent pairs. Youth participants ranged in age from 9 to 18 years old, with mean (SD) age of 14.6 (2.3); CELF-5 scores ranged from 11.0 to 30.0, with mean (SD) 21. (5.2). Participant characteristics are summarized in Table 1.

Of the 101 youths in the final sample, a total of 27 (27%) had experienced a MDD or MDE (major depressive episode; in cases diagnosed with Bipolar Disorder), at least once in their lifetime. 19 (19%) met the criteria for MDD, as verified by the MINI-Kid. Of these 19 cases, 12 (63%) experienced recurrent MDEs, and/or associated suicidality and psychotic features (Fig. 1). Those currently diagnosed with MDD were subsequently flagged up to their primary psychiatrist for management.

The mean (SD) total scores for self- and parent-rated PHQ-9 were 9.0 (6.0) and 6.2 (5.6) respectively. Frequencies of depressive symptoms severity were compared between self- and parent reports (p = 0.0071), summarized in Fig. 2. Frequencies of dysthymia during the past year (p = 0.1228), suicidal thoughts in the previous month (p = 0.0522) and attempted suicide at least once (lifetime; p = 0.0495) were also compared and summarized in Fig. 3. Frequencies of functional impairment severity levels were compared and consistent between self and parent reports (p = 0.3949), summarized in Fig. 4.

Table 2 lists selected cutoffs based on the ROC analyses of self- and parent-rated PHQ-9 scores for diagnosing MDD in autistic youths as verified by the MINI-Kid. ROC curves are presented in Figs. 5 and 6. The area (95% CI) under the curve (AUC) for the parent- and self-rated PHQ-9 was 0.653 (0.52–0.78, p = 0.094) and 0.716 (0.60–0.83, p = 0.007), respectively. The statistically optimal cutoffs based on the YI was 5 for the self-rated PHQ-9 with 100% sensitivity and 35% specificity, and 6 for the parent-rated PHQ-9 with 74% sensitivity and 59% specificity. For the self-rated PHQ-9 with YI cutoff of 5, the negative predictive value (NPV) was 100%, and the positive predictive value (PPV) was 26%; for the parent-rated PHQ-9 with YI cutoff of 6, the NPV was 91%, and the PPV was 29%.

Both self- and parent-rated PHQ-9 showed acceptable internal consistency, as demonstrated by the Cronbach α coefficients of 0.810 and 0.845, respectively. Although statistically significant, the Pearson correlation between self- and parent-rated PHQ-9 scores would be considered weak to moderate (r = 0.491, n = 101, p = 0.01).

Spearman correlation coefficients for associations between self- and parent-rated severity of depressive symptoms and of functional impairment, and number of comorbidities are summarized in Table 3. Age and education demonstrated no significant correlation with self- or parent-rated severity of functional impairment or with depressive symptoms.

Consistent with previous literature [6, 20, 21], our study demonstrated a high co-occurrence of depression in autistic youth, emphasizing the importance of routine screening for depression. Almost half of our total sample self-reported dysthymia symptoms and at least a quarter reported suicidality. 19% met criteria for MDD, and of this sample, 37% had associated psychotic symptoms and/or suicidality (Fig. 1). The high rate of depression may be attributed to ASD-related social issues, such as peer victimization [22], social difficulties [23, 24], impaired ability to emotionally adapt to perceived social failures [25], greater loneliness [26], poorer quality of life [27] and lack of friendships [28] compared to neuro-typical peers. Additionally, autistic individuals may engage in greater use of maladaptive forms of emotional regulation, such as rumination and shutting down (e.g. emotional numbing) [29], which are highly associated with the development of depression and anxiety.

Similar to previous literature [7, 8], the number of comorbidities was found to be positively correlated to self-reported functional impairment (r_s = 0.338; p = 0.01) and depressive symptoms (r_s = 0.327; p = 0.01); this suggests that effective treatment of comorbid psychiatric issues may be important for a holistic approach to depression in autistic youth.

Although the PHQ-9 has been shown to have similar psychometric properties within both the autism and community sample aged 15–80 years old, indicating its usefulness in measuring depression in ASD individuals in that age group [13], our study found it to be a less than ideal screening tool for MDD in autistic youths. High sensitivity is of primary concern in determining the clinical utility of depression screens when false negative outcomes can mean morbidity and fatality. Using the conventional cutoff of 11 for the self- and parent-rated PHQ-9 in autistic youth in this study yielded lower sensitivities and specificities than for neuro-typical adolescents [11] (Table 2). The statistically optimal cutoffs based on the YI for the self- and parent-rated PHQ-9 are 5 and 6, respectively. However, despite these adjustments, the parent- and self-rated PHQ-9 would still be inappropriate depression screens. A sensitivity of 73.7% for a cutoff of 6 in the parent-rated PHQ-9 would lead to a high false positive (58.5% specificity) and negative rate. Even if sensitivity was increased to 78.9% by decreasing the cutoff to 4, the specificity decreases even lower to 40.2%. On the other hand, a self-rated PHQ-9 cutoff of 5 yields 100% sensitivity, but an even higher false positive rate (26.4% specificity). A proposed cutoff of 8 would only minimally increase specificity to 50.0% while compromising sensitivity to 78.9%. Nonetheless our results show relatively poor specificity compared to sensitivity at the adjusted cutoffs for both self- and parent-rated PHQ-9 in detecting MDD in autistic youths, indicating that the PHQ-9 would be relatively better at ‘ruling out’ MDD than ‘ruling in’. Hence, a positive screen on the PHQ-9 could be followed up with a formal supplementary evaluation by a specialist familiar with the manifestation of depression in autistic youths.

Other studies also found relatively poor specificity compared to sensitivity in detecting co-occurring emotional problems in autistic youths at established optimal cutoffs [8]. These findings may be due to depressive symptoms overlapping with baseline characteristics of ASD itself [14, 30], making it difficult to distinguish between the two. Another possible reason for the lack of specificity is the high co-occurrence of other psychiatric comorbidities in ASD, which could account for some symptoms in the PHQ-9 [31].

Possible reasons for lower PHQ-9 cutoffs for autistic youth compared to neuro-typical adolescents are: 1) emotional states not being congruent to what is observed, 2) PHQ-9 not fully capturing the manifestation of depression in autism, 3) alexithymia in ASD [32] and/or 4) difficulty interpreting questions asked [33]. Previous literature suggest that individuals with ASD find identifying, processing [34], regulating [29], and communicating [35] emotions challenging. They also often have marked impairment in the quality of the affect—showing facial expressions that are neutral and difficult to interpret—and disengagement that could mute the association with depressive symptoms [29]. Inadequate emotional regulation abilities could predispose them to rapid escalation of reactions that are out of proportion to what is typically experienced. Taken together with poorer and unconventional communication skills, these could have resulted in depression, despite a lower PHQ-9 score. Another possibility for a lower score is that ASD-specific symptoms, such as changes in repetitive and stereotypic behaviors, are not accounted for. Hence, it may be difficult for current tools to fully capture depression experienced by ASD youth. A screen that incorporates an evaluation of change in clinical symptoms of ASD during the onset of depression, such as intensification of social withdrawal and ritualistic behavior/obsessions coupled with associated features, such as irritability, compulsivity, hyperactivity and decreased adaptive functioning, [30, 36] may be a helpful and appropriate alternative.

Possible reasons for the discrepancy between the parent- and self-rated adjusted cutoffs could be related to parents’ under-detection of depressive symptoms in autistic youth and cultural stigma around mental illness [12, 37]. Our findings indicate almost twice as many youth rating their depressive symptoms as moderate, moderately severe, and severe compared to their parents; this under-reporting by parents is consistent with previous studies [38]. Parents may be less forthcoming when reporting depressive symptoms, less inclined to discuss depression with their child, or simply may not be aware of the indications of depression. Parents may also interpret these symptoms as temporary variations in typical autistic behavior. In this regard, if the autistic youth self-reports depressive symptoms that are significant while their parent does not, it may be prudent to evaluate the case further regardless.

Our study result is preliminary, and our sample size may be too small to support definitive conclusions. Replication of our analyses and evaluation of psychometric properties of the PHQ-9 on a larger and more diverse sample with variables that could be related to depression in autistic youth can improve our understanding of the appropriate uses and interpretations of the PHQ-9 in specific subgroups within the autistic population. For example, males with ASD are found to have more externalizing behavior problems [39] and less internalizing symptoms [40] than females, potentially leading to positively skewed results. Further studies should also explore how depression may present in those with lower communication and cognitive abilities, as these factors could have variably affected and influenced our results. Additionally, 46% of our patients were on psychotropic medications at the time of enrollment, which could affect the presentation of depression. Those on antidepressants for more extended periods may have less florid depressive symptoms and functional impairment, while those on other psychotropic medications, such as anxiolytics or stimulants, may have had medically-induced subthreshold depressive symptoms. There is also a possibility of recall bias when interviewing autistic participants about MDEs and symptoms of other psychiatric comorbidities. For example, since this population may have a greater tendency to ruminate over past negative experiences [41], reported depressive symptoms/recollections might extend beyond the previous two weeks indicated for diagnosis. Controlling for the impact of confounding factors such as psychotropic medications, sex, ethnicity, age, comorbidities and history of previous MDEs in future studies [42] is recommended.

Future studies could also examine the influence of clinical severity profile of participants’ ASD and its consequential functional impairments, on the presentation of co-occurring depression. Severe impairment in social skills may lead to increased dysfunction in interpersonal relationships and negative life events, precipitating relatively more depressive symptoms. On the other hand, severe ASD symptoms could mask the manifestation of depression symptoms, resulting in under-reporting in both youths and parents, or a missed diagnosis by the clinician. A depression screening scale that takes into account the severity of ASD symptoms and corresponding changes would be useful to determine the presentation, and severity of presentation of depression in individuals with ASD.

Due to the high co-occurrence of depression in autistic youth, routine screening, exploration of functional impairment, and effective treatment of other comorbid psychiatric issues should be conducted to allow for early multidisciplinary intervention, and minimization of associated morbidity and suicide risk. Although our study found low sensitivities with unacceptably high false negative rates using conventional, statistically optimal, and proposed cutoffs for the parent- and self-rated PHQ-9, the small sample size does not allow definitive cutoffs to be made with confidence. The discrepancy between proposed parent- and self-rated cutoffs indicate that interpretations based on a single source may be inconclusive and require additional evaluation by a specialist. Fundamentally, it is difficult to evaluate depression in the autistic population as 1) patients may have difficulty in communicating and understanding how they feel, and 2) behaviors suggesting a depressive disorder in ASD have yet to be clearly identified [14, 30]. Future studies should improve on the validity and reliability of existing screening tools, or explore and develop more appropriate screening methods for depression in autistic youths.