A screening tool for psychological difficulties in children aged 6 to 36 months: cross-cultural validation in Kenya, Cambodia and Uganda

Despite the lack and heterogeneity of existing prevalence data, the burden of mental health problems in children and adolescents is estimated to be as high as 10–20% worldwide [1]. The largest proportion of this burden is located in low-resource countries, where up to half of the population is younger than 15 years [1]. In these countries, childhood psychological difficulties often remain undetected and thus untreated [2], limiting children’s full developmental potential and increasing the risk of later mental health difficulties [1]. In particular, infant and toddler’s mental health is often very low on the list of priorities [3]. In low-resource countries, the provision of mental health care is hampered by the lack of qualified personnel and limited access to health services [4] combined with stigma and poor awareness of psychological difficulties in young children [5, 6]. The absence of easy-to-use and cross-culturally adapted tools to assess mental health in young children further complicates disease burden estimation [7‐10] and the delivery of care [7]. Existing screening tools for children younger than three years may focus on specific disorders or symptoms [11, 12]; necessitate a long administration time [13‐18]; require highly-trained administrators [19]; and/or have not been cross-culturally validated in low-resource countries [20, 21]. Validating instruments to assess psychological difficulties in young children living in low-resource countries can provide an important tool to identify those in need. Building on the methods used for the cross-cultural validation of a screening tool designed for children aged three to six years [22, 23], we aimed to develop and to cross-culturally validate a screening tool for psychological difficulties among children aged six to 36 months.

In Kenya, 319 children were included in the first series (including 64 assessed twice for the inter-rater reliability and 56 assessed twice for temporal stability) and 215 in the second series. In Cambodia, 148 children were included to assess the external validity; 20 for inter-rater reliability and 21 for temporal stability. In Uganda, 142 children were included to assess external validity; 20 for inter-rater reliability and 20 for temporal stability. None of the children assessed for eligibility presented a previously diagnosed mental health disorder or visible signs of a severe mental health disorder. Participant characteristics are presented in Table 2. Median age of the children included was between 17 and 20 months. Across the three study settings, 19 children had an apparent development retardation or motor disability.

Due to missing values, 305 (95.6%) PSYCa 6–36 were completed in the first series of Kenya and 145 (98.0%) in Cambodia. There were no missing values in the second series in Kenya or Uganda. The scoring distributions of each item are presented in the Additional file 2. The median total score was a bit lower in the first series in Kenya and in Uganda (Table 3) and there was no evidence for a score difference according to age and sex. The PSYCa 6–36 was administered in a median time less than 15 min (Table 4).

We report the results of a cross-cultural validation study of a new instrument for screening children aged six to 36 months for psychological difficulties. More than 800 children with their caregivers were included across three low-resources settings.

Infant and child psychopathology measurements are challenging, notably due to the rapid motor, cognitive and emotional development in the first three years of life [32, 33]. Considering this complexity, the inherent limitation of quantitative measures to capture human behaviours and emotions [7], as well as the uniqueness and recent development of the PSYCa 6–36, this screening tool showed satisfactory psychometric properties and the ability to classify children with or without psychological difficulties as closely as the CGIS score. The performance of the PSYCa 6–36 was similar in Cambodia and Kenya, two very different cultural and linguistic contexts. This highlights the cross-cultural aspect of the PSYCa 6–36. The performance was lower in Uganda, which might result from actual differences across study populations but also from translation and adaptation flaws [34, 35]. The interviewer guidelines were more frequently used in Uganda than in the two other contexts. A limited comprehension of the items or instructions by the Ugandan participants cannot be excluded. In addition, there were study implementation challenges in Uganda, especially a divergence in the judgments of the psychologists which might have led to suboptimal standardization and clinical assessment. There may also have been administration differences between interviewers. This highlights field constraints and that, despite the ease of use of the PSYCa 6–36, proper training is mandatory.

The PSYCa 6–36 was developed with support of experts in child and transcultural psychopathology and translated by specialists in the local languages and cultural contexts. Translation procedures may not have fully achieved content and sematic equivalence but overall, the PSYCa 6–36 appeared well understood by the participants considering the low frequency (less than 5%) of missing answers. However, some caregivers might have rated some items without full understanding of their meaning or wording, as suggested by some low individual ICC in the test-retest reliability analysis. Also, about 10% of caregivers refused participation and interviewers informally reported that some caregivers felt uncomfortable with the use of quantitative questionnaire and with talking about “abnormal child behaviours” in their household. A lack of awareness of child psychology and the stigma surrounding mental health that affects all populations [4] might have influenced the caregivers’ willingness to disclose information about children’s difficulties. Because of such stigma, caregivers might have provided socially acceptable, consequently biased, answers. A qualitative evaluation might have strengthened the results of this study by shedding light on the caregiver’s perception and acceptance of the use of a questionnaire about child psychology in the different cultures.

For infants and toddlers, direct observation and evaluation of a child interacting with their caregiver in their natural environment remains the best option for mental health assessment [36]. We used the CGIS score assessed by a trained psychologist to assess external validity. The cross-cultural validity of childhood diagnostic criteria in mental health remains debated [9, 37‐39]. Although Kenyan, Ugandan and Cambodian psychologists performing the assessment likely limited misinterpretation of possible expressions of mental health disorders or symptoms that may be culture-dependent [9, 37‐39]. Further, because of their limited experience in young children’s mental health, they were trained by a child psychiatrist before the start of the study and then worked either under the close supervision or in tandem with a psychiatrist or psychologist experienced in cross-cultural and young child psychology. The clinical assessment was also reinforced by the use of the PIR-GAS scale, although not validated for low-resource settings. Despite these precautions, we recognize the limitation of the comparison with the CGIS might have biased [40] and, possibly underestimated the real PSYCa 6–36 performance.

In Western settings, the prevalence of socio-emotional and behavioural difficulties has been reported to range from 7 to 24% in children aged one to three years [3, 7, 41, 42] but there are data gaps for low-resources countries [1, 4, 8]. A systematic review of prevalence studies of child and adolescent mental health (age range 5 to 16 years) in Sub-Saharan African communities estimated that 14.3% of children had psychopathological difficulties, and 9.5% among studies of which measurement relied on a diagnostic interview [10]. In our study, the psychologists identified fewer children with a CGIS higher than one than expected. Children were included only in the presence of their caregiver and the study was conducted during working hours, thereby likely biasing the study sample towards children at lower risk. More vulnerable children, such as those living in households without a caring adult or left alone during the day, or street children were not included. Also, caregivers who refused participation may be caring for more vulnerable children. Another explanation might be that children living in these difficult environments and exposed to poverty and chronic adversity develop stronger coping mechanisms [7], protecting them against psychological difficulties or limiting the expression of psychological difficulties. This is particularly likely when children remain under the stable protection of their caregiver or other close relatives [7]. Nevertheless, the PSYCa 6–36 would benefit from further use and validation in populations with higher prevalence of psychological difficulties, notably in children having recently faced an acute traumatic event such as migration, conflict, or natural disaster.

In Kenya and Cambodia, the cut-off point maximizing the sensitivity and specificity of the PSYCa 6–36 to differentiate children with CGIS score of > 1 lies between eight and eleven but it is lower in Uganda. Hence, a cut-off point of eight appears an optimal compromise but it should remain flexible to favour sensitivity or specificity according to the expected burden of psychological difficulties and available health services of each specific setting. A cut-off point of eight would identify a substantial proportion of the population as falsely positive, possibly overloading mental health professionals with unnecessary referrals. A higher cut-off would better limit referral to children in need of further clinical evaluation. The definite choice of the cut-off requires subsequent documentation and analysis in populations with higher prevalence of mental health difficulties such as migrants, refugees or internally displaced children, children living in conflict situations or in the aftermath of a natural disaster, or sick children. Further investigation is also needed among specific age groups, such children below one year of age.

In the three study settings, follow-up care was offered by the psychologists and counsellors focusing on the reinforcement of the caregiver-child relationship. Although the child psychiatrists in Kenya and Uganda ensured access to specialized care, such care was limited in Cambodia, being only available in the capital city, a few hours drive from Kampong Cham. It is important to note that, although follow-up care was free of charge, psychologists needed to build trust through repeated home visits to ensure referred children were cared for appropriately. Reducing stigma, misperceptions, and increasing awareness of child psychology among the community and health professionals remain a challenge to support community screening efforts and subsequent access to mental health care [43‐46]. The PSYCa 6–36 can also be means to raise awareness of child psychology among the population and of the importance of early recognition to limit long term and developmental consequences.

The PSYCa 6–36 allowed for rapid screening of psychological difficulties among children aged six to 36 months among the studies populations. Use of the tool also increased awareness of children’s psychological difficulties and the importance of early recognition to prevent long-term consequences. The PSYCa 6–36 would benefit from further use and validation studies in populations with higher prevalence of psychological difficulties.