Routine outcome measurement in adolescents seeking mental health services: standardization of HoNOSCA in Kenyan sample

There has been an increase in interventions for the management of child and adolescent mental health problems, with reviews showing positive results for psychosocial interventions with children and adolescents [1‐5]. One way of assessing the outcome of an intervention is by measuring it during the treatment process. Outcome measures have been identified as essential to demonstrate patient improvement [6] and determine whether these interventions are effective [7]. They also improve clinical practice when it is part of a feedback monitoring system for clinicians by informing clinical decision making and enable the clinician to adjust treatment planning accordingly [8].

The need to evaluate routine health services has led to the development of various outcome measures [8, 9], with the purpose and use of outcome measures depending on the end user [10]. The Health of the Nations Outcome Scales for Children and Adolescents (HoNOSCA) is one of these outcome measures. The HoNOSCA is part of a larger body of work - the Health of the Nation Outcome Scales (HoNOS) developed by Wing et al., [11, 12] in the United Kingdom as a brief mental health measure with the aim of tracking progress towards improving the health and social functioning of mentally ill people. The HoNOS were intended to cover the range of symptoms, behaviours and social difficulties associated with mental illness and can be used in any health setting to measure behaviour, impairment, symptoms, and social functioning in patients [13]. It was noted that the HoNOS scales were not appropriate for use with a younger population, leading to the development of the HoNOSCA by Gowers et al. [14, 15]. The HoNOSCA was first developed as a clinician rated measure to track child and adolescent outcomes in routine care. The self-report version (HoNOSCA - SR) was developed from a criticism that it failed to consider the patient’s perspective [16]. A caregiver version is also available. Although few studies have reported on the self-report version of the HoNOSCA [16, 17], the clinician rated version has been used extensively to regularly monitor the patient’s progress [13]. The HoNOSCA has four subscales relating to: (i) a behaviour (disruptive behaviour, overactivity, self-injury, inattention and substance misuse); (ii) an impairment (physical illness and disability - problems with scholastic or language skills, physical illness or disability problems); (iii) symptoms (psychotic and emotional symptoms - problems associated with hallucinations, delusions or abnormal perceptions, problems with non-organic somatic symptoms or emotional and related symptoms); and (iv) a social ((problems with peer, family relationships, problems with self-care and independence, poor school attendance) [17, 18]. Several studies have examined the psychometric properties of the HoNOSCA [13, 14, 16] and because of its good psychometric properties and ease of use in routine clinical services, it has been translated into different languages [18].

In Kenya, there are few translated and/or culturally validated scales to assess and detect psychopathology especially in children and adolescents. The Child and Behaviour Checklist [19] and Strengths and Difficulties Questionnaire [20] have been commonly used in several studies for the assessment of child and adolescent problems [21‐25], but they have not been used to measure outcome during and post treatment. They have also not previously been validated in Kenya.

Evidence on the effectiveness of interventions conducted in mental health services for children and adolescents in Kenya remains limited, in part due to a lack of validated scales. Our study aims to address this issue by evaluating the psychometric properties of the self-report and clinician rated versions of the HoNOSCA for outcome assessment with adolescents seeking mental health services in Kenya.

The study was conducted at the Youth Centre, at the Kenyatta National Hospital (KNH), Kenya’s largest teaching and referral hospital. The clinic caters to young people between the ages of 12–24 years. Services offered include mental health, sexual and reproductive health services including Voluntary Testing and Counselling services. The study targeted adolescents between the ages of 12–17 years seeking services for emotional and behavioural problems. Many of these adolescents are referred from academic institutions or by a caregiver. From time-to-time they may self-refer. Adolescents attending the clinics during the months of May 2019 to March 2020 were invited to participate. To be included in the study, patients had to be between 12 and 17 years, both caregiver consent and assent from the adolescents was given. We also invited clinicians working with these adolescents to participate in the study. All participants provided written consent to participate. Patients with moderate to severe intellectual disability, experiencing acute psychotic symptoms and having any unstable medical condition were not included in the study. Our study protocol is described in more detail in Wambua et al. [26].

The majority of the participants in our study were female at 54.2% and the mean age of our participants was 15.88 years. The most referrals to the clinic were made by academic institutions (84.2%). Only 11.9% of the participants had previously received prior treatment. Most of the participants (61.2%) experienced psychosocial problems. See Table 1.

While the HoNOSCA is widely implemented and continues to garner significant recognition, these studies have all been carried out in the developed world. It has not been used or validated in any LMIC context. Our study found both versions of the HoNOSCA easy to use and reliable to assess global severity of mental health problems experienced by children and adolescents seeking services in the youth clinic at KNH.

Internal consistency of the self-rated HoNOSCA was at .77, which suggests that the different items carried independent weight, similar to Ballesteros-Urpi et al. [17] who found alpha of .74 and .76 on the self-rated version. Our findings on the clinician version showed poor internal consistency (.58), which was similar to other studies that found low alpha at .45 [36] and .65 [37]. Inter-rater reliability for both scales was found to be strong and similar to those found in other studies, which range from 0.72 to 0.96 [17, 18, 27‐29].

Concurrent validity between the self-rated version and the PSC were moderate. The poor correlations between clinician rated and self-reported HoNOSCA or PSC scores were similar as in other studies. The initial study validating the self-report version [16] found it to be weak when compared with the SDQ. Other studies also found poor correlations between the clinician rated HoNOSCA and other adolescent self-reports [17, 22, 29]. The different item scores poorly correlated with each other on both versions and this was found to be similar with the original study by Gowers et al. [14] whose correlations range of correlations were 0.01–0.41, suggesting that the HoNOSCA is unidimensional scale. These poor coefficients suggest that there is limited comparability between the two versions of the scale warranting further research with specific tools for adolescents and clinicians. The low concurrent validity of clinician rated HoNOSCA, requires that we carry out further validation studies with scales geared for clinicians.

Change in the scores at follow-up were moderately correlated with initial scores on both scales of the HoNOSCA. Similar with other studies, our findings show that both versions are sensitive to change, indicating that they can be particularly helpful for use in child and adolescent mental health services [14, 16, 17, 27].

We found a four-factor solution, our factor analysis did not confirm the areas of functioning as hypothesised a priori by Gowers et al. [14] or as evidenced by Tiffin and Rolling [38]. Further investigation is required to assess the factor structure in our local context as sub-scales may not translate easily across cultures.

The HoNOSCA has proven to be easy to use in routine care and useful to achieve outcome measurements in child and adolescent mental health services. We would benefit from further testing of HoNOSCA in our context with a larger sample, and strict parameters between sessions. Internal consistency and concurrent validity of the clinician version was low warranting further evaluation.