Each year about 270.000 children, adolescents and parents/caregivers
a are referred to youth departments of mental health care institutions in the Netherlands [
1]. The success rates of treatments at these institutions (mainly psychotherapy) are low. The small body of outcome studies in community-based, usual care settings in the United States has yielded a mean effect size near zero [
2] and estimates of deterioration between 14% and 24% [
3]. Furthermore, it is estimated that 40% to 60% of the children and adolescents discontinue treatment prematurely (dropout). Many of these dropouts are likely due to a perceived lack of benefit from treatment [
4].
Clients who experience no change or negative change will require a disproportionately greater amount of treatment resources. Prevention of such treatment failures, therefore, becomes an important goal of clinical practice. In addition, clinicians are seldom capable of predicting which clients will and will not improve through treatment [
5]. Factors that might be related to clients’ deterioration include symptom increase, ruptures in the working alliance and treatment goal failure. Although clinicians might recognize these factors, they have trouble noticing these cues in their clients [
6]. Clinicians need more systematic and reliable information about the status of their clients [
7]. Consequently, there has been a growing awareness in research and practice that clients’ response to treatment should be evaluated in a more systematic and structured way in order to adjust treatment when clients do not respond according to expectation. Doing so would likely improve treatment outcomes [
8].
Routine Outcome Monitoring (ROM) refers to regular measurements of clients’ progress in clinical practice [
9], using standardized instruments, aiming to evaluate and, if necessary, adapt treatment [
10]. Clients are invited to fill out questionnaires at the beginning of treatment, during treatment and at the end of treatment. Subsequently, clinicians are provided with feedback about their clients’ response to treatment [
11]. Based on the feedback, clinicians can make decisions on continuing, altering or terminating treatment [
12]. Thus, feedback focuses on the gap between the desired and actual results of care, and possible associated factors, that are within the control of the health care provider [
13]. Therefore, feedback interventions are a promising approach to improve clinical practice [
7].
Effects of feedback
Comprehensive and repeatedly updated reviews in general healthcare have shown that audit and feedback can have small to moderate effects on healthcare professionals’ compliance with desired practice and on client outcomes [
14]. However, effects vary according to the way the intervention is designed and delivered. In adult mental health care, research on the effects of feedback has shown slightly negative to large positive effects [
15‐
17]. Positive effects are shown on the accuracy of diagnosis [
18,
19] and on communication between the client and clinician [
16,
20]. Feedback can also significantly enhance treatment results [
21], even doubling the overall effect size of treatment [
22], and it can decrease the number of treatment sessions [
23]. Moreover, treatment results in feedback groups are maintained for a much longer time [
24] and less treatments are failing when clinicians receive feedback [
25]. Nevertheless, more than one-third of the feedback interventions have been shown to decrease performance [
26] and the effects of feedback did not prevail in the long term [
16].
Studies that have specifically investigated the effects of feedback in youth mental health care are rare. To the best of our knowledge, Bickman et al. [
27] is the only such published study, concerning home-based mental health treatment for children and adolescents in community settings. Assessments by children, adolescents, parents and clinicians indicated that children and adolescents whose clinician had access to weekly feedback improved faster than children and adolescents whose clinician received quarterly feedback. Additionally, a dose–response analysis showed that effects were stronger when clinicians viewed more feedback reports.
Evidence for feedback mechanisms
Most theories (e.g. Contextualized Feedback Intervention Theory [
28]) start with a comparison between the content of the feedback (in mental health care the client’s current health status and progress in treatment so far) and a goal (recovery). This comparison creates a positive or a negative evaluation of the clinician’s performance.
Clinicians who are dissatisfied with their performance will change their behaviour if they are committed to the goal and if they meet a threshold level of self-efficacy for the task (cf. Goal-Setting Theory [
29]). This mechanism is in accordance with studies in general healthcare, which show that feedback is more effective when clinicians previously agreed to review their practice [
30] and when they are actively involved and have specific and formal responsibilities for implementing change [
31]. Theory further states that detailed plans regarding where, when and how behaviours will be enacted may increase goal-directed behaviours by increasing self-efficacy [
13]. Additionally, action plans can facilitate success by increasing goal-commitment. These mechanisms have also been confirmed in general healthcare by research showing that feedback is more effective when it is more concrete; given more frequently (at least once a month) and both verbal and written, presented close to the time of decision-making, aiming to decrease rather than increase provider behaviours, and offering instructions with both explicit goals and a specific action plan [
14]. In mental health care, research has suggested that feedback should (a) be delivered as promptly as possible after the data have been collected to allow clinicians to perceive the connection between the feedback and their behaviour, (b) be given frequently so that changes in processes and outcomes can be observed as they occur and corrective actions can be applied if necessary, (c) be specific, (d) be written or graphic instead of verbally delivered, (e) should focus on the clinician’s actual behaviour instead of clinical outcomes, (f) be supplemented with a directive intervention and concrete suggestions about ways to improve and (g) be tailored to the needs and preferences of the recipients [
21,
26,
32,
33].
Alternatively, feedback may be perceived as inaccurate and therefore disregarded, resulting in external attribution of reasons for a lack of progress [
34], reduction in commitment to the goal [
21] or the recipient lowering the goal to make it more achievable (cf. control theory [
35]). This mechanism has been directly or indirectly confirmed in studies showing that if clinicians do not consider the feedback credible, valid, informative, or useful, they are more likely to dismiss it whenever it does not fit their own preferences [
36]. It has also been shown that characteristics of clinicians, namely whether they have an internal or an external feedback propensity, are committed to using the feedback and are self-efficacious, can moderate the effects of feedback [
37]. Additionally, there can be barriers for clinicians to use feedback, such as no accountability for using feedback and organizational factors [
28] like high administrative pressures, a heavy caseload, lack of time, having other tasks that are competing for attention and having difficulty interpreting the feedback [
37].
Clients can benefit from feedback only when clinicians pay attention to and use feedback [
38]. However, feedback may not be effective under all circumstances [
39]. Repeatedly, researchers have found that feedback appears to be most effective for clients who are not progressing well in treatment or who are predicted to have a poor treatment response, i.e. the not on track (NOT) cases [
9,
17,
40]. The largest treatment gains for potential non-responders are achieved when feedback is supplemented with clinical support tools. These tools are proposed as an evidence-based problem-solving strategy and are arranged in a decision tree to direct clinicians’ attention to certain factors known to be important in treatment outcome [
32].
Aims and hypotheses
ROM has become an increasingly popular method for improving treatment and has been adopted by many mental health care institutions worldwide [
22,
41,
42]. Although it seems that feedback from ROM has potential to enhance outcomes in clinical practice, there are still many unanswered questions. There is a need to operationalize and directly compare different approaches to improve the design and delivery of feedback. Also, in order to explain why feedback leads to improvement in some cases but not in others, more insight is needed into the processes underlying feedback. Moreover, barely research has been conducted in real-world clinical practice and outside the United States. Studies that have specifically investigated the effects of feedback in youth mental health care are rare.
The aim of this study is to investigate several potentially effective components of feedback from ROM in youth mental health care in the Netherlands. We will compare three different feedback conditions. In all three conditions feedback will be given to clinicians, will consist of written and graphic performance results, will be compared to norms, will be given at the start of treatment, one and a half months after the start of treatment and every three months during treatment and will be delivered one day after data collection.
In the first feedback condition, clinicians will receive basic feedback regarding clients’ symptoms and quality of life. However, reduction of symptoms or enhancement of quality of life is not always the primary or the single goal of treatment [
43]. Also, children’s and adolescents’ symptoms and problems often arise in interaction with family members [
44]. Furthermore, many children and adolescents have chronic health problems (e.g. ADHD or ASD), which might well persist into adulthood [
45,
46]. Frequently, children, adolescents and their parents want to learn how to cope with their problems and overcome obstacles that they encounter. Hence, it seems important to assess aspects like self-efficacy, social network and parenting skills, include the results in the feedback to the clinicians and provide clinicians with possible solutions if clients are not responding to treatment. This strategy is consistent with theory and research suggesting that more concrete and practical feedback is more effective. Thus, in the second feedback condition, we will enhance the feedback of the first condition by including information about possible obstacles to a good outcome that clients may be facing (e.g. problems with self-efficacy and the social network but also motivation and the therapeutic alliance) and practical suggestions for improving treatment (youth clinical support tools). Therefore, the feedback will be more specific than in the first condition, will be more tailored to the needs of the clinicians and will provide more information about the clinicians’ actual behaviour.
Although feedback might be intended to be practical and useful, literature shows that if feedback is regarded as inaccurate, incredible or invalid, clinicians might reject the feedback, attribute a lack of progress to external reasons and lower the goal of treatment. Accordingly, in the third feedback condition, we will prevent clinicians from disregarding the feedback. In this condition, the feedback that is provided in the second condition will be discussed with a colleague using a standardized format for case consultation. Because peers who have personal relevance for the clinician will give additional advice about ways to improve treatment, clinicians will probably consider the feedback as more credible, valid, informative and useful and will be more likely to accept the feedback. Therefore, compared to the other conditions, clinicians in the third feedback condition will pay more attention to the feedback, will become more involved and might feel more responsible to act upon the feedback.
The primary outcome measure will be symptom severity and secondary outcome measures will be quality of life, satisfaction with treatment, number of sessions, length of treatment and rate of dropout. Compared to the first condition, we expect that in the second and third feedback conditions (a) children’s and adolescents’ symptoms will decrease more rapidly and to a greater extent, (b) children’s and adolescents’ quality of life will improve more, (c) children, adolescents and parents will be more satisfied at the end of treatment, (d) treatments will contain fewer sessions, (e) treatments will be shorter, and (f) children, adolescents and parents will drop out of treatment less often. Overall, we expect that the largest effects of feedback will occur in the third feedback condition, where the basic feedback will be enhanced with additional information and practical suggestions, and clinicians will discuss the feedback in case consultation with a colleague. Additionally, we expect that feedback will be most effective for children, adolescents and parents who are not progressing well in treatment (i.e. are NOT).