Background
Although child health services in Nordic countries and Scotland are well developed for safeguarding the health and development of children, assessments and surveillance are hindered by the lack of evidence-based screening and evaluations of interventions [
1]. In western countries screening of children’s health is recommended, but delivery and content vary considerably between countries [
2]. According to Alexander [
3], service providers in the United States (US) rarely rely on relevant clinical end-points to assess the intended and/or expected health changes in children which is similarly reported for the Nordic countries [
1].
The proportion of children with poor health is growing slightly [
4], and the trajectories of poor health commence and are identifiable early in the lives of children [
5]. Shonkoff et al. [
6] have suggested the need for initiatives to strengthen the resilience of children and efforts to protect children from unhealthy development. These initiatives should rely on evidence-based measures and interventions that are acceptable, feasible, and affordable, and they should strengthen informed professional and parental engagement [
6].
In a study of the competence needs of child and school health nurses (CSHNs) in Finland, where child health care is comparable with that in Norway, Putkuri et al. [
7] reported that intuitive and interpersonal competencies are well developed in child health care. However, it is necessary to advance and develop theoretical and evidence-based competencies, including the use and interpretation of health assessment scales. Furthermore, advanced electronic patient records and digital tools are required to support evidence-based screening tools and intervention strategies. For example, another study conducted in Finland using online tools determined the utility and appropriateness of a mental health screening tool (strength and difficulties questionnaire, SDQ), as well as how to overcome barriers to support parents and develop competence among CSHNs [
8]. In a Swedish project nurses preferred using SDQ to assess mental health of children in child health clinics, but emphasized the importance of reducing individual and organizational level barriers [
9]. In addition, systematically collecting data on the development and health of children can facilitate the distribution of services to geographical areas with the highest need for follow-up [
10].
Nurses throughout the world have reported a need to establish positive therapeutic relationships with families amid growing concerns about child health challenges [
11,
12], and it is necessary to validate their experience-based and situated concerns about children [
12]. In particular, structured health assessments can improve the identification of children’s problems [
13], provide decision support [
14], create opportunities for evaluating interventions [
15], and enhance communication [
16]. Client-reported questionnaires concerning their health can enhance routine management by promoting client–clinician communication and client involvement [
17]. Moreover, digital health assessments may be preferable and more acceptable for clients than paper-based questionnaires because of their faster completion time, lower cost, and increased response rate with resulting higher data quality [
18]. However, the challenges associated with client-generated health data acquired using electronic tools include financial investment, privacy protection, and the exclusion of populations not familiar with electronic devices and platforms [
18]. In addition, the ability to respond to patient generated health data could be challenging, including integration in clinical practices and processes [
19]. In general, health care professionals are reported to have high digital literacy, but one-fifth report anxiety using information technology, which could endanger patient safety and the quality of service, and thus targeted training is required for staff with low digital literacy [
20].
Validated structured health assessments using information technology need to be implemented in child and school health services. The Norwegian “Starting Right™” project uses digital proxy- and child-reported questionnaires to support CHSNs in assessments of the general and socio-emotional development of children’s health and development. This project has been piloted in routine child and school health care [
21] using an online solution delivered by CheckWare Ltd. The Starting Right child and school health service innovation consists of parent- and child-reported online structured health assessments tools, including practical routines for use in assessments of child and school health services among children aged from six months to 16 years [
21]. Previously validated questionnaires are used in the innovation, such as the SDQ [
22], health-related quality of life (KIDSCREEN-27) [
23,
24], general development (ages and stages questionnaire) [
25], socio-emotional development (ages and stages questionnaire: social–emotional) [
26], and anxiety (Spence child anxiety scale, short) questionnaires [
27]. Parental acceptability and adoption by CSHNs were satisfactory in the pilot phase [
21]. Before implementation, the preparation stage included monthly meetings with two CSHNs, having an official mandate for professional quality improvement in a large municipality in Norway. The instruments and starting age for screening were selected in collaboration with the relevant services, and information about the innovation was attached to an appointment letter sent to parents. Educational sessions concerning the questionnaires (5 h) and online tool (3 h) were provided to all CSHNs before implementation at the centres involved [
21]. In the pilot phase, a medical secretary supported the CSHNs in distributing health assessments to parents prior to scheduled appointments, which increased the rate of adoption by CSHNs [
21].
It is recommended to evaluate clinical experiences to identify challenges and required improvements, and to accommodate and provide further support with implementation [
28]. In their design and evaluation implementation framework, Fixsen et al. [
29] described seven core implementation components: (1) staff selection; (2) pre-service and in-service training; (3) ongoing coaching and consultation; (4) staff evaluation; (5) decision support data systems; (6) facilitative administrative support; and (7) systems interventions [
29]. In the present study, we focused on the core component of staff evaluation with the specific aim of exploring the experiences of CSHNs with the Starting Right innovation.
Discussion
The aim of the present study was to explore the experiences of CSHNs with the implementation of the Starting Right child and school health service innovation. Overall, our findings indicated that the nurses were positive about the innovation. However, their experiences with the implementation varied and some of them were contradictory. Despite this motivation to implement the health service innovation, previous studies have shown that the implementation of structured self-reported health assessments can lead to diverse experiences, such as challenges including assessments as standard practice [
38], given that the implementation process involves complex interactions between various factors [
39]. Our findings are in agreement with previous studies that highlight, for example, scepticism regarding the validity of patient self-reporting, uncertain clinical benefits, concerns about workflow, and the lack of time [
40,
41].
The digital innovation could be used as a method to obtain a good overview of a child’s health and development
The CSHNs emphasised the importance of their involvement in the implementation and use of the innovation. Some CSHNs were involved in planning the innovation and in selecting the instruments used [
21] but the participants in the present study considered that the currently used questionnaires should be changed according to their needs. Their practice was not based on traditions of using structured and validated health assessment tools, standardised forms, or systematic assessments in accordance with the World Health Organisation criteria for these tools [
1], but they had to follow quite detailed national guidelines [
33]. Thus, there could be differences in how to interpret their professional obligations in relation to the implemented assessments, and reciprocal and continuous adjustments of the innovation may be important beyond the pilot phase, as we reported previously [
21]. Waldron et al. [
26] also emphasised the need for continuing education when implementing routine outcome measures. Moreover, competence needs concerning the assessment rating scales and aetiology of symptoms, as well as appropriate interventions, were highlighted by Putkuri et al. [
7] for a comparable child and school health service innovation in Finland.
In the present study, the CSHNs found that using the innovation led parents to reflect more actively on their child’s development and health before the consultation. This finding appears to be consistent with previous investigations of the experiences of patients using self-reports of health, and thus these questionnaires may prompt patients to reflect on their own health [
42]. Reflecting by using structured self-reported health assessments may increase the likelihood of raising personal issues with clinicians [
42]. Furthermore, this may allow CSHNs to be more prepared for the consultation.
Interpreting the questionnaires was a challenge
One of the main concerns raised by the CSHNs was the ability of parents to fill in the questionnaires. They found that some of the parents had problems understanding the meanings of the questions and they were concerned whether this might result in the parents postponing filling in the questionnaires or not responding. In addition, the issue of a language barrier, especially for ethnic minority parents, was considered a further problem that affected the ability of parents to respond. Several questionnaires have been translated into different languages and they were accessible as paper versions for translation support in the centres but online digital questionnaires were only accessible in Norwegian during the implementation phase of the Starting Right innovation. Clearly, the lack of access to questionnaires in different languages could have prevented ethnic minority parents from completing the questionnaires, as shown in previous studies [
43,
44]. In Norway, approximately 15% of the population are immigrants [
45], and in 2020, approximately 26.000 of the population received Norwegian language training of which 1 out of 4 where from Syria [
46].
Another challenge to the implementation process may have been related to the “digital divide”, which is considered one of the major disadvantages of online structured self-reporting of health. A systematic review by Meirte et al. [
18] found that people with limited access to computers or those who are generally computer illiterate are likely to be disadvantaged. If the parents encountered challenges because they lacked access to computers or mobile phones or had problems understanding the questionnaire, then they clearly would have had difficulty in filling in the questionnaires. Our previously reported parental response rate of 80–85% [
21] also indicates that a group of parents did not respond to the questionnaires. These findings correspond with a Swedish study who revealed that nurses experienced that not all parents filled in the questionnaires [
47]. The occasional parents who did not respond could be a challenge for the CSHNs in their daily clinical work regardless of their reasons for not responding.
Some of the CSHNs experienced differences between their own perceptions of a child and the scores provided by some of the parents. They described cases of parents who both over- and under-estimated their child’s development and health challenges compared with their own perceptions. It is well known that people tend to respond in a manner that might be considered socially acceptable, especially when dealing with sensitive issues [
48]. Social desirability can result in under-reporting due to the tendency to present oneself in a manner that is socially acceptable, or even to avoid criticism [
49]. For some parents, social desirability might be an issue when scoring their child because they may be unwilling to reveal certain information. Some parents might also believe that questionnaires do not capture their concerns [
50]. However, some of the nurses experienced the opposite where parents reported concerns about their children that were not perceived by the CSHNs. These findings agree with those obtained in other studies where low agreement was observed between the assessments of health professionals and parents regarding their children [
51,
52]. Previous studies have also highlighted the need for clear cut-off values and interpretations of questionnaires such as the SDQ in different situations and contexts [
53], and it may be necessary to use population-level data to improve these interpretations. On the other hand, the scores obtained from questionnaires might provide health professionals with information regarding previously unidentified issues [
54], thereby allowing them to provide appropriate advice or interventions to meet the needs of children [
55]. Several nurses argued that although most of the information regarding children was known to them, the specific conditions of children were clarified by the questionnaires, as also shown previously [
56]. An integrative review by Lines et al. [
12] highlighted that working with vulnerable children involves putting “pieces of a jigsaw puzzle” together through communication and validation. In the present study, the discrepancies observed by CSHNs between different information sources suggest that health assessments alone may not be sufficient to meet the needs of CSHNs. Lines et al. [
12] argue that nurses have concerns about making mistakes and harming children or families when trying to provide support, or take action based on their “gut feeling” that something is wrong. We will argue that different information sources, including questionnaire-based health assessment scores, are needed to support nurses in taking appropriate action. Thus, although questionnaires can be considered tools for formally assessing the health situations of children, they do not entirely solve the dilemma of balancing surveillance with support. They cannot necessarily guide a quick and obvious response, which may have been required by nurses in the present study, as well as in general [
12]. The systematic mapping of a child’s health and development may result in an increased commitment to the further utilisation of knowledge, and the nurses suggested that there is a requirement for a framework to determine how and when follow-ups should be performed for children with special needs. These findings echo those obtained in a systematic review by Boyce et al. [
41] who reported that one of the barriers that hinders the implementation of questionnaire-based health assessments may be uncertainty about how the scores should be interpreted and how to transfer these interpretations into clinical care. Greenhalgh et al. [
57] note that clinicians often have limited knowledge about how to implement structured questionnaire-based data in clinical practice. Similarly, Putkuri et al. [
7] found that CSHNs need greater competence in using assessment rating scales, in understanding the aetiology of mental health symptoms in children, and of knowledge of the chain of care as well as preventive interventions.
The CSHNs were very concerned about whether the intervention might enhance the identification and support of families and children with health care issues. A realist synthesis by Greenhalgh et al. [
42] showed that the use of questionnaire-based health assessments by clinicians is strongly influenced by their professional roles and relationships with patients. Clinicians indicated that standardised patient reported outcome measures (PROMs) could hinder communication and relationships with patients, whereas more individualised PROMs could strengthen the dialogue [
42].
Implementation of the new digital innovation was time-consuming
The time required by the innovation was considered an important issue by CSHNs and they were concerned about how it influenced their current working situation. In particular, they were concerned about the time taken to use the innovation during the implementation phase, such as when distributing questionnaires and access reports in the system. Implementing questionnaire-based health assessments is time-consuming [
43] and although online tools can facilitate more time-efficient assessments, Meirte et al. [
18] reported that time constraints remain barriers that may be related to digital literacy. The nurses described the value of access to support, especially concerning technical issues during the implementation phase, which may reflect challenges concerning digital literacy among CSHNs and that special support and training could be needed, as reported previously [
20]. The issues of digital literacy and level of digital support could also explain the difference in the adoption rate identified in a pilot study of the implementation of the Starting Right innovation [
21], in which the adoption rate was 96% in a group supported by a medical secretary in the pilot phase, but only 55% among CSHNs with no support. This discrepancy may indicate that CSHNs need extended support to overcome barriers related to time requirements when implementing the innovation.
Strengths and limitations
The present study was conducted during the COVID-19 pandemic. As a consequence, some of the child and school health services temporary suspended their implementation or use of the Starting Right project, which inevitably affected the experiences of the nurses with the innovation. Nevertheless, we successfully recruited 18 CSHNs with a wide range of ages and years working as CSHNs, and thus a broad range of experience in public health nursing; these factors can be considered strengths of this study. Another strength was that the focus groups were moderated by two researchers who were registered nurses, and part of the research group, but they were not involved in the implementation of the innovation.
Although the Starting Right project had only been piloted for about one year, abundant data were obtained from the initial phase of the implementation. However, if the interviews had been conducted later, we could have obtained information about how the questionnaires were used after the initial challenges were addressed. In addition, our participants only comprised female nurses. Including male participants in Norwegian studies is a challenge because the proportion of male CSHNs in Norway in 2018 was only 0.3% [
58].
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.