Development of the national consensus statement on ear health and hearing check recommendations for Aboriginal and Torres Strait Islander children aged under 6 years attending primary care: systematic scoping review and e-Delphi

The project commenced in February 2021 and ended in June 2022. It involved a systematic scoping review of Australian and international literature relevant to the components and timing of ear health and hearing checks in primary healthcare settings, followed by a modified e-Delphi consensus process [24], undertaken with a large, national expert panel. The use of an e-Delphi (electronic Delphi) was well suited to the panel size and nature, providing members with a more flexible timeframe for participation across several time zones. Figure 1 provides an overview of the process.

The project team comprised Aboriginal and non-Indigenous researchers experienced in the provision of clinical care and/or health-related research with Aboriginal and Torres Strait Islander communities, and included a general practitioner, public health researchers, and clinician-researchers from the ear health and hearing sector. A research librarian assisted in the design of the literature search strategy.

A 22-member working group was established for the project. Invitees to the group were people with cultural, clinical, and/or research expertise in primary healthcare or in Aboriginal and Torres Strait Islander children’s ear and hearing health. Members were Aboriginal and/or Torres Strait Islander (46%) and non-Indigenous, and over half (59%) worked in primary health settings. Four meetings were held with the working group during the project, during which members provided guidance on research questions for the scoping review, establishment of the expert panel, interpretation of research findings, resolution of lack of consensus, and formulation of the final recommendations. Meetings were held online. All 22 members attended at least one meeting and an average of 13 members attended each meeting. Six members attended 3 or 4 meetings.

The systematic scoping review addressed the research questions shown in Table 1. The protocol was registered on the Open Science Framework (https://​osf.​io/​259ae/​), an open-source web platform that facilitates open collaboration in science research [25].

Evidence relating to the total Australian population, and to Aboriginal and Torres Strait Islander peoples specifically, was in-scope, as well as evidence from international populations, particularly First Nations, that may be applicable to Aboriginal and Torres Strait Islander peoples. Evidence relating to newborn hearing screening and school-based hearing screening was outside the scope of the review.

Published studies, including meta-analyses, systematic reviews, primary studies, relevant guidelines, and health sector resources were reviewed for evidence on ear health and hearing assessment activities in primary healthcare, for information on effectiveness of these activities where available, and on the timing of the activities. The quality and certainty of the evidence was assessed using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework for primary studies [26] and the AGREE-II for guidelines [27]. Risk of bias was assessed using QUADAS-2 for primary studies [28] and ROBIS for the systematic review [29].

The scoping review search strategy focused on research questions 1 and 2 and was developed in collaboration with a librarian. The search was structured using the concepts ‘children under 6 years of age’ AND ‘ear and hearing screening tools’ AND ‘primary health care settings’. A comprehensive number of electronic databases was searched: Medline (Ovid), Embase (Ovid), Cochrane Database of Systematic Reviews (Ovid), Cochrane Central Register of Controlled Trials (Ovid), CINAHL (EbscoHost), Scopus, and nine health related indexing databases on Informit. The electronic database searches were performed by the librarian on 14-16 July 2021. A publication date limit of 2000-current and journal article limit were applied in Informit databases only. Search terms are available as Supplementary information. Websites targeted for grey literature included Australian state and territory health departments, Australian Indigenous HealthInfoNet [30], Lowitja Institute [31], and Clinical Guidelines [32] .

Extracted data included: year, authors, publication title, research question or study purpose, study design, settings, age of participants, sample size, screening test (sensitivity, specificity), level of evidence and quality, and feasibility requirements. Two authors extracted data from all included studies into an Excel spreadsheet, with other authors double-checking a sample of the extraction. Any disagreements were resolved through discussion with a third reviewer. We used the Covidence systematic review management software to manage the data arising from our searches [33].

In addition, relevant guidelines, policy documents, and parent/caregiver resources published by Australian state and territory health departments assisted with answering questions 3 and 4. These documents were either accessed by web search or shared directly with the research team upon request.

A large, national expert panel was established for the consensus-building (e-Delphi) phase of the project. Recruitment was strategic, and occurred as follows:

Recruitment ceased once there was approximately proportional participation from all Australian geographical and remoteness areas, from community controlled and mainstream primary healthcare sectors, and of Aboriginal and/or Torres Strait Islander experts from as many sectors, professions, jurisdictions, and remoteness areas as possible. In addition to the 22 working group members, 57 (of 69) invitees accepted the invitation to join the expert panel.

Two rounds of a modified e-Delphi survey were developed by the project team. The first survey included background and rationale for the development of recommendations, clarification on intended users and target patient population, and on the scope of the recommendations. Seven goals of routine Ear Health and Hearing Checks, informed by the evidence review, were presented to the expert panel. Participants were invited to indicate on a 4-point Likert scale [34] the extent to which they agreed with each goal, or select ‘I don’t feel qualified to answer’ or ‘I need more information to be able to decide’. The panel was also invited to comment on the goals and to propose additional ones. Further, eight draft Ear Health and Hearing Check recommendations were presented. Each was accompanied by a summary of the evidence, grading of evidence quality, as well as a rationale for the recommendation. Survey participants were asked to indicate, also on a 4-point Likert scale, the extent to which they agreed or disagreed with each recommendation or select ‘I don’t feel qualified to answer’ or ‘I need more information to be able to decide’, and invited to comment on the draft recommendations. Panel members were also asked to rate, on a 4-point Likert scale, how feasible they felt the recommendation would be to implement in a primary healthcare setting. The goals and recommendations were reviewed and revised following this feedback and presented again to the panel in Round 2, with a summary of Round 1 findings and rationales for revisions.

Study data were collected and managed using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at the National Acoustic Laboratories [35, 36]. REDCap is a secure, web-based software platform designed to support data capture for research studies. At the commencement of the consensus-building process, all (79) expert panel members received an email with an embedded REDcap link that allowed access to the survey, participant information, and the summary of evidence. Panel members were asked to keep the link confidential. Information on how the privacy of participants would be protected and how to raise concerns was set out in the email text. They were also given an estimate of completion time (up to 60 minutes), and instructions for completing the survey in stages. They were given six weeks to complete the survey. Members were reminded that participation was anonymous and voluntary, that selecting ‘next’ at the end of the introduction page of the survey would indicate their informed consent, and that they could withdraw at any time by closing the survey. Periodic email reminders were sent during the response period, which included progress updates on number of surveys completed. Following survey closure, level of consensus was calculated based on ‘Agree’ and ‘Strongly agree’ responses. Comments were collated and analysed thematically by the research team using an inductive approach, adhering to processes outlined by Braun and Clarke [37, 38]. The themes generated were presented to the expert panel in the following round of the e-Delphi.

Round 2 of the e-Delphi was administered in the same way. All expert panel members were invited to take part in the second round. Throughout the second survey, links enabled access to the supporting information provided for Round 1 and to the summary of findings from Round 1, both also available as pre-reading. Rationales accompanied each revised recommendation.

By the end of Round 2 of the e-Delphi, consensus was reached for all goals and all but one recommendation which related to the inclusion of audiometry in the checks. The outstanding recommendation was resolved after two meetings, and a follow-up email discussion, with working group members. Subsequently, all expert panel members were invited to one of two feedback meetings, at which the final recommendations were presented. At these meetings, panel members were invited to consider and discuss barriers and enablers to implementation into practice in primary healthcare. Twenty-three members attended the first feedback meeting, and 14 members attended the second; 32 members attended at least one meeting. All panel members received a copy of the information presented at these meetings, as well as a collated summary of the barriers and enablers put forward by panel members during the two meetings.

All methods were carried out in accordance with relevant ethical guidelines and regulations [39, 40]. Expert panel members were provided with information on context and aims of the study, how their privacy would be protected, how to raise concerns and contact the project team for further information. Members were informed how to indicate consent and of their rights in relation to participation and withdrawal.

In relation to ethical considerations when undertaking research relating to Aboriginal and Torres Strait Islander people and communities, we demonstrated spirit and integrity through ensuring that Aboriginal and Torres Strait Islander technical, clinical, and cultural knowledge were embedded throughout the project. We demonstrated credibility of intent through respectful engagement with Aboriginal and Torres Strait Islander people and organisations and ensuring equity of voice in project design and implementation. Team members recognised Aboriginal and Torres Strait Islander understandings of social and emotional wellbeing, the interconnectedness of health, place, and culture, the destructive impacts of colonisation on individual and collective social, emotional, and physical wellbeing, and the strengths of communities, families, and individuals. The project was informed by the National Health and Medical Research Council’s guidelines on ethical conduct in research with Aboriginal and Torres Strait Islander peoples and communities [40], the NSW Aboriginal Health and Medical Research Council’s health ethics key principles [39], and the CREATE checklist for assessing the quality of health research from an Indigenous perspective [41]. All participants in the e-Delphi consensus process provided informed consent.

Ethics approvals for the e-Delphi study were received from the Aboriginal Health and Medical Research Council (NSW) (1858/21), the Western Australian Aboriginal Health Ethics Committee (HREC1108), the Aboriginal Health Research Ethics Committee (SA) (04-21-944), the Menzies School of Health Research (NT) (HREC 2021-4137), and the Hearing Australia Human Research Ethics Committee (HAHREC 2021-07).

In total, 26 studies were included in the review. A systematic database search (undertaken by JH, JS, and NO) identified 22 studies published between 2000 to 2021, ten of which were subsequently excluded because of an incorrect target condition, setting, workforce, or lack of clarity in the presentation of findings. Furthermore, researchers (SH, VM, CK, MW, NO, IO) identified 25 studies via other methods, 11 of which were excluded for reasons relating to target condition or workforce, comparator, or level of directness (Fig. 2).

The 12 remaining studies compared tests against an established standard (usually standard otoscopy and audiometry). Most studies were non-randomised comparative studies. One study was a randomised controlled trial [42], and one paper was a systematic review [43] (see Table 2). Two studies concerning ear health and hearing screening related to Aboriginal and Torres Strait Islander populations [44], and seven related to other Indigenous populations or to low resource settings, including South Africa [45], Kenya [46], and India [47]. Heterogeneity among the studies related to study populations, study designs, and healthcare settings. Table 2 provides an overview of the included papers.

In the systematic review, six versions of the whispered voice test were evaluated against audiometry to determine test accuracy in detecting hearing impairment in adults and children. Sensitivity in the childhood studies ranged from 80% to 96% and specificity ranged from 90% to 98%. The systematic review authors concluded that the whispered voice test was simple and accurate but raised concerns about reproducibility, particularly in primary care settings.

The certainty of the evidence of the above studies was assessed using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework (VM, CK, MW, IO, SH). In general, the quality and certainty of the evidence reviewed on efficacy of common ear health and hearing check procedures (e.g., tympanometry) in primary health-like settings or by primary health-like practitioners was ‘low’ or ‘very low’, due to inadequate study designs for testing efficacy of assessments and/or the relevance of the study to the current review (e.g., small sample non-randomised studies and indirectness – such as different healthcare settings or different target populations). GRADE ratings by Ear Health and Hearing Check domains are included in Supplementary Material. The risk of bias of each included study was assessed by two independent researchers using QUADAS-2. Overall, high or unknown risk of bias was found in most studies, and particularly in studies relating to tools for screening ear health. See Supplementary Material for GRADE ratings by Ear Health and Hearing Check domain and QUADAS-2 risk of bias ratings for each included study. A limitation of the overall study evidence is that, although most studies included practitioners who may work in primary healthcare, few of the studies were carried out in primary health settings.

In addition to the 12 primary studies and reviews, eleven guidelines published between 1998 and 2020 relating to screening for otitis media and OM-related hearing loss in children were reviewed for information on components of ear health and hearing screening checks (Table 3). The quality of the reviewed guidelines was appraised using the AGREE II tool. Four researchers (VM, CK, IO, MW) independently rated the guidelines and weighted five items felt to be of most relevance to this study and came to a consensus. Weighted items were 12, 15, 18, 19, and 20 in the ‘Rigour of development’, ‘Clarity and presentation’, and ‘Applicability’ domains. Three guidelines were subsequently excluded as they did not reach a minimum average rating of 3 for the above items. Eight guidelines were included in the review: two are Australian and relate specifically to the health of Aboriginal and Torres Strait Islander people. Refer to Supplementary material for AGREE II Guidelines ratings.

Key findings from the evidence synthesis include:

The finding that most reviewed guidelines recommend enquiring about parent/caregiver observation led to a non-systematic citation and key-word library and Google Scholar search for studies relating to efficacy of this activity [60‐65]. A review of these studies using the GRADE approach found low certainty of evidence. Although parent/carer concern (or absence) is not a good predictor of ear health and hearing status, a draft recommendation on this was put to the expert panel, as the act of asking about and following up on concerns acknowledges the importance of parent/caregiver knowledge of their children, and the primacy of their role in their child’s health and wellbeing. Additionally, when there is parent/carer concern, a proportion will be correct.

The components of ear health and hearing check activities identified in the evidence review could be grouped into four domains (see Fig. 3), which then provided a structure for the draft recommendations to be put to the expert panel.

In the absence of certain and direct evidence, the perspectives and judgement of key experts on the timing of checks was particularly valued. Strong recommendations, confirming confidence that desirable effects outweigh undesired consequences, can be developed based on limited, low-quality, low-certainty evidence and a structured, expert consensus process.

In total, 93 people were invited to the expert panel, including the 22 working group members. Of these, 79 people accepted the invitation (see Table 4). Four state/territory-level health services and nine peak bodies nominated representatives to the process. The panel included 18 Aboriginal and/or Torres Strait Islander members (23%) and 26 members (33%) working in Aboriginal and Torres Strait Islander community-controlled settings. 25% were practitioners who receive primary health referrals for children with persistent OM, including otolaryngologists, audiologists, paediatricians, and speech therapists. Four invitees were New Zealand-based, working with Māori and Pasifika communities, who experience similar ear health and hearing concerns.

During Round 1 of the e-Delphi survey, open during January and February 2022, sixty-five (82%) panel members took part. Seven goals of Ear Health and Hearing Checks for young Aboriginal and Torres Strait Islander children were developed by the project team and proposed to the expert panel: all reached consensus agreement in Round 1 (Fig. 4) and no further goals were identified.

Eight draft recommendations were presented to the panel, and five reached consensus agreement in the first round. In Round 2, the five recommendations that reached consensus were refined and re-presented to the panel for feedback on feasibility. Panel members were invited to comment on draft goals and recommendations in each Round. Comments were analysed thematically by members of the research team (IO, MW, SH) and the themes generated informed the refining and revising of the recommendations (refer to Supplementary Material). The three recommendations that did not reach consensus agreement in Round 1 were revised significantly and re-presented in Round 2 (open during March and April 2022). Fifty-one (65%) members of the expert panel participated in Round 2, and two more recommendations gained consensus agreement.

A recommendation relating to audiometry as part of Ear Health and Hearing Checks did not reach the pre-defined consensus level of 80%. In Round 1, a draft recommendation that ‘screening audiometry is not recommended as part of Ear Health and Hearing Checks in primary health settings for children aged three years and younger' reached 76.8% agreement, and a question on whether screening audiometry should be included in Ear Health and Hearing Checks for 4–5-year-old Aboriginal and Torres Strait Islander children reached 62.5% agreement. Analysis of invited comments found many pertained to audiometry use in broader primary healthcare, and not specifically to its use within routine Ear Health and Hearing Checks. Responses identified by the research team that specifically related to inclusion of audiometry within the Checks did not support such use. Analysed themes related to training and skill maintenance, time constraints, access to equipment, a lack of need to include audiometry (as other components can indicate when audiometry may be required), and the challenges and risks associated with primary health practitioners undertaking and interpreting audiometry with children aged under 5 years. In Round 2, the draft audiometry recommendation was substantially revised to ‘As part of routine Ear Health and Hearing Checks in primary health settings, audiometry is not recommended’. Once again, the pre-determined 80% consensus level was not reached (the result was 71%). Thematic analysis of comments identified some recurring themes from Round 1, as well as new themes including the limitations of single-instance audiometry for assessing children with OM, the need for clinical governance of audiometry, and that audiometry should only be required when there are indications of persistent or recurrent OM or hearing impairment. Analysis of comments made by 7 of 12 respondents who felt audiometry should be part of routine Checks identified a common theme supporting inclusion of audiometry under certain conditions: when concerns arise during the Check, and prior to starting school. After two Working Group discussions, a final consensus recommendation on the role of audiometry in the Checks could not be made. However, a general recommendation on the role of audiometry for Aboriginal and Torres Strait Islander children with OM, aligning with the Otitis Media Guidelines for Aboriginal and Torres Strait Islander Children [16] was agreed upon.

The eight finalised recommendations outline the agreed goals, components, and timing of primary health Ear Health and Hearing Checks for Aboriginal and Torres Strait Islander aged under 6 years (Table 5). Recommendations 1, 2, 3, and 7 relating to parent/carer report, listening skills, assessment of appearance and movement of the middle ear, and timing of the checks are classified as strong despite low certainty of evidence. These discordant recommendations, where strong recommendations are formed in the absence of high-quality evidence, were each felt to be justified in view of the known high prevalence and harms of asymptomatic, undetected/untreated, persistent OM in the target population [9, 11‐13, 15, 66, 67], and the high level of agreement from expert panel members, as well as meeting the GRADE criteria for a strong recommendation despite low quality evidence [68]. For recommendation 1 (asking about parent/carer concern), although the evidence relating to the benefits of asking parents/carers about signs and symptoms was uncertain, the risk of missing a persistent hearing problem, potentially leading to developmental harm, and the value of affirming parental/carer centrality in the child’s life, warranted a strong recommendation. In relation to recommendation 2, there is currently little evidence associated with the benefits of screening listening skills as part of routine primary health ear health and hearing checks; however, no other component of the check has the potential to differentiate OM persistence and flag possible auditory deprivation and associated developmental risk in isolation of the child’s clinical history. For recommendation 3 (assessing appearance and mobility of the middle ear), high certainty evidence exists for the use of pneumatic otoscopy and for tympanometry for detecting otitis media with effusion, including in primary health settings, for populations not known to experience high rates of persistent ear disease. Although there is very little direct evidence from Aboriginal and Torres Strait Islander primary health settings, given high certainty of similar benefits, a strong recommendation was formed relating to use of either simple otoscopy plus tympanometry or pneumatic otoscopy. In relation to recommendation 7 (the timing of checks), relative to current recommendations to check ears at every primary health visit, there is little-to-no evidence relevant to the benefits of either approach. However, the consensus opinion was that a pre-determined schedule of checks that reflected the natural history of OM for Aboriginal and Torres Strait Islander children and the sensitive period for development of children’s listening and communication skills would be less risky than checks undertaken at indeterminate intervals, and therefore warranted a strong recommendation.

Three recommendations (4, 5, and 8) are conditional, or relevant in certain circumstances, including those relating to the use of video-otoscopy and otoacoustic emissions, and when more frequent checks may be required. These are outlined in Table 5 and the companion paper provides further detail on the recommendations [23].