Are current scoliosis school screening recommendations evidence-based and up to date? A best evidence synthesis umbrella review

Comprehensive literature searches and selection of retrieved papers (Box 2, Fig. 2, Table A2—supplementary material) resulted in including six systematic reviews [1, 3, 16, 18, 30, 32], of which five [1, 3, 16, 18, 30] were conducted in the process of systematically developing recommendations. Full text analyses of a further 34 papers resulted in excluding 15 reviews addressing the subject matter, but not fulfilling the minimal criteria for a systematic review. Nineteen full texts were also excluded for other different reasons (Fig. 1, Table A2—supplementary material). Therefore, to our knowledge this study presents the first comprehensive and in-depth analysis of systematic reviews regarding screening programs for adolescents for idiopathic scoliosis.

Overall, the quality of systematic reviews ranged from the comparatively recent (2009) moderate quality (AMSTAR score 6) Fong et al. [32] and Sabirin et al. [16] through to the outdated (2002) moderate quality NHMRC, reviews [3], to the poor quality recent (2011) UK NSC [1] and outdated (2004, nonetheless used for recent (2012) recommendations [24, 25] ) USPSTF [30] and Italian (2005) [18], reviews.

The reviews which supported the recommendation for school screening [16, 18, 32] as well as those recommending against screening [1, 3, 30] are based on different papers selected for inclusion (Figs. 2, 3, 4, 5) [and thus on different evidence or assumptions of the absence of evidence (Table 2)]. Further, regardless of the different time limits used within searches, the reviews also differed significantly with regard to the databases selected, and other resources, searched (Table 2). Moreover, significant heterogeneity was found within the reviews as follows: different research designs were considered (prospective trials and retrospective observational studies, systematic reviews, editorials), which were, except in the Fong et al. [32] meta-analysis, analyzed separately (Table 2). Conclusions were based on different criteria as follows: the set of criteria for appraising—feasibility, effectiveness and appropriateness of a screening program [1, 3], accuracy of screening tests [16, 18, 32], treatment effectiveness as a criterion justifying the need for screening [16, 18, 30] and cost-effectiveness [1, 16].

Three of the systematic reviews were found to be of moderate quality; Fong et al. [10], Sabirin et al. [16] and NHMRC [3].

Fong et al. [32] (2010) reported that only 17 % of the primary studies included within their meta-analysis of retrospective cohort studies found screening to be ineffective. The authors advocate for school screening, but recommended that the forward bend test should not be used alone within screening programs and that large, retrospective cohort studies are needed to better study the reliability of screening tests. As regards the studies included for analysis, the results of the Fong review only overlaps partially with the Sabirin et al. [16] study, but shows no overlap with the UK NSC 2011 review [1] (Fig. 5) and the older reviews reported by the USPSTF [30] (Fig. 2) and NHMRC [3] (Fig. 3). The conclusions from the Fong study were in part based on different evidence as well as different assumptions to the other reviews. Whilst the AMSTAR score for this meta-analysis is only of moderate quality (6), the report does meet the essential criteria of a good quality quantitative systematic review (Tables 2 and 3).

MAHTaS (2009) systematic review by Sabirin et al. [16] included a methodological assessment of individual studies as well as a cost–utility analysis. The authors recommended continuation of the screening program in Malaysia, under specific conditions: in high-risk group of 12-year-old girls, using combination of screening tests, and with well-secured training, manpower, good referral system, treatment and funding. The review was clearly reported (AMSTAR score of 6 (Table 3); however, we encountered difficulties with regard to the assessment of question 8 within AMSTAR: unfortunately, only the results of 8 out of 24 papers appraised, utilizing the CASP tool were reported. Furthermore, studies of different types designs were included and analyzed separately (Table 2), which in some instances may diminish the internal validity of the review as well as the strength of the conclusions.

The NHMRC (2002) document [3] recommends against screening for scoliosis. All the criteria for conducting a systematic review are described in detail. Moreover, unlike the other reviews evaluated, a defined methodology for the quality assessment of systematic reviews was provided. Nonetheless, the meta-analysis included in the systematic review of evidence regarding screening for scoliosis [55], and 8 out of 12 included primary studies, were only discussed descriptively (Table 2), without conducting a critical appraisal of their quality. Moreover, several other eligible primary studies were not included for analysis (Fig. 4).

The NHMRC report has not been updated since 2002, while the AMSTAR does not reflect the recency of the analyzed reviews. Also, the NHMRC review [3] was not developed according to the subsequently (2009) improved NHMRC evidence hierarchies [56].

In fact, most of the six reviews analyzed matched neither the improved (2009) NHMRC [56], nor the new (2011) Oxford Centre for Evidence-Based Medicine (OCEBM) levels of evidence hierarchy (Levels of Evidence 2 [57]), with the exception of Fong et al., which can be classified as a level 3 evidence in the OCEBP classification (Box 3).

The UK NSC review (2011) [1], prepared as an update of the 2004 USPSTF analysis [30], sustains the recommendations against screening. The report scored 4 with AMSTAR (low quality review, Tables 2 and 3). The authors addressed several criteria regarding the mandate for a screening program, including the cost-effectiveness of the programs, as well as the effectiveness of conservative and surgical treatment. The observational, experimental studies as well as the systematic reviews included within this paper discussed diverse aspects of scoliosis management and are described broadly and narratively. Eligibility criteria for paper inclusion are not reported. The lists of 470 retrieved reports [45‐47] supplement the review, which factually discusses 53 papers but no list of excluded studies is provided. The review reports on search limits from 2002 to 2011, but single older studies are also included. Thus, it would be difficult to judge how the studies described in the review were chosen. Further, while the hierarchy of evidence level is suggested—the designs of the discussed papers are reported, crucially the included studies are not critically appraised. Without knowledge of the methodological quality of included papers it is very difficult to come to a conclusion as to whether the results of this review can be applied to practice and whether the recommendations are truly valid and evidence-based.

The USPSTF resource review for the 2012 USPSTF recommendations against screening [24, 25]—the 2004 “Brief Evidence Update” [30], recommending against school screening—has been shown to be of poor quality, with an AMSTAR score of 2. The “critical” “key question”—“Is there new evidence that scoliosis treatment leads to better health outcomes if applied at an early stage”, was answered by analyzing one RCT, three reports of a large retrospective observational studies, one case series and a meta-analysis of studies from between 1975 and 1995 on the effectiveness of conservative treatments [55]. No list of excluded studies was provided, and crucially no methodological quality appraisal of their included studies was conducted. Further, the search period is outdated (1994–2002) and the literature search was not comprehensive (Fig. 4). Consequently, this review scored a low value of 2 with AMSTAR (Table 3). Whilst the USPSTF did elaborate on the development process for their recommendations, including the process of the evidence review [27, 28, 43, 44], this development process does not seem to have been followed in their study (Tables 2 and 3). Nonetheless, when appraising the review, we considered the USPSTF methodology descriptions current for that period of time, described elsewhere [27, 28, 43, 44] (Table 3). Otherwise, the review would have attained an even lower score. As the review was published in 2004, the present USPSTF methodology of guideline development, improved subsequently in 2007 and 2008 [58‐60], was not analyzed. It also means that the current (2012) USPSTF recommendations against scoliosis screening are based on the outdated methodology of the USPSTF guideline development.

Finally, in the USPSTF “review of the process” paper the authors say that comprehensive systematic reviews are “long, detailed reports of interest to a minority of readers and of limited value to busy clinicians” [44]. It is difficult to agree with this notion, as researchers, clinicians, commissioners and service users routinely make use of conclusions/recommendations from systematic reviews (e.g. Cochrane reviews with separately published summaries). In fact, systematic reviews are considered to be equivalent to research studies that produce the highest level of evidence according not only to OCEBM [57] and NHMRC [56] but to numerous other organizations. Additionally, the detailed guidelines for reporting explicit systematic reviews of different types of studies have been in use (such as QUORUM (1999) [61], then PRISMA (2009) [38], MOOSE (2000) [62], and reporting guidelines from the US Institute of Medicine (2011) [63]).

The authors of the Italian guidelines (2005) [18], recommending for school screening, reported on a systematic analysis of the literature. It achieved an AMSTAR score of 2 (Table 3). A classification of the levels of evidence is provided in this paper, including expert opinions. Within these recommendations the evidence for scoliosis screening was classified as “E2: fair scientific consensus” [18]. In brief, the document provides a recommendation on screening for scoliosis driven from a consensus of a panel of experts, rather than on a systematic analysis of available evidence.