Process evaluation of school-based high-intensity interval training interventions for children and adolescents: a systematic review and meta-analysis of randomized controlled trials

A comprehensive search for the relevant literature was conducted in SPORTDiscus (EBSCOhost), Web of Science, Scopus, Medline (Ovid) and Cochrane Central Register of Controlled Trials from inception to March 2022. Search strategy was formulated based upon the guideline of Peer Review of Electronic Search Strategy (PRESS) [41] and was checked with an information specialist before commencement. The full search strategy is available in Additional File 2.

Upon removal of duplicates on Endnote (Clarivate Analytics, Philadelphia, USA), two reviewers (YL and CW) independently screened the titles and abstracts against the inclusion and exclusion criteria in a blinded manner on Rayyan [42]. Subsequently, a discussion was organised to compare and reconcile the independent screening results, reaching a consensus on the papers to undergo a full-text review. This process was reiterated via another round of independent screening and discussion to complete the selection. Disagreements were resolved by discussion with two additional authors (CAW and ARB). The references cited in the included studies were manually checked for identifying additional eligible studies and an updated search was made in November 2022. Whenever not enough information in the manuscript for deciding, an inquiry email was sent to the authors for clarification. If the authors did not reply, these studies were listed as awaiting classification and were excluded if no responses were received after a second inquiry attempt at least fourteen days apart. These excluded papers are presented in Additional File 3.

A data extraction sequential list (Table 1) was predefined in case multiple measurements of the same outcome emerged. The rationale for the prioritisation was based upon measurement properties (e.g., validity and reliability) and popularity of the measurements. Data extraction on process evaluation measures was guided by the Medical Research Council (MRC) process evaluation framework [24], including domains of implementation, mechanisms of impact, and context. The framework is elaborately conducted, providing a systematic, comprehensive, and exhaustive process evaluation review [25, 30]. A process evaluation framework by Ma et al. [30] was referred to adapt the framework into practice. The process evaluation measures were predefined in Table 2. Of note, although defined as session quality (e.g., attendance and dose received) and intensity in school-based HIIT studies [37, 43], fidelity was solely represented by intensity in the current review, as session quality was reported separately as other process evaluation measures (e.g., dose delivered).

The following data were extracted: (1) key study characteristics (e.g., author name, publication year, participant characteristics, intervention details, and sample size); (2) process evaluation measures (implementation, mechanisms of impact and context); (3) post intervention (the closest to intervention endpoint) outcomes and results, including sample size, mean and standard deviation (SD) in intervention and non-exercise control groups. Where trials reported 95% confidence intervals [33, 50] or median and interquartile range [53], these were converted to means and SDs using established methods [54]. In addition, mean change values [50, 55] were extracted when post-means and SDs were not reported, and data were extracted from figures, via GetData Graph Digitizer, when not reported numerically [35]. Data extraction forms were developed, piloted, and refined through discussions across the authorship team. The extraction was conducted by YL and partially (30%) checked by CW for accuracy.

The Cochrane risk-of-bias tool for randomized trials and revised Cochrane risk-of-bias tool for cluster-randomized trials [56] were adopted for quality assessment of RCTs and cluster-RCTs, respectively. Risk of bias for the outcomes of CRF, body composition and muscular strength were assessed separately from the following five domains: randomization process, intended interventions, missing outcome data, measurement of outcome and report of results by answering signalling questions. The judgments for each domain were summarised as “low risk of bias” “some concerns” or “high risk of bias”. An overall risk of bias judgement was reached through algorithms that map responses to signalling questions. The first author (YL) performed the risk of bias assessment, and the accuracy of the assessment was subsequently verified by a second author (CW) through a random sub-sample (30%) of studies, with less than 80% of consensus triggered a check for all the studies. Conflicts were discussed with consultation been made with either CAW or ARB whenever disputes occurred.

Process evaluation measures in relation to the included interventions were thematically assembled in line with the prescribed definition (Table 2) and were narratively reported. All the interventions were qualitatively synthesised. Random-effects meta-analyses were performed, incorporating subgroup analyses for studies with and without process evaluation, to determine the differences between the two groups concerning CRF, body composition, muscular strength and blood pressure. Where outcomes were reported using different measurement units, a standardised mean difference (SMD) effect size was reported. SMD was set as 0.2, 0.5 and 0.8, corresponding to small, medium and large effects [57]. Several sensitivity analyses were performed including a leave one out meta-analysis, the removal of high risk of bias studies, and studies that had a computed outcome score. A spectrum of intervention characteristics and process evaluation measures (Table 3) were selected and regressed to determine mediators of CRF, body composition, muscular strength and blood pressure. Heterogeneity was assessed via I² and tau² (τ²) statistic [39], publication bias was assessed via funnel plots and the Egger’s test [58]. The analyses were performed in STATA version 17 (College Station, Texas 77,845 USA) [59].

In total, 15 interventions (33%) were cluster-RCTs. When combined, over 25,104 participants, across 17 Countries/regions, were included in this review. Twenty-one interventions (47%) were conducted in primary schools, while twenty-four interventions (53%) were delivered in secondary schools. The mean age was 12.2 years (n = 38), and 7 studies provided the age range of the participants only. There were 8 and 7 interventions targeted exclusively at female and male participants, respectively. Eleven interventions examined children living with overweight and obesity. The HIIT intervention duration ranged from 2 weeks to one academic year and was 12 weeks on average. The session frequency ranged from 1 to 5 times per week, with 3 (24 interventions, 53%) or 2 (14 interventions, 31%) sessions per week being most frequently reported. The HIIT sessions were as short as 4 min to as long as 43 min in total length. The work and rest intervals varied significantly, ranging from 10 s up to 240 s, with more than half of the interventions (n = 27) adopted a work interval less than 30 s. Additionally, the work-to-rest ratio was different from interventions, with the ratio = 1 (n = 17), < 1 (n = 5), > 1 (n = 9) or varied (n = 14). Twenty interventions (44%) adopted the traditional running or cycling modality. The details of all the intervention characteristics are summarised in the Additional File 4.

No conflicts were found between the two authors regarding the risk of bias assessment of the 30% sub-sample. The details of risk of bias assessment for CRF, body composition and muscular strength are presented in Additional File 5 and 6, and are briefly displayed in Figs. 2, 3 and 4, respectively. Of the 30 studies reporting CRF, 5 (17%) studies were assessed as “low risk”, 11 (37%) “some concerns” and 14 (46%) “high risk”. For body composition outcome studies, 3 (14%) were rated as “low risk”, 8 (36%) “some concerns” and 11 (50%) “high risk”. For muscular strength outcome studies, 3 (23%) were assessed as “low risk”, 5 (38%) “some concerns” and 5 (39%) “high risk”. The major reasons for raising the concerns were: 1) lack of proper randomization; 2) no blinding; 3) not accounting for missing data; and 4) lack of pre-determined protocol.

Table 4 summaries the process evaluation measures across the included interventions. In total, 4 interventions labelled and nested process evaluation in a section of the paper [33‐36] and one intervention reported process evaluation in a separate paper [37]. Although not labelled as “process evaluation”, two interventions published a separate paper to evaluate and reflect on the implementation process [60] and to identify the facilitators and barriers across the intervention delivery process [61]. On average, half of the process evaluation measures (n = 6) were reported upon, and most interventions (n = 43) reported on multiple process evaluation measures. Implementation was the most frequently reported domain (59%), followed by mechanism of impact (42%) and context (34%). The detailed information of how each process evaluation measures were met is presented in Table 5.

A total of 33 studies were included for pooled random-effects meta-analyses and meta-regressions, with subgroup comparison between studies with and without process evaluation. In total, eight studies were appraised as fulfilling implementation process (intended to report process evaluation measures) and were allocated to the studies with process evaluation subgroup.

In total, 77 studies from 45 school-based HIIT interventions were identified, with an average of 6 out of 12 process evaluation measures being reported. Five interventions (11%) explicitly labelled and reported process evaluation in either a section of the paper or summarised in a separate publication. However, most of them were atheoretical except one intervention [37] adopted a framework by McKay et al. [121]. Overall, half (6/12) of the process evaluation measures were reported on average across all the interventions, with implementation being the most frequently reported domain (59%), followed by mechanism of impact (42%) and context (34%). The current study did not identify any favourable associations between studies intended to report process evaluation and the intervention outcomes in terms of CRF, body composition and muscular strength, and neither did reporting any of the process evaluation measures elicit better treatment effects for these health parameters. Rather, the overall pooled studies and those studies without deliberately reporting process evaluation were found to have beneficial effects for CRF, body composition and muscular strength, despite the studies without process evaluation group being characterised by higher heterogeneity and risk of bias. These findings indicate that process evaluation elicits no salient potentiation to the intervention effectiveness. However, it is undeniable that process evaluation completes the outcome-oriented RCTs of school-based HIIT interventions by providing sound implementation details, exploring potential mechanisms of impact and clarifying context factors. Hence, the understanding of the intervention effectiveness, generalizability and transferability are enhanced through the “lens” of process evaluation. Nevertheless, process evaluation has been largely neglected and under-reported, which may have potentially tempered the value of the existing school-based HIIT interventions.

Fidelity is the key to process evaluation and is defined as the degree of the intervention being delivered as intended by multiple frameworks [24, 121‐123]. In the present review, it was represented solely by HIIT intensity. Of the 45 included interventions, the majority of studies (84%) monitored HIIT intensity. Notwithstanding, the prescribed intensity was monitored in various ways, and to some extent, in an incomplete or invalid manner. First, although HR monitors provide an objective measure of HIIT intensity, failing to report the actual HR data can compromise intervention fidelity and leave the mechanisms underlying intervention effectiveness ambiguous. Likewise, MAS is another way to prescribe intensity, however, not without the premise of articulating how to individualise the running speed. Second, RPE enables a simple and convenient way for intensity monitoring. Even so, notably, RPE is not yet validated in the context of school-based HIIT interventions. Interestingly, VPA (as measured via accelerometery) has emerged as a means of gaging HIIT intensity in two studies [35, 43], which ushered in a new direction and consideration for determining HIIT fidelity. Both studies aimed to assess intervention fidelity by comparing accelerometer determined VPA between the HIIT and control groups, with one study [35] reporting significantly greater amount of VPA time in the HIIT group, while the other [43] reported nonsignificant findings. However, neither study established a predetermined standard for ‘high intensity’ using VPA, despite fidelity being defined as intervention delivered as intended [24, 121‐124]. Future studies are recommended to thoroughly consider the pros and cons of HIIT monitoring tools before incorporating into their study designs. These considerations should include factors such as precision, affordability, validity and time commitment. Additionally, the pursuit of more efficient tools for prescribing and monitoring HIIT could be a future research endeavour. Our research team has recently demonstrated the validity of utilising RPE, as evidenced by Liu et al. [125], and session RPE, as illustrated by Duncombe et al. [126], for monitoring HIIT in laboratory and school settings, respectively. Furthermore, a call is claimed to fully report the HIIT intensity data and to continuously working on developing convenient and feasible measurements for establishing HIIT fidelity, especially for large-scaled studies [18].

The current review found that the common reasons for dropping out were absence in testing days, illness, lack of time and school transfer. High level of attrition may lead to biased intervention effects [53], therefore, where possible, measures should be taken to motivate and retain participants or by using appropriate statistical analysis methods (e.g., intention-to-treat). Several strategies were applied to do so in the studies included in the present review, including providing a flexible intervention schedule [63], rescheduling the missed sessions or tests [47] and offering choices [35, 111] or rewards [36, 105, 119]. These are practical solutions for researchers to boost future “buy-in” of potential stakeholders. Since no “one size fits all” approach to the study design, adaptation(s) may be necessary at times. Yet, it was the least reported process evaluation measures. Findings suggested that the purposes for making the adaptation(s) were to: 1) adjust intensity (e.g., introduce new rules to avoid participants staying still [34]); 2) to ease implementation (e.g., substitute HR monitors with RPE for monitoring intensity [48]); and 3) to make a compromise (e.g., re-schedule sessions due to busy curriculum [47]). The current study demonstrated that HIIT is generally safe for children and adolescents in view of the fact that only minor injuries (bruises and strains) [86] and dizziness (due to blood sampling) [34] were reported. This was in accordance with previous reviews which demonstrated that HIIT is safe to be applied in children and adolescents [11, 127]. While dose delivered was reported by all the interventions, half of them (n = 23) were not clear on how many doses were received by participants, overshadowing the quality of implementation and the understanding of effectiveness. Although barriers and facilitators were underreported, the existing information sheds some light on future interventions. Based on the qualitative reporting of the included studies, it appears that busy curriculum [43, 50], lack of time [36, 111] and inconvenient equipment use [35, 105] were frequently reported as barriers to implementation, whereas training workshops [44, 48, 113], incentive strategies [105, 113] and theoretical instructions [37, 50] were reported as effective boosters. In addition, findings of the present review suggest that short, simple, enjoyable interventions that do not heavily rely on equipment may be better suited to meet the needs of stakeholders [48, 119].

Nesting process evaluation within RCTs enables a comprehensive and lucid description of both the process and outcome evaluations, thereby facilitating the replication and synthesis of evidence [24]. Nevertheless, it remains unclear whether process evaluation leads to better effects in school-based HIIT interventions. The current review revealed that better reporting of process evaluation posed no potentiation to the outcomes of CRF, body composition and muscular strength in school-based HIIT interventions. Despite the context differences (e.g., setting, outcomes), findings in this review were contrary to those of previous studies. Seral-Cortes et al. [31] found that reporting of process evaluation measures was associated with significantly decreased BMI. Similarly, Ma et al. [30] claimed that the inclusion of a process evaluation aim tended to benefit the overall effectiveness of motor competence programmes.

Several explanations have been proposed for a better understanding of the counter intuitive findings. First, both studies with and without process evaluation were no longer significant with respect to CRF and body composition improvement after removing high risk of bias studies. This observation suggests that some of the high risk of bias studies might have distorted the overall effectiveness. Indeed, previous work has shown that higher risk of bias is associated with exaggerated (approximately 10%) treatment effects [128]. Given that 21/23 and 17/19 interventions in studies without process evaluation group for CRF and body composition respectively were appraised as either high risk or some concerns (see Figs. 2 and 3), there is a possibility that the effect size of these two outcomes in studies without process evaluation may have been overestimated.

Second, the studies included in this review had a disproportionately larger number of participants in the studies with process evaluation group (n = 8, participants = 17,774) compared to the studies without process evaluation group (n = 25, participants = 2,552), which may have contributed to the discrepancy in intervention effectiveness. Indeed, it is reported that when interventions are conducted at larger scales, they may experience a 'scale-up penalty' or 'voltage drop', where the effectiveness of the intervention diminishes due to adaptations made to accommodate the contexts [129, 130]. In agreement with this, the current review revealed that reporting of adaptation is associated with lower improvement in CRF (β = -0.34, p = 0.05) compared to studies that did not report adaptation. Third, studies with process evaluation group were mostly cluster-RCTs and utilised exercise modalities other than running/cycling, which were associated with lower improvements in CRF and body composition compared to the studies without process evaluation (Tables 6 and 7). It is, therefore, conceivable that process evaluation is unlikely the only reason, or not responsible, for intervention ineffectiveness. Fourth, it is worth considering that process evaluation may be independent of outcome evaluation if the intervention is conducted without a prescribed process evaluation aim and without the support of a process evaluation framework. This is supported by Ma et al. [30], who found that including a process evaluation aim tended to benefit the overall effectiveness of motor competence programmes. Thus, it seems plausible that process evaluation did not contribute to the outcome assessments in the current review since only one study [37] adopted a process evaluation framework. Nevertheless, future studies are encouraged to further explore the relationship between process and outcome evaluation.

The current review showed that studies conducted among overweight and obese cohort were associated with favourable intervention effects on CRF (β = 0.66, p = 0.01) and body composition (β = -0.93, p = 0.01) compared to studies without specifying weight status. This is in agreement with previous reviews [18, 22], which have demonstrated that overweight and obesity significantly mediates CRF, waist circumference, percentage body fat and BMI in school-based HIIT interventions targeting children and adolescents. Hence, HIIT may be a particular effective and beneficial form of exercise for this “at-risk” cohort [131, 132].

Previous school-based HIIT reviews have shown that running- or cycling-based HIIT was the most adopted modality [18, 20, 22]. Despite this, our review is the first to systematically compare the differences between the traditional running/cycling HIIT and other HIIT modalities (e.g., resistance-based HIIT). The pooled evidence suggested that running/cycling HIIT was superior in improving CRF and body composition, but not muscular strength, compared to other modalities. This finding corroborated the speculation made by Costigan et al. [11] that cycling/running-based HIIT was likely to improve CRF, rather than muscular strength, due to the lack of training specificity.

The current review found that shorter intervention duration elicited higher CRF improvement. This is consistent with Leahy et al. [133], where they concluded that shorter intervention duration emerged as a better predictor of well-being. In addition, Ma et al. [30] suggested that a shorter intervention duration led to greater effects on motor competence, proposing that longer interventions were more susceptible to interruptions and less supported. This assertion is supported by some of the studies included in the current review, indicating that extended interventions are associated with high drop-out rates and diminished implementation quality (e.g., inadequate exercise intensity monitoring and reduced dose delivery) [43, 53].

Interestingly, the pooled HR data revealed that lower HR was associated with significantly higher improvement in CRF, whereas no effects for body composition and muscular strength were observed. However, given the relatively low effect size (β = -0.01, p = 0.01), we speculate that HR induces no effects on the intervention outcomes once a certain level is reached (above 70% in the current review). This is supported by McKay et al. [134] and Schaun et al. [135] in which they stated that the magnitude of CRF would not be influenced by intensity once it is above 60% VO_2max, despite their conclusion being based upon young adults.

This is the first review exploring the implementation process of school-based HIIT interventions in children and adolescents. The comprehensive literature search and combination of all possible outcomes regarding CRF, body composition and muscular strength contribute to the complete and overarching findings of the current school-based HIIT review. There are some limitations within this review. Since few studies had specified the process evaluation measures, most of the measures are subjectively assessed by authors. Although a second author in our review has checked for the accuracy and consistency of the judgements, disagreements may still exist from a reader’s point of view. To minimise author bias, the current review adopted the MRC process evaluation framework [24], combined with study by Ma et al. [30] for process evaluation. However, currently there is no single definition of process evaluation. The results may be different if another framework is adopted. Nevertheless, the MRC framework is probably one of the most comprehensive process evaluation guidance in the literature [136]. In addition, the preferential orders of data extraction for outcome variables in Table 1 is based on the understanding of authors in the present review. Consequently, it is essential to admit that these decisions may involve nuanced considerations. Finally, the results for all the outcomes showed significant methodological and statistical heterogeneity and publication bias, therefore future studies should interpret these results with caution.

The sentence “cardiorespiratory fitness (20 m shuttle run in laps finished), body composition (BMI), strength (push-ups in times and standing long jump in meters) and blood pressure (mmHg)” in the protocol was changed to “cardiorespiratory fitness (e.g., 20 m shuttle run in laps finished), body composition (e.g., BMI), strength (e.g., push-ups in times and standing long jump in meters) and blood pressure (mmHg)”. In addition, the initial process of risk of bias assessments, conducted by two independent authors (YL and CW), was modified. The first author (YL) performed the risk of bias assessment, and the accuracy of the assessment was subsequently verified by a second author (CW) through a random sub-sample (30%) of studies, with less than 80% of consensus triggered a check for all the studies.