Background
Nutrition risk factors are the leading causes of the global disease burden [
1]. Consequently, United Nations member states declared 2016–2025 as the decade of action on nutrition [
2]. Dietary guidelines provide evidence-informed recommendations regarding the dietary patterns recommended for optimal health and well-being and to reduce the risk of dietary-related chronic diseases [
3]. The World Health Organization has recommended that nations implement evidence-based nutritional guidelines and policies in settings such as schools and childcare services to improve public health nutrition [
4]. School and childcare-based nutritional guidelines typically make recommendations regarding the types of foods and beverages that should be provided (or made available to children) and in quantity, variety and frequency [
5,
6]. A considerable challenge to achieving such objectives is the limited evidence base regarding the effectiveness of implementation strategies [
7,
8]. Reviews of strategies that aim to improve the implementation of nutrition policies in schools and childcare services have identified few trials and report strategies that achieved equivocal effects [
9,
10].
Understanding the causal mechanism by which implementation strategies exert their effects can improve the impact of strategies to enhance guideline implementation [
7]. Approaches to improve policy or practices consistent with guideline recommendation are often multi-strategic and target a range of intermediary factors (or mediators) that are hypothesised to be causally be linked to successful implementation [
7,
11]. The effectiveness of such strategies may be improved by retaining (or strengthening) strategies that target mediators which cause improvements in guideline implementation. Strategies could also be refined by discarding intervention components that target mediators that do not cause improvements in guideline implementation, or those that fail to shift important mediators [
7,
12]. Despite the importance of understanding mechanisms of effect, few mechanistic evaluations of implementation strategies exist, and to our knowledge, none have been conducted on trials of strategies to improve implementation of nutrition guidelines.
Recent methodological advances have developed robust analytical techniques to quantify the extent to which of intervention effects are channelled through selected mediating variables. These new methods are based on clearly outlined counterfactual definitions of causal effects along with explicit assumptions required for making causal inferences [
13‐
15]. However, conducting mechanistic evaluations of pragmatic trials in settings such as schools and childcare services is particularly challenging as guideline implementation typically occurs at the organisational level. Often, an insufficient sample of organisations is recruited to allow for sufficient power to undertake mediation analyses. There is also a lack of agreement as to which constructs should be tested as possible mediators of implementation and how they should be measured [
16].
In this exploratory study, we aimed to overcome power limitations by aggregating data from three homogeneous implementation trials in schools and childcare services and used a theory-driven consensus approach to identify key constructs that could plausibly mediate the effects of implementation strategies on nutrition policy uptake. Thus, the overarching objective of this study was to quantify the extent to which selected Theoretical Domains Framework (TDF) constructs mediate the effect of implementation strategies on nutrition policy uptake in schools and childcare services.
Discussion
This exploratory study sought to identify the mechanisms by which implementation strategies improve nutrition guideline implementation in schools and childcare services. Our aggregate causal mediation analysis found that although the implementation strategies increased adherence to policy and guidelines, none of the theoretically targeted factors (knowledge, skills, professional role and identity, and environmental resources) mediated this effect.
The implementation strategies evaluated in the included trials targeted a range of implementation barriers that were not analysed in this study. It is possible that the interventions were operating via mechanisms that were not captured by the four TDF constructs we explored. For example, previous studies have reported a range of factors associated with successful implementation of healthy eating interventions in childcare––including the support of parents, service management, and the structural availability of time and resources [
34]. These factors could have been the active mechanisms of the interventions evaluated in this study but were not captured by the four TDF constructs we explored. Previous work has posited that organisational level change often involves the interplay between many individuals, group and environmental factors [
35]. Future research should consider investigating the casual mechanisms that work through system-level factors that could impact the implementation of nutritional policies within childcare services and schools. An ecological systems framework that considers the complex interactions between individuals and their social structures may be appropriate to guide such investigations [
36]. Furthermore, the use of the Control Theory in the CAFÉ trial may have caused the intervention to work through mechanisms that were not captured by the TDF.
Another explanation for our findings is that we may have failed to measure the targeted TDF constructs with adequate precision. While the TDF questionnaire used in this analysis has some evidence of validity and reliability [
20], it has some limitations. For instance, only one of the three goodness-of-fit statistics from the original confirmatory factor analysis met acceptable criteria [
25], which indicates limitations in construct validity. Furthermore, all four constructs of the TDF were negatively skewed for both control and the intervention groups, with most organisations reporting high scores (i.e. low levels of barriers) at follow-up. The skew towards high scores on all four TDF constructs for both groups may reflect possible ceiling effects. If ceiling effects are present, it is possible that our measure was not sensitive enough to detect between group differences, as the measure cannot distinguish respondents at the upper end of the construct [
37]. Ceiling effects can attenuate statistical associations, thus resulting in a possible underestimation in the relationship between variables [
38,
39]. This may explain why we did not detect a mediating effect through these variables. Future work should seek to make improvements in the current TDF measures to allow for greater sensitivity to detect underlying mechanisms. Possible strategies that may be considered to help reduce the ceiling effects and increase the response variability of the TDF measure could include (i) using more extreme response options at the positive end of the scale, which could help differentiate people who score high. This strategy has been suggested to reduce ceiling effects in other surveys [
40,
41], (ii) including survey items that assess respondents’ actual behaviour rather than self-perceived behaviour. For example, rather than asking respondents to indicate whether they are aware of guideline content, it would be more precise to specifically assess the respondents’ actual knowledge of the guideline content. This should help provide a more objective and standardised assessment of barriers.
A strength of this study is the use of pooled data from three relatively large randomised trials in the public health nutrition setting. This was possible through the aggregation of a homogenous collection of trials (same geographic location, similar interventions, similar population and matched target behaviour). This is a key strength as it is often difficult to collect large organisation-level datasets for mechanistic evaluations. Many studies have used the TDF to guide the development of interventions; however, no study has quantitatively tested the TDF constructs as causal mechanisms to refine future implementation strategies [
42]. Building an evidence base for the mechanistic role of the TDF constructs will assist in future intervention design and adaptation [
7,
12,
42]. Research partnerships between health organisations and clinical trial units should employ similar approaches to conduct mechanism-focused implementation studies. Planning and executing a concerted set of trials that assess similar implementation mechanisms could yield robust evidence for how implementation strategies work or do not work. These techniques can and should be applied across various settings for better implementation of preventive and healthcare strategies.
Our findings should also be interpreted in the context of its limitations. We were unable to adjust for any confounders of the mediator-outcome effect. However, our sensitivity analysis indicated that our estimations of the mediation effect would remain stable even at high levels of residual confounding. We may have measured the TDF constructs with error and poor precision. It is possible that the questionnaire used to assess the TDF constructs was limited in its construct validity and displayed ceiling effects. The temporal precedence between the mediator, and outcome is unclear in our analyses. Future work should aim to measure the mediator prior to the outcome and assess the possibility of reverse causation. The trial participants may have felt under pressure to report excellent knowledge and skills after training (social desirability bias). However, given that we did not detect between group differences on knowledge and skills, the likelihood of this bias is low. Lastly, 19 (16%) organisations were lost to follow-up, and this may have induced bias if the missingness mechanism was not at random.
Acknowledgements
We thank Li Kheng Chai for providing risk of bias assessments of the included trials.