Development and use of an index for measuring implementation of a weight management program in children in primary care clinics in Texas

Over one-third of children in the United States have overweight or obesity, with more Hispanic, non-Hispanic black, and children from low socioeconomic conditions being affected than non-Hispanic white children [1, 2]. Primary care providers (PCP) are tasked with a periodic screening of children’s health and growth, and thus play a critical role in recognizing and addressing childhood obesity, a condition associated with heavy health and economic burdens [3‐5]. However, at present, there is low active participation from PCPs in assessing or managing children with overweight or obesity in the primary care clinics [5], even though pediatricians can easily identify children with obesity [6]. Thus, it is essential to equip PCPs with effective prevention and treatment tools to address childhood obesity that they can easily implement.

Implementation research focuses on the processes and factors associated with successful integration of evidence-based programs within a particular setting, e.g., healthcare settings [7]. Many studies have implemented and assessed the effectiveness of different weight management programs in healthcare settings [8]; however, none, to the best of our knowledge, have developed and/or assessed the level of implementation of an intervention specific to weight management in healthcare settings using a composite score. Composite indicators are widely used in healthcare settings to measure and track provider performance as well as for external and internal benchmarking against other providers or institutions for quality improvement [9]. Developing a similar indicator or index for weight management interventions in primary healthcare settings will not only help to evaluate the fidelity of the intervention but also help to compare how the different primary care clinics perform.

The overall goal of this study was to describe the methodology and development of an index that can assess the implementation of the Next Steps program component of the Texas Childhood Research Demonstration (TX CORD) primary prevention study. The Next Steps program, which comprises weight management counseling materials [10], was pilot tested in the TX CORD study. The materials support brief counseling during busy office visits, with a goal of encouraging healthcare providers’ attention to obesity. Thus, it served as an engagement tool in study recruitment for the TX CORD primary prevention study and was also utilized in the comparison arm for the TX CORD secondary prevention study [11]. As a result, in our study, as a secondary objective, we examined the association of the implementation index of Next Steps with the enrollment of children with overweight and obesity in the TX CORD secondary prevention study.

Our study examined the acceptability, adoption, appropriateness, and feasibility of the Next Steps program materials, which were pilot tested in primary care clinics that participated in the primary prevention component of the TX CORD study over two years of implementation. Implementation scores were lower in Year 2 than Year 1, though not statistically significant. There were no significant associations between levels of implementation of Next Steps and enrollment into TX CORD secondary prevention study.

We used a novel methodology to develop an index using process evaluation measures that assessed the implementation of weight management programs to improve the health of children with overweight and obesity in primary care clinics. Indices to assess implementation of child weight management programs in healthcare clinical settings are lacking; thus, our study is the first to develop such an index. Multiple studies have assessed the different components of a weight management program implementation such as reach, effectiveness, adoption, climate, cost, etc., in different settings [11, 19‐21], although none has comprehensively assessed the implementation of the program with a single index score. The clinic index we developed was reliable (Cronbach’s alpha> 0.80) in assessing the underlying constructs; however, it warrants further psychometric analysis to include both measures of reliability and validity.

Although the providers in the current study perceived the Next Steps materials to be acceptable, appropriate and feasible, the frequency of use was around 10–50% of the time. The relatively low adoption of Next Steps material in the current study indicates that other factors led to a lower utilization of the materials during a clinic visit, such as limited time during a clinic visit, lack of training and retraining of providers, parental disengagement, providers’ preference for other obesity prevention approaches, lack of clinic administrative support for this protocol, or something as simple as materials being misplaced [11]. Future studies can assess the reasons for low adoption and then examine ways to increase providers’ use and accessibility to these materials [22].

The results of our study demonstrated some degree of fidelity of the Next Steps program materials across the primary care clinics, particularly during Year 1. Given that primary care clinics in this study primarily serve low-income populations, and since many factors influence these populations and their access to healthcare, implementing the Next Steps program 60% of the time in eight out of 11 clinics indicates the initial feasibility of the program in disadvantaged settings. Implementation scores were lower in Year 2 than Year 1, though not statistically significant. The providers received formal training in the Next Steps materials in 2012 when the primary care clinics were first recruited into TX CORD primary prevention study. During the Year 1 assessment of implementation, the research team was actively encouraging the providers to use the Next Steps tools with any potential patients that might benefit from them, as well as the recruiting tools for the TX CORD secondary prevention study. The second process evaluation data collection occurred 9–10 months after participant recruitment into TX CORD secondary prevention study had ended, when the research team was less visible in the clinics. Furthermore, 13.5% of the providers were in the primary care clinics for < 1 year in Year 2, and the new providers in Year 2 may not have been familiar with the material and study. These, along with the cross-sectional nature of the study and the perception of the TX CORD project as a study and not necessarily a program, might explain the decrease in the implementation of Next Steps materials between Year 1 and Year 2, and the inverse relationship observed between feasibility and enrollment in Year 2. Our results suggest the need for ways to increase engagement of providers such as repeat training of providers to increase the sustainability of Next Steps in primary care clinics [22].

There were no significant associations between levels of implementation of Next Steps and enrollment into TX CORD secondary prevention study. The potential effect of implementation of weight management materials on referral and enrollment of children into weight management programs is uncertain from this study as some of the largest clinics had a high implementation but low referral and enrollment, while the reverse was true in the small clinics (Table 3). The small number of clinics in this study limits the generalizability of the study, and the wide variation in percent of patients referred and enrolled in the TX CORD secondary prevention study, even within implementation level categories, delineates the need for future studies to understand the barriers and enablers to referral and enrollment to a weight management program both from provider and patient perspective [23]. For example, studies [24, 25] have reported that some of the provider-level barriers were lack of actual or perceived skill to discuss weight issues with patients and families and being ill-informed about existing services. At the patient-level [25‐27], barriers included parents not perceiving the need for such programs for their children, parents not initiating the provider’s recommended care, lack of motivation from both parent and children, personal health problems, and logistics (e.g., scheduling issues, weather, transportation). Furthermore, referral to the program may be an inadequate outcome measure of implementation; the PCPs may have been using the weight management materials without referral of the children to the program. A more appropriate indicator of the use of Next Steps could be an improvement in BMI rather than enrollment into TX CORD secondary prevention. Because the TX CORD secondary prevention study was an experimental study that compared two weight management programs where Next Steps was the comparison group, we limited our outcome to an immediate one, enrollment of children with overweight and obesity into a weight management program.

There were certain limitations to our study. The study design was serial cross-sectional, thus, the implementation index scores at Year 1 came from a somewhat different cohort than in Year 2. Surveys with < 80% completion were excluded from the analysis (n = 5). This might introduce selection bias to the study findings if the non-responses were due to non-adherence with the Next Steps program. The process measures were self-reported, thus subject to recall or social desirability bias. Additional objective measures such as direct observations may provide a deeper insight into implementation fidelity of programs, and comparison of implementation of programs from different perspectives. The process measures were reported by the PCPs, so future studies could also assess the implementation of these measures by other clinicians such as dietitians or health educators. Only 11 primary care clinics participated in the study, providing limited extrapolation of study findings to other weight management primary care clinics. The TX CORD study was the first to pilot test Next Steps as a weight management program in primary care clinics, and there were no comparison clinics. Future studies should evaluate and validate the implementation of the Next Steps program in healthcare settings in a more rigorous design.

Despite these limitations, our study had strengths. This is the first study to develop a simple methodology for assessing implementation of weight management programs in healthcare settings. Survey scales had good internal consistencies at both years of implementation. Additionally, the implementation levels at the different primary care clinics demonstrated face validity. The Next Steps program materials were available in Spanish, thus, tailored towards the target population (predominantly, Hispanic/Latino). Finally, the homogeneity of the targeted population (2–12-year-old children from low-income underserved areas in Houston and Austin, Texas) also reflected a fair comparison of implementation of Next Steps program materials across the different primary care clinics.

The results of our study highlight the importance of process evaluation and sustainability in primary care settings that are part of intervention programs. The implementation index scores created in this study provide an opportunity to assess how effective and comfortable the providers are at using the different weight-management tools in such settings. Using a standardized scoring procedure, we observed there was wide variability in the implementation levels across the primary care clinics that help to identify the best practices, barriers, and the need for modification of weight management tools tailored towards specific primary care clinics.

In conclusion, our study demonstrated an application of a novel implementation assessment technique to weight-management materials in primary care clinics in the TX CORD study.