Using a RE-AIM framework to identify promising practices in National Diabetes Prevention Program implementation

More than 30 million U.S. adults have diabetes [1], with 90–95% being type 2 diabetes [2]. In addition, the Centers for Disease Control and Prevention (CDC) estimates as many as 84.1 million U.S. adults, more than 1 in 3, have prediabetes [1]. According to the American Diabetes Association (ADA), prediabetes is defined as blood glucose levels that are elevated, though not high enough to be diagnosed as type 2 diabetes [3]. Prediabetes can lead to type 2 diabetes, heart disease, and stroke; however, it is reversible. Multiple studies have demonstrated that lifestyle change programs are effective in preventing or delaying the onset of type 2 diabetes [4‐9].

The Diabetes Prevention Program (DPP) randomized clinical trial, initiated in 1996, found that a yearlong intensive lifestyle intervention aimed at reducing risk through healthy eating, physical activity, and stress reduction helped participants lose 5–7% of their body weight and reduce the risk of developing type 2 diabetes by 58% among high risk adults aged ≥ 25 and 71% for those aged ≥ 60 [10]. Since the DPP research study, many translational studies have shown that the structured lifestyle intervention is effective and feasible in community settings to prevent or delay the incidence of type 2 diabetes among people at high risk [11‐18]. However, large-scale implementation in the U.S. had not been attempted. In order to do this, a coordinated national effort was needed to increase the supply of quality programs, demand for the program among people at risk, referrals from healthcare providers, and coverage among public and private payers [19].

In response to this need, CDC established the National DPP—a partnership of public and private organizations working to reduce the burden of type 2 diabetes by building the infrastructure for nationwide delivery of the evidence-based lifestyle change intervention (LCI) [19]. The National DPP LCI is a yearlong, structured lifestyle intervention consisting of a minimum of 16 weekly sessions about an hour in length in months 1–6 and a minimum of 6 monthly sessions about an hour in length in months 7–12 [20]. To assure quality and fidelity to scientific evidence, CDC also established the Diabetes Prevention Recognition Program (DPRP), the quality assurance arm of the National DPP. Through the DPRP, CDC recognizes organizations that successfully deliver the yearlong LCI consistent with DPRP Standards and Operating Procedures (Standards) [20]. CDC develops and updates its Standards based on current evidence in the literature, analyses of organizational outcome data, and input from public stakeholders, including organizations delivering the LCI. The DPRP provides technical assistance to delivery organizations and evaluation reports on organizational outcomes linked to the Standards. Additional information about delivery of the National DPP LCI is described elsewhere [20, 21]. CDC grants pending recognition to organizations that have successfully applied to the DPRP, agreeing to use a CDC-approved curriculum and adhere to the specified intensity and duration of the LCI [20]. CDC full recognition is granted to organizations that have implemented the National DPP LCI for at least 12 months and achieved the outcomes described in the Standards [20].

Evaluation of the first 4 years of the program, describing the experience of 14,747 participants, found an average weight loss of 4.2%, with 35.5% of participants achieving ≥ 5% weight loss [21]. In addition, the study found that better retention in the program, with more sessions attended and longer duration from first to last session, resulted in a higher percent of weight loss overall and within subgroups, with a median weight loss of 6% among the 34.6% of participants who attended 17 or more sessions and remained in the program for 7 or more months [21]. Similarly, findings from a 2009 cluster randomized trial of 43 general practices in the United Kingdom (UK) showed that remaining in a type 2 diabetes prevention lifestyle change program was associated with improvements in fasting and 2-h glucose, glycated hemoglobin (HbA1c), weight loss, waist circumference, anxiety, quality of life, and daily step count [22]. A systematic review of translational studies and practical implementation trials of the DPP research study over the last 15 years worldwide also showed that program intensity played a major role in weight loss outcomes [23]. A meta-analysis of randomized clinical trials assessing long-term sustainability of the DPP LCI showed that weight loss was the key factor associated with a delay in type 2 diabetes progression; specifically, every kilogram of weight lost was associated with an additional 7% risk reduction [24]. Despite consistent findings on the effectiveness of the DPP LCI in preventing or delaying onset of type 2 diabetes [10‐18, 22‐24], there is limited evidence on implementation of the National DPP LCI in real-world settings where people live, work, play, and worship.

In 2012, CDC funded six U.S. national organizations through a 5-year cooperative agreement (National Diabetes Prevention Program: Preventing Type 2 Diabetes Among People at High Risk (1212)) to scale and sustain multistate networks to deliver the structured, evidence-based LCI consistent with DPRP Standards in communities nationwide. Funding for the cooperative agreement was allocated based on the organizations’ capacity to reach large numbers of people at risk in multiple states, while working to also obtain health benefit coverage by employers and insurers [25]. The funded national organizations were the American Association of Diabetes Educators, America’s Health Insurance Plans, the Black Women’s Health Imperative, the National Association of Chronic Disease Directors, Optum HealthCare Solutions, and the YMCA of the U.S.

Funded organizations participated in a CDC-led national evaluation of their program implementation to identify promising practices for scaling and sustaining the National DPP. The evaluation was designed using the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) [26] framework. RE-AIM was originally developed by Glasgow, et al. for consistent reporting of research results and later expanded to literature reviews, with particular attention on how to organize study findings on health promotion and disease management in different settings [27, 28]. The RE-AIM framework has been widely used to help guide the translation of research into practice and to help understand the efficacy and effectiveness of programs implemented in real-world community settings [27, 28]. The detailed definitions and levels of measurement for the RE-AIM dimensions are described in Appendix 1: Table 9. This evaluation included qualitative and quantitative measures pertinent to the five dimensions of RE-AIM to assess program implementation over the 4-year funding period.

The present work describes the results of the evaluation of funded organizations’ collective efforts and subsequent lessons learned in scaling up and sustaining the National DPP. The primary objective of this study is to describe program reach, adoption, and maintenance. This study also aims to assess associations between site-level factors and program effectiveness in terms of three outcomes: overall program attendance, attendance in months 7–12, and duration of participation.

The results are presented according to the principles of the RE-AIM framework. We present data on program adoption, implementation, and maintenance as well as information on program reach to participants and program effectiveness regarding site-level factors associated with participant overall attendance, attendance in months 7–12, and duration of participation.

Over the 4-year funding period, the six national organizations established a total of 164 new CDC-recognized organizations across 38 states, providing 1,239 LCI classes and enrolling nearly 15,000 participants. This represented 10% of all sites and 17% of all eligible participants in the DPRP as of September 2016. In addition, the funded organizations worked with a total of 27,440 new employer groups to offer the National DPP LCI as a covered health or wellness benefit, covering a total of 5,013,449 employees, members, or beneficiaries of whom a subset would be eligible for the program; 198 of these employer groups offered the LCI on site.

Our results suggest that program sustainability was high; one quarter of the funded sites achieved full CDC recognition during the study period, and most sites (88.2%) continued to offer the National DPP LCI and maintain their CDC recognition after funding ended. Additionally, the funded national organizations influenced nearly 50 insurers to cover the program. Beyond the nearly 15,000 people directly served by the funded sites, these changes in coverage have the potential to reach many more people with prediabetes or at high risk for type 2 diabetes.

Although participants were racially diverse (at least 24% reported racial or ethnic minority status), the reach into some underserved population groups was limited (e.g., 20.0% male participants). While this study did not address specific recruitment strategies for men, it showed no statistically significant difference in attendance and duration of participation among men and women. This finding is similar to a systematic review and meta-analysis of gender-specific differences in diabetes prevention, which found no significant differences in incidence of type 2 diabetes and weight change between men and women who received lifestyle interventions [30]. Several new pilot studies have been adapted from the DPP research study to increase reach in men. For example, “Power Up for Health”, a program facilitated by male lifestyle coaches only, was implemented at five different recreation centers located in disadvantaged neighborhoods across New York City [31]. The study showed improvement in weight loss, depressive symptoms, healthy eating and exercise, and health status of male participants, although recruitment was still challenging [31]. In focus group interviews, male participants indicated that the all-male aspect of the program and its use of male coaches were main facilitators for participation [32]. The addition of interactive components such as exercise or healthy cooking demonstrations was also recommended [32]. Similarly, the adapted Kerala DPP in India incorporated male peer-leaders and offered sessions in the evening and on weekends, which was shown to enhance male participation [33]. Based on analysis of CDC’s DPRP registry as of May 2019, there was increased participation by men (27.4%) in virtual programs, compared with 19.6% of male participants in in-person programs [34].

To address the gender gap and other priority populations, in 2017 CDC began funding ten national/regional organizations with affiliate program delivery sites in at least three states to start new CDC-recognized organizations in underserved areas through a 5-year cooperative agreement (1705). This new project incorporates lessons learned from the current evaluation to address gaps and aims to identify and evaluate strategies to enroll populations currently under-represented in the program relative to their estimated numbers and disease burden, such as men, Medicare beneficiaries, African-Americans, Asian-Americans, Hispanics, American Indians, Alaska Natives, Pacific Islanders, and people with visual impairment or physical disabilities.

The RE-AIM model provided a solid framework to assess the implementation of this project, and offers a unique contribution to the type 2 diabetes prevention literature. It allowed simultaneous examination of both participant-level outcome data and detailed organizational and site-level data. Although this evaluation focuses on implementation of the National DPP LCI based on the work of six national organizations in the U.S., it provides a practical evaluation framework and pragmatic measures, which can help address gaps in current evaluations of global diabetes prevention interventions [35]. This evaluation is especially timely, given that the Centers for Medicare & Medicaid Services (CMS) began payment for Medicare participants in CDC-recognized organizations participating in the Medicare Diabetes Prevention Program (MDPP) Expanded Model effective April 1, 2018 [36, 37]. The MDPP will scale the program to a high risk population of Medicare beneficiaries with reimbursement directly tied to CDC recognition to assure program quality and effectiveness [37].

Program-funded incentives, non-CDC funded, were frequently used by sites and were effective in increasing program utilization and retaining participants. We also found that non-monetary incentives such as access to physical activity, pedometers, food measuring devices, or cookbooks were significantly associated with better outcomes. Similarly, recent findings from the 2017 “We Can Prevent Diabetes” trial [38], a collaborative approach with primary care clinics and the YMCA aimed at Medicaid beneficiaries, found that both monetary and gift card incentives increased enrollment, attendance, and weight loss among low-income, high-risk participants in a yearlong type 2 diabetes prevention program [38]. It is also important to note that there was mixed evidence in the literature on various types of incentives used to promote public health interventions among different populations. For example, Sen et al. studied the effectiveness of two lottery incentives (expected daily value of $2.80 vs. $1.40) for improving adherence to remote-monitoring regimens among patients with poorly controlled diabetes and found no difference in adherence between the two incentive arms [39]. However, the low incentive arm had better monitoring rates relative to controls and had significantly better efficacy than the higher incentive arm once incentives were removed [39]. A systematic review of impact of financial incentives on the implementation of asthma or diabetes self-management showed mixed results on diabetes control impact, but there was evidence in improved process and health outcomes in asthma control [40]. A randomized controlled trial comparing the effectiveness of individual vs. team-based financial incentives to increase physical activity showed that financial incentives awarded for a combination of individual and team performance were most effective for increasing physical activity [41].

Funded sites frequently implemented cultural adaptations to address participants’ needs or preferences, which were positively associated with participants’ overall attendance, attendance in months 7–12, and duration of participation. This finding is consistent with Chesla et al., who adapted the CDC-approved Group Lifestyle Balance curriculum for Chinese Americans with prediabetes, resulting in greater achievement of the 5% weight loss goal when compared to Chinese Americans receiving a non-adapted curriculum [42]. We did not, however, find improved results among sites that used language adaptations. In contrast, a systematic review of studies on translated versions of the DPP curriculum in various communities found that translation strategies, including use of bilingual study personnel, had positive results [43]. Further, Taylor et al. identified low literacy and language difficulties as barriers to successful delivery of the DPP to vulnerable and disadvantaged adults and found tailored and flexible program design to be facilitators [44]. Our study did not collect information on lifestyle coaches’ bilingual status, and participants’ socioeconomic status may have been correlated with other barriers to accessing the LCI. To further investigate these issues, CDC plans to assess factors related to lifestyle coaches’ qualifications, training, and language use in a rigorous evaluation of the new cooperative agreement, 1705. In addition, CDC added an education level as a required data field in the new 2018 DPRP Standards to better assess participants’ socioeconomic status [20].

While few sites (29.9%) reported using self-referral or word of mouth as a recruitment strategy, those that did found it to be effective in retaining participants. In addition, sites receiving referrals from healthcare providers/systems had higher participant attendance and longer duration of participation. However, detailed information on how sites implemented these referral strategies was not collected in this study. These results support a systematic review from the Community Preventive Services Task Force [45] and subsequent 2015 recommendation from the U.S. Preventive Services Task Force for clinicians to screen and refer patients at risk to intensive behavioral counseling interventions that promote a healthful diet and physical activity [46]. The ADA followed in 2016 with recommendations that patients with prediabetes be referred to an intensive diet and physical activity behavioral counseling program [47, 48].

At the participant-level, those who were younger (18–44 and 45–64) had significantly lower overall attendance, attendance in months 7–12, and duration of participation than those 65 years and older. This finding is consistent with a study of a multisite diabetes prevention translational project among American Indians and Alaska Natives showing that younger participants were at higher risk for both short-term (not completing all 16 weekly sessions) and long-term (loss to follow-up) retention failure [49]. This finding is also consistent with another study of a similar program in the UK, which found that attendance per 100,000 population was significantly higher as age increased [50]. Despite limited studies comparing retention strategies among younger vs. older participants, results from alternative delivery modalities such as mobile phone-based (i.e., mHealth) or other technology-assisted interventions seem encouraging as a means to reach and engage younger adults. For example, a randomized controlled trial was conducted to assess autonomous motivation and healthy behaviors among young adults with prediabetes who previously declined participation in a diabetes prevention program offered at no cost [51]. The study showed that retention was significantly higher among participants who received an app plus a physical activity tracker and wireless enabled digital scale than participants in the other two study arms combined [51]. These findings offer insight on potential approaches to engage more young adults in the National DPP LCI.

Although Hispanics and non-Hispanic blacks had lower overall attendance than non-Hispanic whites, those who attended the LCI at sites that used cultural adaptations actually had higher attendance compared with those who attended at sites that did not use cultural adaptations. This finding is consistent with a study on community-based translation of the DPP’s lifestyle intervention in an underserved Latino population [52]. Ruggiero et al. found statistically significant improvements in anthropometrics and behavioral outcomes as well as consistent participant attendance rates with other community-based lifestyle intervention programs focused on type 2 diabetes prevention [52].

Finally, this study found that participants who were overweight had significantly higher overall attendance, attendance in months 7–12, and duration of participation, compared with those who had obesity. This finding is consistent with a study of an intensive behavioral intervention for weight management, which found that lower baseline BMI was independently associated with higher retention [53]. It is also consistent with the Let's Prevent Diabetes trial, which found that those who attended all sessions had lower baseline BMI than those who did not [22].

There are several limitations of this study. First, program implementation data were self-reported by the funded organizations and their delivery sites; however, data reported were rigorously assessed for data quality and completeness through a series of validation checks (Additional files 3 and 4). Second, program implementation strategies were collected at site-level only, not participant-level. Multilevel statistical modeling at the site and participant-level was used to address intra-class correlation between participants attending the lifestyle change classes at the same sites and clustering between funded national organizations. Third, many sites were no longer funded by the end of year 4, which may limit generalizability of the study. However, the majority (88%) of those sites became self-sustained and continued to offer the program without cooperative agreement funding. Finally, this study is an observational, retrospective evaluation of CDC’s funded organizations. There were challenges in reaching some population groups such as males, African-Americans, Asian-Americans, Hispanics, American Indians, Alaska Natives, Pacific Islanders, and people with disabilities. As a result, these groups have been identified as populations of focus in the current cooperative agreement. The results may or may not be generalizable to all population groups and small or local community-based organizations that may not have strong infrastructure or resources.

Findings from this evaluation can assist those offering or supporting the National DPP LCI in increasing participants’ retention and thereby potentially improving their weight loss outcomes [21‐23, 54, 55] and reducing their risk of developing type 2 diabetes [22‐24, 54, 55]. CDC has incorporated lessons learned from this evaluation to inform strategies in its new cooperative agreement (1705) and evaluation framework. Lessons learned from this evaluation were also used to develop and share a variety of technical assistance resources through the National DPP Customer Service Center [56] with new, established, and potential program delivery organizations, as well as to inform the implementation of the MDPP Expanded Model.

Results from program adoption at the institutional level, and reach at the participant level, suggest that the National DPP LCI is feasible to implement broadly. In a time of rapid growth for the National DPP, lessons learned from this evaluation may be translated to similar organizations and settings. Findings may also be useful to enhance program implementation for the more than 1600 CDC-recognized organizations operating in the U.S., including those with plans to participate as MDPP suppliers as well as those currently funded through the 1705 cooperative agreement. Effective strategies for increasing participant enrollment and retention in the yearlong lifestyle change intervention have tremendous potential to help these organizations achieve the outcomes proven to prevent or delay onset of type 2 diabetes in those at highest risk.