Reach, engagement and effectiveness of in-person and online lifestyle change programs to prevent diabetes

The sample included adults referred to LCP using the Solera Health platform between 2015 and 2018. Eligibility criteria included ages 18–85 years and at high risk for diabetes. “High risk” for diabetes was defined as at-risk weight for diabetes (body mass index (BMI) > 24 kg/m² or > 22 kg/m² among Asian American), suggestive blood glucose testing (fasting glucose of 100–125 mg/dl, plasma glucose of 140–199 mg/dl measured 2 h after a 75 g glucose load, or glycosylated hemoglobin of 5.7–6.4), history of gestational diabetes, or high-risk on a self-administered CDC written prediabetes risk assessment [18]. Adults with government-sponsored insurance or no insurance were ineligible. Eligible participants registered for an online account and completed health and demographic questions. Participants whose address was within 25 miles of an in-person program with open enrollment were given a choice of in-person or online LCP; this subset of participants was not identifiable within the dataset. If in-person programs were unavailable, only an online program was offered.

The full registry included 26,932 registration records from 26,743 unique individuals referred to 20 CDC-recognized LCP (9 online, range of records registered by organization 1–34% of online registrants; and 11 in-person, range of records registered by organization 1–80% of in-person registrants). One thousand two hundred four registration records were excluded for ineligibility (n = 518) or incomplete registration (n = 686). The final analytic sample included 25,728 registration records (19,377 online LCP, 6351 in-person LCP, see Fig. 1).

Online LCP were defined as year-long programs delivering the entire CDC-approved LCP curriculum online with peer support, self-monitoring of diet and physical activity and virtual interaction with a human coach [18]. Online programs deliver the standardized content asynchronously in varying modules and formats.

In-person LCP were group-coaching programs with at least sixteen weekly core sessions in the first 6 months and bimonthly maintenance sessions through the remainder of the year [2]. All the organizations in this study were pending, preliminary or met full recognition per the CDC standard through the CDC Diabetes Prevention Recognition Program (DPRP) at the time of the study.

Our secondary outcomes included reach and engagement. We selected the secondary outcomes a priori to explore mechanisms leading to our primary endpoint and to evaluate the public health impact of LCP.

Reach was defined as number of participants enrolled in LCP divided by the number of participants referred to LCP. At-risk adults were considered ‘referred’ after they a) registered on the Solera platform, b) qualified for enrollment, and c) matched to a program. Program matching was based on individual preferences (class size, online vs. in-person, class schedule, and location; See Additional file 1). Participants were considered ‘enrolled’ if they completed their intake questionnaire and provided a baseline weight (online: set-up their cellular scale and completed their baseline weight; in-person: baseline weight measured by the health coach at class intake).

In-person engagement was defined as physical attendance at the weekly hour-long session tallied by the health coach. Weekly online engagement was defined by a proprietary algorithm which includes curriculum delivery, health coach interaction, peer support and self-tracking that is equivalent to an hour of in-person class engagement. The CDC standard for engagement during the time of data collection was engagement at ≥9 sessions by week 26, a threshold associated with meaningful weight loss [18]. Six month engagement in this analysis was defined as the number of participants reaching the CDC-defined engagement threshold at 26 weeks divided by the number of participants enrolled. We also examined the adjusted odds of participants achieving meaningful weight loss for every session engaged.

We stratified outcomes by participant-, setting- and intervention-level characteristics. Participant-level variables included demographics (self-reported age, sex, and race/ethnicity), baseline body mass index (BMI; normal < 25 kg/m², overweight 25–29 kg/m² and obese > 30 kg/m²) and method of determining elevated diabetes risk (blood glucose testing, history of gestational diabetes, or high-risk on a self-administered CDC written prediabetes risk assessment). Individuals that self-identified as multiracial or did not report race or ethnicity were defined as ‘other’. Setting-level variables included median income of zip code of residence and rural/urban designation. Income estimates were generated using participant ZIP codes cross-tabulated to ZIP Code Tabulation Areas (ZCTAs) created by the US Census Bureau [19]. Median household income estimates were generated based on ZCTA-level estimates from the 2016 American Community Survey [20]. Rurality estimates were generated by cross-tabulating participant ZIP codes to Rural–Urban Commuting Area Codes (RUCA) [21]. Intervention-level characteristics included registration date and LCP provider.

We used t-tests for continuous, normally distributed variables, non-parametric testing for non-normally distributed variables, and χ² tests for categorical variables. We used logistic regression modelling to assess the association between participant- and setting-level characteristics with effectiveness (primary outcome), reach and engagement. We used multilevel mixed-effects modelling with participants nested within the program as random effect, adjusted for participant- and setting-level factors, to estimate per week percent weight loss. We used separate models for in-person and online programs to account for varying patterns in weekly program missingness (higher missingness in weekly weight measures among in-person participants – see Additional file 2). A p-value of < 0.05 was considered to be statistically significant. All analyses were conducted using Stata 15.0 (Stata Corporation, College Station, TX, USA).

We examined effectiveness among participants who engaged in at least four sessions, the minimal exposure defined by the CDC as engagement at the time of program participation [18]. We opted to examine this subset of the population as a sensitivity analysis, rather than our primary population, to minimize potential measurement error from the varying definitions of program engagement between online and in-person platforms.

The comparative ease of home weigh-ins may have resulted in differentially more last recorded weigh-ins among less engaged online participants who regained weight and remained in their program compared to in-person participants who may have stopped attending their program after regaining weight. Therefore, we also investigated effectiveness defined as percentage weight change from baseline weight to the lowest recorded weight.

Lastly, we excluded participants who participated in multiple programs to limit the effect of individuals with higher propensity to engage (n = 165 participants, see Fig. 1 footnote for details).

Table 1 displays characteristics of at-risk adults referred to LCP by enrollment status. Overall, participants referred to LCP were mostly female (75% female), non-Hispanic (61%), White (56%) and from urban areas (85%). Participants were considered referred after they registered and matched to a lifestyle change program per their preferences. Participants that lived more than 25 miles from the closest in-person program were limited to online programs. Participants matched to online LCP were mostly non-Hispanic (64%), White, from California or Florida (59 and 27%, respectfully), and living in higher-income, urban-designated areas. Compared to adults who were matched to in-person programs, adults matched to online LCP were significantly younger (median age 50 vs 52,p < 0.001), male (28% vs 16%,p < 0.001), more white (White 58% vs 51%,p < 0.001) and lived in more rural areas (10% vs 5%,p < 0.001). A lower percentage of adults who enrolled in an online program were obese compared with adults enrolled in an in-person program (65% vs 73%,p < 0.001).

Table 2 shows the descriptive findings of the main implementation outcomes. Online LCP effectiveness was lower, with 23% of online-enrolled participants achieving the 5% weight loss threshold, compared with 35% of in-person-enrolled participants (p < 0.001). Meaningful weight loss differed significantly by specific online provider in this study (16–27%,p < 0.001; data not shown). Mean weight loss was 2.0% (95%CI 1.9–2.1) among online participants compared to 4.3% (95%CI 4.1–4.5) among in-person participants. Median percent weight loss increased by total program week engagement but was consistently below the meaningful weight loss threshold among participants enrolled in online programs (Fig. 2). Table 3 presents the odds ratios of effectiveness among those enrolled by participant- and setting-level factors. After adjusting for age, sex, race/ethnicity, baseline BMI, region, income, rural/urban, registration date and program, the odds of online participants achieving meaningful weight loss was lower among those 18–44 years old (AOR 0.67,95%CI 0.56–0.80) and 45–64 years old (AOR 0.80,95%CI 0.68–0.94) than among those ages 65 and older (Table 3). Similarly, meaningful weight loss among online participants was lower among African American participants compared to White participants (p < 0.05). Multilevel modelling revealed that adjusted weight loss was 0.07% per program week among online programs and 0.11% per program week among in-person programs.

A higher proportion of adults matched to online programs enrolled, compared with adults referred to in-person programs (Table 2; 56% vs 51%, p < 0.001). After adjusting for age, sex, race/ethnicity, region, income, rural/urban, registration date and program, the odds of an online participant enrolling after being referred (i.e. reach) was twice as high among those 18–44 years old than among those ages greater than 65, and 55% higher among those 45–64 years old than among those ages 65 and older (p < 0.001). However, age was not significantly associated with increased reach among in-person participants. Reach was higher in rural participants (compared to urban participants) for both online (p < 0.05) and in-person delivery (p < 0.05).

Over half (55%) of enrolled online participants remained in the program after 1 year, compared to about a quarter (28%) of enrolled in-person participants (p < 0.001). Engagement at 6-months (engagement with ≥9 sessions by week 26) was significantly lower among online-enrolled participants compared to in-person (46% vs 66%, p < 0.001). After adjusting for age, sex, race/ethnicity, baseline BMI, region, income, rural/urban, registration date and program, the odds of an online participant engaged at 6-months was lower among those 18–44 years old (AOR 0.39,95%CI 0.33–0.45) and 45–64 years old (AOR 0.72,95%CI 0.62–0.84) than among those ages 65 and older (Table 3). Similarly, 6-month program engagement among online participants was lower among participants with baseline obesity compared to those who were overweight (p < 0.01). After adjusting for age, sex, race/ethnicity, baseline BMI, region, income, rural/urban, registration date and program, the odds of participants achieving meaningful weight loss was significantly higher for each session engaged (in-person - AOR 1.08, 95%CI 1.07–1.10; online 1.04, 95%CI 1.03–1.04, see Additional file 3). Adjusting for engagement partially attenuated the effectiveness of in-person programs, suggesting factors other than engagement may be driving the differential effectiveness of the in-person format (see Additional file 5).

There was a similar pattern of effectiveness (5% weight loss) by format-type among participants who engaged in at least four sessions (28% online compared to 41% in-person, p < 0.001). The difference in effectiveness by format was only partially attenuated when defining percent weight loss as the difference between baseline and lowest measured follow-up weight (38% online compared to 45% in-person among enrolled participants, p < 0.001). Excluding participants with multiple participation records did not modify patterns. None of the sensitivity analyses substantively impacted results overall or by subgroups (Supplemental material; Additional file 3).

Our study has several important limitations. Because many participants selected their programming, propensity to engage was likely higher than in research studies. Further, we do not have data on the participants that were offered a choice of in-person versus online programming. While this limits our ability to weigh the contribution of selection bias and examine the role choice may have on engagement, this analysis reflects the real-world, coinciding delivery of online and in-person platforms where only a minority have access to in-person programming. Program differences may be attenuated since we include a small sample of participants that participated in both online and in-person programs. We present the unadjusted proportions to demonstrate the impact of LCP among high-risk adults in a large, real-world setting. We used separate models within each platform (in-person and online) to optimize model fitness and differences in measure definitions (e.g. ascertainment of weight, engagement). These unadjusted comparisons may be biased given the participant and setting level differences between the two platforms (see Table 1). However, our findings remain similar when adjusting for participant and setting difference between the two platforms in a combined analysis (see Additional file 4).

There was a high degree of missingness in our data, particularly among in-person participants. This is likely partially related to differences in ascertainment of weight (in-person weigh-in during sessions vs. online cellular scale). To address this limitation, we designed our study to reflect national program benchmarks that reflect real-world metrics that inform policy. Further, we tested multiple assumptions of missingness in our sensitivity analyses. The difference in missingness raises the uncertainty of effectiveness and estimated weight loss among in-person participants. We caution readers that these findings reflect real-world delivery of LCP and do not reflect a comparative effectiveness trial of online and in-person lifestyle change programs within a controlled setting. Since this dataset was fully deidentified, some of our participant-level variables are area estimates (income, rurality). Finally, the CDC-recognized LCP were limited to Solera’s referral network among commercially-insured adults and may not generalize to NDPP as a whole.

This study also has several strengths. This is a large, real-world dataset of online-LCP participants. Our in-person results align with the CDC evaluation, suggesting our findings may be generalizable to national estimates of commercially-insured adults participating in DPRP-recognized organizations.