The Clinical Resource Hub Telehealth Program and Use of Primary Care, Emergency, and Inpatient Care During the COVID-19 Pandemic

To identify our study cohort, we began with all VA clinics or sites (n = 1110) and excluded atypical such as community living centers or sites with fewer than 450 patients over the study period, similar to prior work.²¹ The final study cohort included 1050 VA sites (CRH-PC: 254; non-CRH-PC: 796). All patients assigned to a PACT team in each site were included; patient assignments were identified from the Reengineered Patient Care Management Module in the Corporate Data Warehouse (CDW).²²

As adoption of CRH can impact both the number of care visits and patients seen at sites, we examined the quarterly number of visits and patients served per site for two types of care:

We considered reporting change in visits per patient assigned to each site; however, many patients assigned to a PACT team are not expected to and do not use primary care in each quarter. We therefore reported results as the percentage change in volume of visits and patients served compared to baseline patterns of utilization, adjusting for site size. We also reported the number of visits per 1000 primary care patients served to indicate the extent of changes for patients who do utilize primary care in each quarter.

Note that if a hub provider had a telehealth visit for a patient assigned to a spoke site, the visit was attributed to the spoke site. Data on PC and ED encounters were obtained from Managerial Cost Accounting (MCA) OUT (Outpatient) National Data Extract (NDE),²³ where we categorized primary and secondary clinic stop code pairs into categories and modalities of care. Hospitalizations were obtained from MCA TRT (Treating Specialty) NDE.²³

We adjusted for differences in outcomes due to potentially differing site characteristics. We adjusted for site rurality (e.g., urban, rural, highly rural), site type (e.g., VHA Medical Center (VAMC), PC Community-Based Outpatient Clinic (PC-CBOC), multi-specialty CBOC (MS-CBOC), other), site size (number of patients with PACT team assignments by quarter), geographic region of each VA service network (East Coast, Southeast, Rocky Mountain/Gulf, Midwest, West Coast), and the quarterly average Elixhauser Comorbidity Score of patients assigned to each site. Importantly, we also included a binary indicator of whether a site ever adopted CRH-PC to adjust for any remaining unobservable differences across program and non-program sites that were time-invariant. In sensitivity analyses, we also adjusted for site-level summaries of other patient covariates such as mean age, as well as percents male, White, Black, Hispanic, and VA enrollment priorities 1 and 2.

In all models, we included the quarterly count of COVID-19 cases in each county²⁴ and included quarter indicators to adjust for any shocks to health care systems or care use in each quarter, including lingering effects of the pandemic outside of the case count.

Covariate data were obtained from the CDW,²² VHA’s Geospatial Service Support Center,²⁵ and New York Times’ COVID-19 county-year level data downloaded from GitHub.²⁴

We first examined baseline site characteristics for CRH-PC and non-CRH-PC sites from FY2020Q1 (October-December 2019) before pandemic-related shutdown of in-person care began. We then examined unadjusted trends of CRH-PC and non-CRH-PC sites before and after the onset of COVID-19 to determine whether a difference-in-difference (DiD) framework was appropriate. Event studies improve on the traditional DiD estimator^26‐28 because they estimate differences between treatment and control group for each period prior to and after treatment (i.e., each quarter pre- and post-pandemic onset in our case).^{26, 28} This allows to visually and more transparently assess whether pre-pandemic model-adjusted differences between CRH-PC sites and non-CRH-PC sites were significant or trending upward or downward in a manner that could obscure or mask true differences in the post-pandemic onset period.^{26, 28, 29} An absence of pre-pandemic differences across CRH-PC and non-CRH-PC sites after covariate adjustment followed by abrupt differences post-pandemic signals attributability of findings to the pandemic.^{26, 28, 29} We also generated traditional DiD estimates to obtain the average effect of CRH-PC across all post-pandemic onset quarters (methods details in Appendix Section A).

As effects may vary across types of VA sites or site size, we conducted identical analyses, stratified by site type as site size is vastly different across site types and among these stratified analyses, further adjusted for site size. We also examined the number of visits per 1000 primary care patients served.

To strengthen attributability of findings to the CRH program, we conducted sensitivity analyses restricting the sample of CRH-PC sites to sites with pre-pandemic program implementation and to sites with program implementation during the majority of the post-pandemic onset period. We also examined reliance on CRH-PC services at CRH-PC sites (Appendix Section D).

All statistical analyses were conducted in Stata 17.0 (StataCorp, LLC).

This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines. It was funded by VA’s Office of Primary Care for quality improvement purposes and was therefore exempted from review by the Stanford institutional review board.

We found no differences in site rurality and average Elixhauser comorbidity score of patients across CRH-PC and non-CRH-PC sites, but noted some differences in site type, site size, and geographic region (Table 1). We found that VAMCs, MS-CBOCs, and larger sites were more likely to adopt CRH-PC. West Coast VA sites, followed by Rocky Mountain/Gulf region sites, were more likely to adopt CRH-PC. Similar comparisons stratified by site type are also provided in Appendix Tables 5, 6, and 7.

Unadjusted trends in primary care visits, ED visits, and hospitalizations show that prior to pandemic onset, CRH-PC sites and non-CRH-PC sites had roughly parallel trends, providing support for the use of DiD methods (Fig. 1). Unadjusted trends were very similar when we examined total number of patients served (Appendix Fig. 3).

Event study graphs show that after covariate adjustment, there were virtually no differences between CRH-PC and non-CRH-PC sites prior to the pandemic onset, whereas there were abrupt differences across CRH-PC and non-CRH-PC sites just after the onset of the pandemic (Fig. 2). This pattern of no differences pre-pandemic onset followed by abrupt differences post-onset signals attributability of post-onset changes to the pandemic onset. Event study results were very similar when we examined number of patients served (Appendix Fig. 4).

We also found similar results in sensitivity analyses restricting the sample of CRH-PC sites to sites pre-pandemic program implementation (Appendix Fig. 5) and to sites with program implementation for a majority of the post-pandemic onset period (Appendix Fig. 6). In analyses focusing on CRH-PC sites only, we demonstrated that reliance on CRH-PC increases during the post-pandemic onset period but that sites were using CRH infrastructure also for non-CRH services (Appendix Fig. 7).

In traditional DiD analyses (Table 2), we found that CRH-PC sites, compared with non-CRH-PC sites, had 221 (95% CIs 11:431) additional primary care visits across all modalities per quarter, a 3.4% increase compared to prior to the pandemic. By modality, we found that CRH sites had 643 (95% CIs − 910: − 377) fewer in-person visits (− 14.4% compared to pre-pandemic utilization at CRH-PC sites) but had 723 (95% CIs 433:1,014) and 128 (95% CIs 56:199) more visits via phone and video telehealth per quarter (+ 39.9% and + 159.5% compared to pre-pandemic utilization) than comparison sites, respectively. We also found that CRH-PC sites had 29 (95% CIs − 57: − 1) fewer VA ED visits and 10 (95% CIs − 18: − 2) fewer VA hospital stays post-pandemic onset per quarter (− 4.2% and − 5.1% compared to pre-pandemic), respectively, compared with non-CRH-PC sites. DiD estimates were broadly similar when we examined the impact of CRH-PC on the number of patients served (Table 3).

Examining the number of visits per 1000 primary care patients served (Table 4), we did not find significant differences in in-person visits or video visits per patient served, but found that CRH-PC sites had 48 (95% CIs 10: 85) additional phone primary care visits per 1000 primary care patients served, compared to comparison sites.

In analyses stratified by site type (Table 2), we found that PC-CBOCs with CRH-PC, compared to PC-CBOCs without CRH-PC, had 240 fewer in-person visits (− 10.4% compared to pre-pandemic) but 300 additional phone visits (+ 31.6%). MS-CBOCs with CRH-PC, compared to MS-CBOCs without CRH-PC, had additional 521 primary care visits across all modalities (+ 7.1%) and 915 more phone visits for primary care (+ 46.4%). Examining the number of patients, we found that PC-CBOCs and MS-CBOCs with CRH served more patients through telephone visits than comparison PC-CBOCs and MS-CBOCs (+ 29.6% and + 38.7%, respectively, compared to pre-pandemic) (Appendix, Table C1).

There are some limitations of this study. As our study focused on the impact of the CRH-PC program during the COVID-19 pandemic, we did not examine utilization pre-post CRH-PC adoption at sites as we did not have site-specific start dates for CRH. As such, it is possible that some CRH sites continued to adopt CRH-PC post-pandemic onset. Analyses focusing on the association with the specific timing of uptake or penetration of CRH at VA sites would be helpful for further isolating the impact of CRH. Nonetheless, in sensitivity analyses restricting the sample of CRH-PC sites to sites that adopted CRH-PC pre-pandemic (Fig. 5 in the Appendix) or sites that had CRH-PC during the majority of the post-pandemic period (Fig. 6 in the Appendix), we observed very similar results.

Next, an unavoidable methodological limitation in evaluating a population-level intervention is there may have been unobservable factors that influenced outcomes and the adoption of CRH-PC. As such, we leveraged a DiD design which allows for level differences in baseline characteristics across CRH-PC and comparison sites. DiD allows for unobservable reasons for program adoption as long as CRH-PC and comparison sites exhibit parallel trends in outcomes. We first provided evidence that unadjusted trends exhibit parallel outcome trends for CRH-PC and non-CRH-PC sites (Fig. 1), and then in adjusted event study graphs (Fig. 2), we demonstrated that pre-pandemic differences observed between CRH-PC and non-CRH-PC sites were eliminated after covariate adjustment. The stark contrast in differences across CRH-PC and non-CRH-PC sites immediately post-pandemic then strengthened attributability of post-pandemic differences to CRH-PC. Nonetheless, our methods are not able to distinguish between differences in CRH-PC and non-CRH-PC sites unrelated to the CRH program which may have occurred simultaneously with pandemic onset. For example, if CRH-PC sites improved their infrastructure or management in response to the pandemic for reasons unrelated to the CRH program, our analysis cannot disentangle these associations from the associations with the CRH program. Additional studies of CRH will be important to validate our findings.

We also did not study health care utilization that may have occurred outside VA so utilization by patients in CRH and non-CRH sites may have been under-measured. Additional studies are needed to examine the impact of CRH on total outpatient, acute, and inpatient care for each patient. Furthermore, as CRH also provided mental health care and specialty care, our analyses focusing on primary care remain limited in scope. Future work should also examine CRH’s impact on these other types of care. Finally, our results may not readily generalize to all types of telehealth programs or to non-VA settings.