Social distancing causally impacts the spread of SARS-CoV-2: a U.S. nationwide event study

We source our U.S. COVID-19 data from the John Hopkins Whiting School of Engineering’s Center for Systems Science and Engineering’s GitHub repository [17]. This data consists of confirmed cases in each county at the end of every day since the start of the outbreak in late January 2020. We calculate the number of new cases for each county and each day by subtracting the cumulative number of confirmed cases at the end of the day from the number of cumulative cases from the previous day.

We obtain our county-level population data and our county-level demographic data from the U.S. Census Bureau [18]. We extract our county-level Gross Domestic Product (GDP) data from the U.S. Bureau of Economic Analysis’ (BEA) Regional Economic Accounts database (Table CAGDP1) [19]. We retrieve county-level data on the prevalence of obesity, diabetes, smoking, and hypertension from the University of Washington’s Institute for Health Metrics and Evaluation (IHME) [20]. The hypertension and obesity data are for the years 2009 and 2011, respectively, and the diabetes and smoking prevalence data are for 2012. The IHME reports hypertension and obesity data for females and males separately, so we construct a population-weighted average measure for these two covariates based on the proportion of females and males in each county, as reported by the U.S. Census Bureau.

The social distancing restrictions data is from the University of Washington’s State-Level Social Distancing Policies in Response to the 2019 Novel Coronavirus in the U.S. repository [21]. The social distancing restrictions include: (1) restrictions on public gatherings exceeding 5, 10, 25, 50, 100, 250, 500, or 1000 people, (2) limits on restaurant operations, (3) closure of specific businesses, e.g. fitness centres, gyms, casinos, etc., (4) closure of non-essential businesses, (5) stay-at-home orders for non-essential activities, (6) state curfews on non-essential activities, (7) mandated quarantines for people entering the state, (8) travel restrictions prohibiting residents from leaving the state, non-residents from entering the state, or residents from travelling across counties within the state, (9) self-isolation requirement for individuals with confirmed COVID-19 incidence, and (10) mandatory wearing of masks or other mouth and nose coverings in public places. We construct our social distancing restrictions index by adding the number of restrictions that are in place in a state on any given day, based on the date at which each restriction is enacted, relaxed, or expired.

We obtain our mobility data from the Descartes Labs [22]. This data consists of mobility indexes calculated at the end of every day and aggregated at the county level. The indexes, which we will refer to as the social mobility indexes, are based on geolocation reports from smartphones and other mobile devices, and track the movements of individual mobile phone subscribers. The methodology employed to construct these indexes is described in Warren et al. [23]. The mobility index data is available at a daily frequency from March 1, 2020, for 2669 counties.

Finally, we construct a comprehensive list of protests that took place across the U.S. based on the List of George Floyd protests in the United States assembled by Wikipedia [24]. At the time of writing, the main Wikipedia page cited 134 news articles from national, regional, and local media outlets, and the secondary pages cited hundreds more. From these media citations, we extracted the location and the date at which the protests reportedly took place, as well as the estimated number of individuals involved in each protest. We complement this process with a search on the Dow Jones Factiva database [25]. The onset of the protests among the counties in which protests took place, i.e. the treatment, is staggered across time and ranges between May 26, 2020, and June 7, 2020, so we center our experiment on the first protest date in each treated county, as opposed to the date of George Floyd’s death, May 24, 2020. Therefore, the George Floyd protests produce a quasi-natural experimental setting with staggered treatment dates, rather than a single treatment date setting.

Our sample period begins on March 1, 2020, when the social mobility data becomes available and ends on July 7, 2020. This ending date enables us to carry out our estimation on a balanced panel dataset consisting of a 30-day event window centered on the onset of the protests in each protest county. Our sampling procedure yields a panel dataset consisting of a total of 256,202 county-days representing 2617 (541 protest and 2076 non-protest) counties from all fifty states with incidence rate and covariates data available for our entire estimation window. From this dataset, we form covariate-balanced treatment and control groups using the propensity score matching technique described below and carry out our estimation of the treatment effect.

We report descriptive statistics for new and cumulative SARS-CoV-2 cases in Table 1, broken down by state, along with the total number of counties and the total number of county-days represented in our sample. In Table 2, we report the earliest and the latest ‘first protest’ date within each state’s counties, along with the size of the protest, according to media reports. We provide a map of the continental U.S. in Fig. 1, which reveals the geographic distribution of counties where protests took place along with the size of the first protest that took place within them. Figure 2 shows the evolution of our social distancing restrictions index for a selection of states. Figure 3 shows the social mobility index for a small and a large county in the states of New York and Alabama.

We examine the impact of the spontaneous relaxation of social distancing behavior that was brought on by the George Floyd protests across the U.S. on the SARS-CoV-2 incidence rate with an Ordinary Least Squares (OLS) staggered differences-in-differences (DID) panel regression equation, which is specified as follows:where \(IR_{i,j,t}\), the incidence rate, corresponds to the number of new SARS-CoV-2 infections in county i from state j on day t, per 100,000 population. \(Post_{FPi,j,t}\) is an indicator variable that is set equal to one on the day where protests begin in county i, as well as every day thereafter, and to zero otherwise. This indicator variable is set to zero on each day t for non-protest counties included in our control group. \({\mathbf {X}}_{i,j,t}\) and \({\mathbf {Y}}_{j,t}\) are vectors of county-level and state-level characteristics, which we use as control variables. \(\gamma _{i}\) captures time-invariant state fixed effects, and \(\eta _{t}\) represents time (day) fixed effects to control for changes in the aggregate SARS-CoV-2 incidence rate and common trends between our treatment and control group counties over time.

In Eq. (1), \(\alpha\) is a constant term and \(\beta _{1}\) captures the impact of the relaxation of social distancing brought on by the protests on the SARS-CoV-2 incidence rate. Hence, \(\beta _{1}\) is the parameter of interest in this regression. Under the null hypothesis that the relaxation of social distancing behavior has no causal impact on the SARS-CoV-2 incidence rate, \(\beta _{1}\) should be statistically indistinguishable from zero. We cluster the standard errors at the county level to account for any potential cross-sectional and time-serial dependence in the error terms, \(\epsilon _{i,j,t}\) [26, 27]. We perform our statistical analysis with STATA 16 and use and use the REGHDFE command to estimate Eq. (1) [28].

We include county-level control variables that may influence the incidence rate of SARS-CoV-2 in our staggered DID regression specification. These control variables account for demographic, health, living proximity, and income level variations across counties. For demographic controls, we include sex (Male) and age (60 years+) since these factors are associated with both an increased risk of testing positive for SARS-CoV-2 and greater illness severity [29]. We also include ethnicity, i.e., Asian, Black, Hispanic, and White, as demographic control variable, to account for the increased risk of a positive SARS-COV-2 test observed among certain ethnicities, especially Blacks and Hispanics. Our demographic variables are expressed in decimals, and represent the fraction of a county’s total population that falls in a particular group, based on the U.S. Census Bureau’s county-level population statistics for 2018. We include Diabetes prevalence, Hypertension prevalence, Obesity prevalence, and Smoking prevalence as health control variables. Obesity, diabetes, and hypertension are clinical risk factors that are associated with an increased risk of severe illness, and a greater risk of mortality from COVID-19 [30]. Smoking is also a clinical risk factor, as some evidence suggests that smoking may be associated with an increased severity of COVID-19 [31]. We include the natural logarithm of population density, ln(Population density), among our control variables, as higher incidence rates of SARS-CoV-2 are observed in more densely populated, urban, areas [30, 32]. Finally, consistent with previous research showing that residents from more economically deprived areas are more likely to test positive for SARS-COV-2, we use the natural logarighm of real GDP per capita, ln(Per capita RGDP), to control for income in our regressions [30].

In the period preceding the onset of the protests, the number of new COVID-19 cases began to drop steadily across the country [3]. Accordingly, several states began to relax their social distancing restrictions in a carefully staged manner. Figure 2 illustrates this trend in Alabama, California, Florida, and New York, for instance. Starting in mid-March, we observe a steady rise in our social distancing restrictions index in these four states and we observe the start of a slow unwind by mid-April. Notably, while social distancing restrictions were being relaxed across the nation, social mobility was also on the rise (see Fig. 3). The concurrent relaxation of social distancing restrictions and the increase in social mobility around the onset of the protests may very well have contributed to an increase in the SARS-CoV-2 incidence rate during the event period that is unrelated to the protests, so we include our social distancing restrictions and social mobility indexes in our DID regression equation (1), as additional control variables.

The first panel of Table 3 reveals statistically significant differences between protest and non-protest counties included in our sample on just about every dimension represented by the covariates introduced in the previous sub-section, barring the proportion of blacks included in the two groups. Non-protest counties have a significantly higher proportion of males, whites, 60-years+, are less healthy and wealthy, live in less densely populated areas, and are significantly more socially mobile than their counterparts from protest counties. These differences between the two groups may introduce selection bias into our experiment. This is a common concern with observational studies, such as the present one, where the subjects are not randomly assigned to the treatment and control groups by the researcher [33]. To ensure that our control group is as similar as possible to our treatment group from the perspective of all these covariates, i.e., to mimimize any potential selection bias in our experiment, we form our treatment and control groups using the propensity score matching technique [34]. In the context of our experiment, the propensity score represents the estimated likelihood that a county will experience an increase in its SARS-CoV-2 infection rate.

Essentially, the matching process begins with a logistic regression in which the dependent variable is set to one for the 541 protest (treated) counties included in our sample, and to zero for the remaining 2077 non-protest (untreated) counties. The independent variables included in this regression correspond to our covariates, all of which have been shown to influence the likelihood of contracting SARS-CoV-2. Next, we match treated counties to their nearest neighbour from the untreated group, without replacement, with standard caliper of 0.25 standard deviations, based on the propensity scores from the logistic regression [35, 36]. This process yields a balanced sample consisting of 356 treated and 356 untreated counties. As Table 3 shows, from the perspective of our covariates, these two groups do not exhibit any statistically significant differences from each other, with the exception of Hypertension prevalence, which is significantly higher in our treatment group than in our control group, albeit at the 5% level.

Our quasi-natural experimental setting satisfies at least two key requirements for the identification of the causal link between social distancing and the spread of SARS-CoV-2, namely: (1) the existence of a strong theoretical basis supporting the relationship in question and, (2) exogenous variation in the variable of interest, i.e social distancing [37]. The presence of an exogenous shock in our setting, i.e., protests arising spontaneously in some counties as a result of a tragic event, is key to establish causality, as this mitigates concerns that omitted variables correlated with both the protests and the spread of SARS-CoV-2 might be driving our findings. This setting also minimizes concerns about endogeneity and self-selection, which beset most non-randomized-trial experiments.

In sum, thanks to the covariate balance that we are able to achieve with our propensity score matching process, our staggered DID regression specification is uniquely well positioned to separate the impact of the relaxation of social distancing behaviour on the SARS-CoV-2 incidence rate from other factors that may potentially affect the spread of the disease. Next, to address any potential concerns that our findings may be contaminated by confounding events, we exclude from our regression the county-days that fall outside of the [− 30, + 30]-day event window centered on the day when protests begin in a protest county [33, 38].

We report results from regression equation (1) in Table 4. The coefficient of interest in this regression is \(\beta _1\), which is associated with \(Post_{FP}\), our post-protest indicator variable. This coefficient is positive and highly statistically significant (1.34; 95% CI 0.21–2.47), implying that the SARS-CoV-2 incidence rate increases by 1.34 cases per day, per 100,000, on average, following the onset of the protests in protest counties, relative to their propensity score matching non-protest counterparts. To put this finding into perspective, recall that the average number of new cases across all counties is equal to 5 per day, per 100,000 population, in the week preceding the onset of the protests (see Column (2) of Table 1). Using this number as a reference point, this finding suggests that the SARS-CoV-2 incidence rate increases by 1.34/5 = 26.8% following the onset of the protests, due to the relaxation of social distancing brought on by the protests.

Even if our observed covariates are well-balanced, one still needs to assess whether the parallel trends assumption underpinning the DID design is satisfied. We assess whether pre-treatment trends for our treatment and control groups are parallel by estimating a “leads and lags model” [39]. In this model, we replace our \(Post_{FP}\) indicator variable in Eq. (1) with a family of period-specific indicator variables spanning the pre- and post-protest event window. Each indicator variable is set equal to one for treated counties for a specific 5-day period surrounding the onset of the protests, and to zero otherwise. Under the null hypothesis that pre-treatment trends are parallel, the coefficients associated with the pre-treatment indicator variables should not exhibit any pattern and should be statistically insignificant. Meanwhile, the coefficients associated with the post-treatment indicator variables will reveal the treatment effect as it manifests itself in the data during the post-protest period.

Figure 4 plots the value of the coefficients associated with our pre- and post-protest indicator variables. In this figure, p corresponds to the five-day period starting on the protest date and ending 4 days later, i.e., [0, 4], + 1p is for days [5, 9], and -1p is for days [5, 1]. We don’t observe any clear trend in the pre-treatment periods and none of the coefficients are statistically different from zero, suggesting that the parallel trends assumption is satisfied. Post-protest, we observe a clear upward trend in the magnitude of the coefficients, which is reversed in period + 4p. The treatment effect becomes statistically different from zero in period + 2p, roughly ten days following the onset of the protests. This is consistent with SARS-CoV-2’s incubation period and typical testing wait times. Finally, we note the attenuation of the treatment effect in period + 4p. This is to be expected, as the impact of the relaxation of social distancing brought on by the protests must eventually die out. In sum, the treatment effect documented in Table 4 unfolds over time in a manner that supports the hypothesis that social distancing causally impacts the spread of SARS-CoV-2.

We conduct a placebo test to assess whether the causal impact of the protests on the spread of SARS-CoV-2 that we document in Table 4 can be attributed to chance. For this purpose, we implement a Monte Carlo simulation exercise centered on our staggered DID panel regression specification, i.e., Eq. (1). In each iteration of this simulation, we assign 541 counties randomly to the potential treatment group and the remaining 2077 counties to the potential control group. We then implement our propensity score matching process to create a balanced sample of treated and control counties. We perform this matching process without replacement with the 0.25 standard deviation caliper, as per “Propensity score matching” section. Next, we assign a [− 30, + 30]-day event period to each treated county randomly with start dates ranging between March 1, 2020, and May 8, 2020. Then, we align each control group county’s timeline to its treated counterpart’s event timeline and create the \(Post_{FPi,j,t}\) indicator variable. Once this step has been completed, we estimate our staggered DID regression specification on the simulated sample and collect the \(\beta _{1}\) coefficient estimate, along with its county-cluster robust t-statistic. We implement this process 5000 times to produce the simulated distribution of \(\beta _{1}\) coefficients and associated statistics. If the \(\beta _{1}\) estimate from Table 4 lies above the 95% threshold from the distribution of simulated \(\beta _{1}\) coefficient estimates, we can conclude with a high level of confidence that the treatment effect that we document in this paper cannot be attributed to chance.

We present results from this placebo test in Table 5. The 95% and 99% threshold values for the \(\beta _{1}\) coefficient from the simulated distribution are equal to 0.57 and 1.42, respectively, while our empirical estimate in Table 4 is equal to 1.34. Likewise, the 95% and 99% threshold values for the robust t-statistics from the simulated distribution are equal to 0.44 and 1.01, respectively, while the robust t-statistic associated with our \(\beta _{1}\) coefficient estimate in Table 4 is equal to 2.32. Since our \(\beta _{1}\) estimate and its associated robust t-statistic are well beyond their respective 95% simulated threshold values, we can safely reject the null hypothesis that relaxing social distancing behavior has no impact on the spread of SARS-CoV-2 and, with a high degree of confidence, we can rule out the possibility that the treatment effect that we document in Table 4 is attributable to chance.

Early predictive models assessing the effectiveness of social distancing have suggested that a greater spread of SARS-CoV-2 would occur in the absence of social distancing measures [40‐42]. Similarly, our study demonstrates that when social distancing is reduced, i.e., by individuals protesting in close proximity, the spread of SARS-CoV-2 increases. Our study differs from its predecessors because instead of examining the effectiveness of social distancing measures following their imposition [11, 12, 14], we examine the impact of social distancing on the spread of COVID-19 when social distancing behavior is abruptly relaxed. Additionally, unlike previous studies, we do not use mobility as a measure of social distancing, instead we use social mobility as a control variable in our analyses. By explicitly controlling for the concurrent increase in social mobility and the relaxation of state-imposed social distancing restrictions during the period surrounding the protests, our study demonstrates that social distancing directly impacts the spread of SARS-CoV-2. We also control for a host of covariates known to influence the transmission of SARS-CoV-2, and implement placebo tests to rule out the possibility that our results are attributable to chance. Therefore, we can be confident that the increase in SARS-CoV-2 incidence that we observe following the onset of the protests can be attributed to the relaxation of social distancing behavior.

Our study is not without limitations. In particular, over 70 testing centers across the U.S. were closed following the onset of the protests. We are also unable to assess protest participants’ vulnerability (e.g. age, underlying health conditions, personal protective wear, etc.), and variability along these dimensions may influence the risk of SARS-CoV-2 incidence. Additionally, we cannot control for the actual degree of physical proximity between participants, which would impact the transmission rate of SARS-CoV-2 during the protests. We are also unable to control for any potential under-reporting of COVID-19 cases over time and across counties [43]. This would be a concern if protest counties and non-protest counties were impacted differently by this phenomenon. Moreover, we rely on the accuracy of media reports to identify the counties in which protests took place. Finally, we do not account for the magnitude of the protests in each county, however, expressing the case counts in rates rather than in levels should minimize any potential scale-related effects.