Regression discontinuity inference with specification error

doi:10.1016/j.jeconom.2007.05.003

Journal of Econometrics

Volume 142, Issue 2, February 2008, Pages 655-674

https://doi.org/10.1016/j.jeconom.2007.05.003 Get rights and content

Abstract

A regression discontinuity (RD) research design is appropriate for program evaluation problems in which treatment status (or the probability of treatment) depends on whether an observed covariate exceeds a fixed threshold. In many applications the treatment-determining covariate is discrete. This makes it impossible to compare outcomes for observations “just above” and “just below” the treatment threshold, and requires the researcher to choose a functional form for the relationship between the treatment variable and the outcomes of interest. We propose a simple econometric procedure to account for uncertainty in the choice of functional form for RD designs with discrete support. In particular, we model deviations of the true regression function from a given approximating function—the specification errors—as random. Conventional standard errors ignore the group structure induced by specification errors and tend to overstate the precision of the estimated program impacts. The proposed inference procedure that allows for specification error also has a natural interpretation within a Bayesian framework.

Introduction

In the classic regression-discontinuity (RD) design (Thistlethwaite and Campbell, 1960) the treatment status of an observation is determined by whether an observed covariate is above or below a known threshold. If the covariate is predetermined it may be plausible to think that treatment status is “as good as randomly assigned” among the subsample of observations that fall just above and just below the threshold.¹ As in a true experiment, no functional form assumptions are necessary to estimate program impacts when the treatment-determining covariate is continuous: one simply compares average outcomes in small neighborhoods on either side of the threshold. The width of these neighborhoods can be made arbitrarily small as the sample size grows, ensuring that observed and unobserved characteristics of observations in the treatment and control groups are identical in the limit. This idea underlies the approach of Hahn et al. (2001) and Porter (2003), who describe non-parametric and semi-parametric estimators of RD gaps.

In many applications where the RD design seems compelling, however, the covariate that determines treatment is inherently discrete or is only reported in coarse intervals. For example, government programs like Medicare and Medicaid have sharp age-related eligibility rules that lend themselves to an RD framework, but in most data sets age is only recorded in months or years. In the discrete case it is no longer possible to compute averages within arbitrarily small neighborhoods of the cutoff point, even with an infinite amount of data. Instead, researchers have to choose a particular functional form for the model relating the outcomes of interest to the treatment-determining variable. Indeed, with an irreducible gap between the “control” observations just below the threshold and the “treatment” observations just above, the causal effect of the program is not even identified in the absence of a parametric assumption about this function.

In this paper we propose a simple procedure for inference in RD designs in which the treatment-determining covariate is discrete. The basic idea is to model the deviation between the expected value of the outcome and the predicted value from a given functional form as a random specification error. Modeling potential specification error in this way has a number of immediate implications. Most importantly, it introduces a common component of variance for all the observations at any given value of the treatment-determining covariate. This creates a problem similar to the one analyzed by Moulton (1990) for multi-level models in which some of the covariates are only measured at a higher level of aggregation (e.g., micro models with state-level covariates). Random specification errors can be easily incorporated in inference by constructing sampling errors that include a grouped error component for different values of the treatment-determining covariate. The use of “clustered” standard errors will generally lead to wider confidence intervals that reflect the imperfect fit of the parametric function away from the discontinuity point.

More subtly, inference in an RD design involves extrapolation from observations below the threshold to construct a counterfactual for observations above the threshold. As in a classic out-of-sample forecasting problem, the sampling error of the counterfactual prediction for the point of support just beyond the threshold includes a term reflecting the expected contribution of the specification error at that point. Since the estimated (local) treatment effect is just the difference between the mean outcome for these observations and the counterfactual prediction, the precision of the estimated treatment effect depends on whether one assumes that the same specification error would prevail in the counterfactual world. If so, this error component vanishes. If not, the confidence interval for the local treatment effect has to be widened even further.

The paper is organized as follows. Section 2 describes the RD framework and why discreteness in the treatment-determining covariate implies that the treatment effect is not identified without assuming a parametric functional form. Section 3 describes the proposed inference procedure under a model where specification errors are considered random. Section 4 describes a modified procedure under less restrictive assumptions about the specification errors. Section 5 proposes an alternative, efficient estimator for the treatment effect, and Section 6 relates this estimator to a Bayesian approach. Section 7 concludes.

Section snippets

The problem of discreteness

To illustrate how discreteness causes problems for identification in an RD framework, consider the following potential outcomes formulation.² There is a binary indicator D of treatment status which is determined by whether an observed covariate X is above or below a known threshold $x_{0}$ : $D = 1 [X ⩾ x_{0}]$ . Let $Y_{1}$ represent the potential outcome if an observation receives

Random specification error

Suppose a polynomial is chosen to approximate $h (\cdot)$ . The regression in Eq. (2) can be re-written as $Y_{ij} = α_{0} + D_{j} β_{0} + X_{j} γ_{0} + a_{j} + ɛ_{ij},$ where $X_{j}$ is a row vector of polynomial terms in $x_{j}$ (with the normalization $x_{k} = 0$ ), and $a_{j} \equiv h (x_{j}) - X_{j} γ_{0}$ is specification error—the degree to which the true function $h (\cdot)$ deviates from the polynomial function.⁵

Mis-specification of counterfactual functions

In this section, we show that the special structure of an RD design implies that in some circumstances, the clustered standard errors may still understate the variability of $\hat{β}$ . If the specification error is random, then it is necessary to decide how the error in estimating $E [Y_{1} | X = x_{k}]$ is related to the specification error in estimating $E [Y_{0} | X = x_{k}]$ . As shown below, if the errors are assumed to be identical, then the approach described above is appropriate. If the errors are independent, then the

Efficient estimation

When the specification errors $a_{1 j}$ and $a_{0 j}$ are assumed to be different, there is an estimator for $E [Y_{1} - Y_{0} | X = 0]$ that is more efficient than the OLS estimator $\hat{β}$ . This is because the least squares estimate of $β_{0}$ amounts to the difference between the prediction for $E [Y_{1} | X = 0]$ and the prediction for $E [Y_{0} | X = 0]$ , using data away from the discontinuity threshold. While it is necessary to make such an extrapolation for $E [Y_{0} | X = 0]$ (since this quantitity is unobservable), information on $E [Y_{1} | X = 0]$ is available

Relation to Bayesian estimation

There is a close connection to the proposed estimator $\hat{β^{*}}$ and a Bayesian approach to the problem. Specifically, the confidence intervals proposed above can be interpreted as Bayesian posterior intervals.

For example, note that (14) can be re-written as $β^{*} = [λ {\bar{Y}}_{k} + (1 - λ) (\hat{α} + \hat{β})] - \hat{α} .$ The expression in brackets can be viewed as an estimate of $E [Y_{1} | X = 0]$ —a $λ$ -weighted average of the kth cell mean and the predicted value from the regression—and the term $\hat{α}$ as an estimate of $E [Y_{0} | X = 0]$ .

Consider a simple

Summary

This paper draws attention to functional form issues in the estimation of RD designs when the index variable determining treatment, X, has discrete support. In the discrete case, the conditions for non-parametric or semi-parametric methods are not satisfied; indeed, the treatment effect is not non-parametrically identified. Our goal is to formally incorporate uncertainty in the necessary parametric modeling of the underlying RD function.

We have proposed a procedure for inference that explicitly

Acknowledgments

We are grateful to Guido Imbens and Thomas Lemieux for helpful suggestions, and to Michael Jansson, James Powell, Keisuke Hirano, Bill Evans, and participants in the 2003 Banff International Research Station Regression Discontinuity Conference for helpful discussions and suggestions.

References (15)

J. Angrist et al.
Empirical strategies in labor economics
J. Angrist et al.
Using Maimonides’ rule to estimate the effect of class size on scholastic achievement
Quarterly Journal of Economics
(1998)
R.L. Brown et al.
Techniques for testing for the constancy of regression relationships over time (with discussion)
Journal of the Royal Statistical Society B
(1975)
D. Card et al.
Using discontinuous eligibility rules to identify the effects of the federal medicaid expansions on low income children
Review of Economics and Statistics
(2004)
G. Chamberlain
Quantile regression, censoring, and the structure of wages
J. DiNardo et al.
Economic impacts of new unionization on private sector employers: 1984–2001
Quarterly Journal of Economics
(2004)
J. Hahn et al.
Identification and estimation of treatment effects with a regression–discontinuity design
Econometrica
(2001)

There are more references available in the full text version of this article.

Cited by (648)

Keeping refugee children in school and out of work: Evidence from the world's largest humanitarian cash transfer program
2024, Journal of Development Economics
This paper investigates whether unconditional cash transfers can keep refugee children in school and out of work. We raise this question in the unique context of Turkey, which hosts the world's largest refugee population (including 3.6 million Syrians). Refugees in Turkey are supported by the world's largest cash transfer program for refugees, the Emergency Social Safety Net (ESSN). We exploit a program eligibility criterion to identify the causal impacts of the ESSN program using a regression discontinuity design. The results show a large effect on child labor and school enrollment among both male and female refugee children. Being a beneficiary household reduces the fraction of children working from 14.0% to 1.6% (a decrease of 88%) and the fraction of children aged 6–17 not in school from 36.2 to 13.7% (a reduction of 62%). By unpacking the mechanisms at play, we show that ESSN cash transfers become a significant part of a household's income, substantially alleviate extreme poverty, and reduce a family's need to resort to harmful coping strategies. Investigating the reasons for children not attending school, we find that the beneficiary households become more likely to send children to school because the cash transfer addresses both the opportunity cost and direct cost of schooling—although the former channel is more important. The findings have important implications for the design of policies aimed at supporting refugee children at scale.
Reducing carbon emissions at the expense of firm physical capital investments and growing financialization? Impacts of carbon trading policy from a regression discontinuity design
2024, Journal of Environmental Management
This study examines the effects of China's carbon trading policy on firm emissions and explores its impact mechanisms through financial and physical asset investments. The empirical analysis utilizes a fuzzy regression discontinuity design based on a sample of 427 industrial firms in China between 2014 and 2019. The results indicate that China's carbon trading policy incentivized firms to increase their financial investments while simultaneously discouraging physical capital investments. These shifts in investment patterns helped firms achieve their emission reduction targets. The study reveals that carbon trading policy in China has contributed to the financialization of firms, resulting in the erosion of firm assets and a decline in their overall competitiveness. Based on these findings, some policy recommendations are put forward.
Fertility responses to cash transfers in Uruguay
2024, World Development Perspectives
Conditional cash transfer (CCT) programs have been the most used tool to reduce poverty and inequality in developing countries in the last decades. In addition to the objectives pursued by these programs, it has been shown that they can have unintended effects on different dimensions. Particularly, they can have an impact on fertility due to an increase in the household's income. This paper examines the relationship between non-labor income and women's childbearing behavior in a developing country. The assignment mechanism of the Uruguayan cash transfer program Asignaciones Familiares – Plan de Equidad (AFAM-PE) alters non-labor incomes across the applicant’s households. I estimate the impact of this program on women's fertility and teenage pregnancy. The identification strategy exploits the discontinuity present in the program eligibility criteria. I combined longitudinal vital statistics provided by the Ministry of Public Health and administrative data to assemble a panel of AFAM-PE applicants aged between 15 and 49 (in 2008 and 2009). The study finds no statistically significant impact of AFAM-PE on fertility rates or teenage pregnancy. These results are robust to different specifications and women samples. This provides evidence against the idea that transfer programs targeting disadvantaged individuals generate a direct effect on fertility.
The value of improving insurance quality: Evidence from long-run Medicaid attrition
2024, Journal of Health Economics
The US government increasingly provides public health insurance coverage through private firms. We examine associated welfare implications for beneficiaries, using a ‘revealed preference’ framework based on beneficiaries’ program attrition rates. Focusing on the Medicaid program in New York State, we exploit quasi-random variation in the initial assignment at birth to public versus private Medicaid based on birth weight. We find that infants assigned to private Medicaid at birth are less likely to subsequently leave Medicaid. We provide suggestive evidence that reduced attrition reflects beneficiary responses to improved program quality, rather than alternative mechanisms such as private Medicaid plans reducing re-enrollment barriers.
The effects of superstition on firms' investment behavior: Evidence from Vietnam, an irreligious country<sup>✰</sup>
2024, Journal of Comparative Economics
This study examines the impact of superstition on corporate decision-making in Vietnam, a highly irreligious country. We focus on the folk belief that the ages of 49–53 are considered calamitous and use a regression discontinuity design to show that companies significantly decrease their investment in fixed assets during these ages of their directors. The effect is more pronounced in smaller firms and is not accompanied by a decrease in employment growth. We introduce a novel two-stage difference method to identify the role of superstition in causing the ‘calamitous ages’ effect.
ESG and CEO turnover around the world
2024, Journal of Corporate Finance
We investigate whether CEOs around the world are held accountable for stakeholder-related corporate misbehavior. The likelihood of CEO turnover increases significantly when the media coverage of the ESG incidents reaches extreme levels. CEO turnovers occur even in the cases where an incident does not lead to a stock price decline. In such cases, the board likely has a non-pecuniary motive for the turnover. This suggests that such non-pecuniary reputational concerns are an important determinant of CEO turnover decisions around the world, especially when the firm is facing intense public pressure due to stakeholder-related corporate misbehavior. This effect is more pronounced when firms are headquartered in stakeholder-oriented countries like many European countries.

View all citing articles on Scopus

View full text

Regression discontinuity inference with specification error

Abstract

Introduction

Section snippets

The problem of discreteness

Random specification error

Mis-specification of counterfactual functions

Efficient estimation

Relation to Bayesian estimation

Summary

Acknowledgments

Empirical strategies in labor economics

Using Maimonides’ rule to estimate the effect of class size on scholastic achievement

Quarterly Journal of Economics

Techniques for testing for the constancy of regression relationships over time (with discussion)

Journal of the Royal Statistical Society B

Using discontinuous eligibility rules to identify the effects of the federal medicaid expansions on low income children

Review of Economics and Statistics

Quantile regression, censoring, and the structure of wages

Economic impacts of new unionization on private sector employers: 1984–2001

Quarterly Journal of Economics

Identification and estimation of treatment effects with a regression–discontinuity design

Econometrica