nach oben

Health Services and Outcomes Research Methodology

Erschienen in:

26.02.2021

Heterogeneous treatment effects and bias in the analysis of the stepped wedge design

verfasst von: Stephan Lindner, K. John McConnell

Erschienen in: Health Services and Outcomes Research Methodology | Ausgabe 4/2021

Einloggen, um Zugang zu erhalten

Abstract

The effect of an intervention in a stepped wedge design can vary across clusters or with time since exposure to treatment, but consequences of such heterogeneous treatment effects for the analysis of stepped wedge designs are not well recognized. In this article, we advance the idea that the stepped wedge design can be framed as a special case of a difference-in-differences design with staggered treatment exposure. Using this perspective, we show that the standard difference-in-differences regression approach estimates the average treatment effect of the treatment period with bias when treatment effects vary by cluster or time since exposure. We then use Monte-Carlo simulations of stepped wedge designs to examine the performance of the standard regression approach as well as alternative approaches that estimate treatment effects relative to intervention exposure. Simulation results confirm that estimates from the standard approach are biased when treatment effects are heterogeneous. Alternative regression approaches may perform better, but only if the research design allows estimation of the full treatment effect across all clusters. We conclude by offering recommendations for the design and analysis of stepped wedge trials.

Nur mit Berechtigung zugänglich

We would like to thank two anonymous reviewers for this suggestions.

Abraham, S., Sun, L.: Estimating dynamic treatment effects in event studies with heterogeneous treatment effects. Technical report, MIT (2018)

Allen, L.A., McIlvennan, C.K., Thompson, J.S., Dunlay, S.M., LaRue, S.J., Lewis, E.F., Patel, C.B., Blue, L., Fairclough, D.L., Leister, E.C., Glasgow, R.E., Cleveland, J.C., Phillips, C., Baldridge, V., Walsh, M.N., Matlock, D.D.: Effectiveness of an intervention supporting shared decision making for destination therapy left ventricular assist device. JAMA Int. Med. 178(4), 520 (2018). https://doi.org/10.1001/jamainternmed.2017.8713CrossRef

Angrist, J.D., Pischke, J.S.: Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press, Princeton (2008)CrossRef

Callaway, B., Sant’Anna, P.H.: Difference-in-differences with multiple time periods and an application on the minimum wage and employment. Technical report, SSRN Working Paper 3148250 (2018)

Cameron, A.C., Miller, D.L.: A practitioner’s guide to cluster-robust inference. J. Human Resour. 50(2), 317–372 (2015). https://doi.org/10.3368/jhr.50.2.317CrossRef

Davey, C., Hargreaves, J., Thompson, J.A., Copas, A.J., Beard, E., Lewis, J.J., Fielding, K.L.: Analysis and reporting of stepped wedge randomised controlled trials: Synthesis and critical appraisal of published studies, 2010 to 2014. Trials 16(1), 358 (2015). https://doi.org/10.1186/s13063-015-0838-3CrossRefPubMedPubMedCentral

Daw, J.R., Hatfield, L.A.: Matching and regression to the mean in difference-in-differences analysis. Health Services Res. 53(6), 4138–4156 (2018)CrossRef

Ellenberg, S.S.: The stepped-wedge clinical trial. JAMA 319(6), 607 (2018). https://doi.org/10.1001/jama.2017.21993CrossRefPubMed

Golden, M.R., Kerani, R.P., Stenger, M., Hughes, J.P., Aubin, M., Malinski, C., Holmes, K.K.: Uptake and population-level impact of expedited partner therapy (ept) on chlamydia trachomatis and neisseria gonorrhoeae: the washington state community-level randomized trial of ept. PLOS Med. 12(1), e1001777 (2015). https://doi.org/10.1371/journal.pmed.1001777CrossRefPubMedPubMedCentral

Goodman-Bacon, A.: Difference-in-differences with variation in treatment timing. Tech. rep., NBER Working Paper No. 25018 (2018)

Hemming, K., Girling, A.: The efficiency of stepped wedge vs cluster randomized trials: Stepped wedge studies do not always require a smaller sample size. J. Clin. Epidemiol. 66(12), 1427 (2013)CrossRef

Hemming, K., Haines, T.P., Chilton, P.J., Girling, A.J., Lilford, R.J.: The stepped wedge cluster randomised trial: Rationale, design, analysis, and reporting. BMJ 350(1), h391–h391 (2015). https://doi.org/10.1136/bmj.h391CrossRefPubMed

Holland, P.W.: Statistics and causal inference (with discussion). J. Am. Stat. Assoc. 81, 945–970 (1986). https://doi.org/10.2307/2289064CrossRef

Huffman, M.D., Mohanan, P.P., Devarajan, R., Baldridge, A.S., Kondal, D., Zhao, L., Ali, M., Krishnan, M.N., Natesan, S., Gopinath, R., Viswanathan, S., Stigi, J., Joseph, J., Chozhakkat, S., Lloyd-Jones, D.M., Prabhakaran, D.: Effect of a quality improvement intervention on clinical outcomes in patients in india with acute myocardial infarction. JAMA 319(6), 567 (2018). https://doi.org/10.1001/jama.2017.21906CrossRefPubMedPubMedCentral

Hughes, J.P., Granston, T.S., Heagerty, P.J.: Current issues in the design and analysis of stepped wedge trials. Contemp. Clin. Trials 45, 55–60 (2015)CrossRef

Hussey, M.A., Hughes, J.P.: Design and analysis of stepped wedge cluster randomized trials. Contemp. Clin. Trials 28(2), 182–191 (2007). https://doi.org/10.1016/j.cct.2006.05.007CrossRefPubMed

Imbens, G.W., Rubin, D.B.: Rubin causal model. New Palgrave Dictionary Econ. pp. 1–10 (2017)

Kahn-Lang, A., Lang, K.: The promise and pitfalls of differences-in-differences: Reflections on ’16 and pregnant’and other applications. Technical report National Bureau of Economic Research (2018)

Kennedy-Shaffer, L., Gruttola, V., Lipsitch, M.: Novel methods for the analysis of stepped wedge cluster randomized trials. Stat. Med. 39(7), 815–844 (2019). https://doi.org/10.1002/sim.8451CrossRefPubMedPubMedCentral

Lechner, M.: The estimation of causal effects by difference-in-difference methods estimation of spatial panels. Found. Trends Econ. 4(3), 165–224 (2010). https://doi.org/10.1561/0800000014CrossRef

Lindner, S., McConnell, K.J.: Difference-in-differences and matching on outcomes: a tale of two unobservables. Health Serv. Outcomes Res. Methodol. 19(2–3), 127–144 (2019). https://doi.org/10.1007/s10742-018-0189-0CrossRef

Lindner, S., Kaufman, M.R., Marino, M., O’Malley, J., Angier, H., Cottrell, E.K., McConnell, K.J., DeVoe, J.E., Heintzman, J.R.: A medicaid alternative payment model program in oregon led to reduced volume of imaging services. Health Affairs 39(7), 1194–1201 (2020). https://doi.org/10.1377/hlthaff.2019.01656CrossRefPubMed

Matthews, J.N.S., Forbes, A.B.: Stepped wedge designs: Insights from a design of experiments perspective. Stat. Med. 36(24), 3772–3790 (2017). https://doi.org/10.1002/sim.7403CrossRefPubMed

Nickless, A., Voysey, M., Geddes, J., Yu, L.M., Fanshawe, T.R.: Mixed effects approach to the analysis of the stepped wedge cluster randomised trial-investigating the confounding effect of time through simulation. PLoS ONE 13(12), e0208876 (2018). https://doi.org/10.1371/journal.pone.0208876CrossRefPubMedPubMedCentral

Parent, B., LaGrone, L.N., Albirair, M.T., Serina, P.T., Keller, J.M., Cuschieri, J., Addison, E.J., Choe, L., Delossantos, G.B., Gaskill, C.E., Moon, S.D., MacDonald, J.T., Stolzberg, M.J., Eaton, E.G.V., Zech, J.M., Kritek, P.A.: Effect of standardized handoff curriculum on improved clinician preparedness in the intensive care unit. JAMA Surgery 153(5), 464 (2018). https://doi.org/10.1001/jamasurg.2017.5440CrossRefPubMed

Rubin, D.B.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educat. Psychol. 66(5), 688–701 (1974). https://doi.org/10.1037/h0037350CrossRef

Ryan, A.M., Burgess, J.F., Dimick, J.B.: Why we should not be indifferent to specification choices for difference-in-differences. Health Service Res. 50(4), 1211–1235 (2015). https://doi.org/10.1111/1475-6773.12270CrossRef

Sikkens, J.J., van Agtmael, M.A., Peters, E.J.G., Lettinga, K.D., van der Kuip, M., Vandenbroucke-Grauls, C.M., Wagner, C., Kramer, M.H.: Behavioral approach to appropriate antimicrobial prescribing in hospitals. JAMA Int. Med. 177(8), 1130 (2017). https://doi.org/10.1001/jamainternmed.2017.0946CrossRef

Sun, L., Abraham, S.: Estimating dynamic treatment effects in event studies with heterogeneous treatment effects. Technical report, MIT (2020)

Thompson, J., Davey, C., Fielding, K., Hargreaves, J., Hayes, R.: Robust analysis of stepped wedge trials using cluster-level summaries within periods. Stat. Med. 37(16), 2487–2500 (2018). https://doi.org/10.1002/sim.7668CrossRefPubMedPubMedCentral

Thompson, J.A., Fielding, K.L., Davey, C., Aiken, A.M., Hargreaves, J.R., Hayes, R.J.: Bias and inference from misspecified mixed-effect models in stepped wedge trial analysis. Stat. Med. 36(23), 3670–3682 (2017)CrossRef

Woertman, W., de Hoop, E., Moerbeek, M., Zuidema, S.U., Gerritsen, D.L., Teerenstra, S.: Stepped wedge designs could reduce the required sample size in cluster randomized trials. J. Clin. Epidemiol. 66(7), 752–758 (2013)CrossRef

Zeileis, A.: Econometric computing with hc and hac covariance matrix estimators. J. Stat. Software (2004). https://doi.org/10.18637/jss.v011.i10CrossRef

Titel: Heterogeneous treatment effects and bias in the analysis of the stepped wedge design
verfasst von: Stephan Lindner
K. John McConnell
Publikationsdatum: 26.02.2021
Verlag: Springer US
Erschienen in: Health Services and Outcomes Research Methodology / Ausgabe 4/2021
Print ISSN: 1387-3741
Elektronische ISSN: 1572-9400
DOI: https://doi.org/10.1007/s10742-021-00244-w

Die Highlights vom Kongress des American College of Cardiology 2024

Springer Medizin

Heterogeneous treatment effects and bias in the analysis of the stepped wedge design

Abstract

Die Highlights vom Kongress des American College of Cardiology 2024

Springer Medizin

Abstract

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 4/2021

Modelling the size, cost and health impacts of universal basic income: What can be done in advance of a trial?

The ADI-3: a revised neighborhood risk index of the social determinants of health over time and place

Using a spatiotemporal model to estimate the impact of suicide prevention in small areas

Which patients benefit most from completing health risk assessments: comparing methods to identify heterogeneity of treatment effects

Distinguishing frontloading: an examination of medicare home health claims

Authentic assessments: a method to detect anomalies in assessment response patterns via neural network