Skip to main content
Erschienen in: Prevention Science 3/2019

21.07.2018

Sample Size Planning for Cluster-Randomized Interventions Probing Multilevel Mediation

verfasst von: Ben Kelcey, Jessaca Spybrook, Nianbo Dong

Erschienen in: Prevention Science | Ausgabe 3/2019

Einloggen, um Zugang zu erhalten

Abstract

Multilevel mediation analyses play an essential role in helping researchers develop, probe, and refine theories of action underlying interventions and document how interventions impact outcomes. However, little is known about how to plan studies with sufficient power to detect such multilevel mediation effects. In this study, we describe how to prospectively estimate power and identify sufficient sample sizes for experiments intended to detect multilevel mediation effects. We outline a simple approach to estimate the power to detect mediation effects with individual- or cluster-level mediators using summary statistics easily obtained from empirical literature and the anticipated magnitude of the mediation effect. We draw on a running example to illustrate several different types of mediation and provide an accessible introduction to the design of multilevel mediation studies. The power formulas are implemented in the R package PowerUpR and the PowerUp software (causalevaluation​.​org).
Fußnoten
1
Core assumptions include (a) stable unit treatment value assumption, (b) sequential ignorability, (c) consistency, (d) no downstream confounders, and (e) no treatment-by-mediator interaction.
 
Literatur
Zurück zum Zitat Cox, K. & Kelcey, B. (2018). Optimal Sample Allocation in Group-randomized Studies of Multilevel Mediation with a Group-level Mediator. Journal of Experimental Education (in press). Cox, K. & Kelcey, B. (2018). Optimal Sample Allocation in Group-randomized Studies of Multilevel Mediation with a Group-level Mediator. Journal of Experimental Education (in press).
Zurück zum Zitat Dong, N., Kelcey, B., Spybrook, J., & Maynard, R. A. (2016). PowerUp!-Mediator: A tool for calculating statistical power for causally-defined mediation in cluster randomized trials. (Version 0.4) [Software]. Available from http://www.causalevaluation.org/. Accessed 1 May 2007. Dong, N., Kelcey, B., Spybrook, J., & Maynard, R. A. (2016). PowerUp!-Mediator: A tool for calculating statistical power for causally-defined mediation in cluster randomized trials. (Version 0.4) [Software]. Available from http://​www.​causalevaluation​.​org/​. Accessed 1 May 2007.
Zurück zum Zitat Gottfredson, D. C., Cook, T. D., Gardner, F. E., Gorman-Smith, D., Howe, G. W., Sandler, I. N., & Zafft, K. M. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science, 16, 893–926.CrossRefPubMedPubMedCentral Gottfredson, D. C., Cook, T. D., Gardner, F. E., Gorman-Smith, D., Howe, G. W., Sandler, I. N., & Zafft, K. M. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science, 16, 893–926.CrossRefPubMedPubMedCentral
Zurück zum Zitat Kelcey, B., & Phelps, G. (2013). Considerations for Designing Group Randomized Trials of Professional Development with Teacher Knowledge Outcomes. Educational Evaluation and Policy Analysis, 35, 370–390. Kelcey, B., & Phelps, G. (2013). Considerations for Designing Group Randomized Trials of Professional Development with Teacher Knowledge Outcomes. Educational Evaluation and Policy Analysis, 35, 370–390.
Zurück zum Zitat Kelcey, B., Dong, N., Spybrook, J., & Cox, K. (2017). Statistical power for causally-defined indirect effects in group-randomized trials. Journal of Educational and Behavioral Statistics. Kelcey, B., Dong, N., Spybrook, J., & Cox, K. (2017). Statistical power for causally-defined indirect effects in group-randomized trials. Journal of Educational and Behavioral Statistics.
Zurück zum Zitat Kelcey, B., Dong, N., Spybrook, J., & Shen, Z. (2017a). Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators. Multivariate Behavioral Research, 52(6), 699–719. Kelcey, B., Dong, N., Spybrook, J., & Shen, Z. (2017a). Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators. Multivariate Behavioral Research, 52(6), 699–719.
Zurück zum Zitat Kelcey, B., Dong, N., Spybrook, J. & Cox, K. (2017b). Statistical power for causally-defined indirect effects in group-randomized trials. Journal of Educational and Behavioral Statistics. Kelcey, B., Dong, N., Spybrook, J. & Cox, K. (2017b). Statistical power for causally-defined indirect effects in group-randomized trials. Journal of Educational and Behavioral Statistics.
Zurück zum Zitat Kelcey, B. & Shen, Z. (2018). Effective and Efficient Experimental Design 2-1-1 Mediation Studies. Journal of Experimental Education (in press). Kelcey, B. & Shen, Z. (2018). Effective and Efficient Experimental Design 2-1-1 Mediation Studies. Journal of Experimental Education (in press).
Zurück zum Zitat Krull, J. L., & MacKinnon, D. P. (2001). Multilevel modeling of individual and cluster level mediated effects. Multivariate Behavioral Research, 36, 249–277.CrossRefPubMed Krull, J. L., & MacKinnon, D. P. (2001). Multilevel modeling of individual and cluster level mediated effects. Multivariate Behavioral Research, 36, 249–277.CrossRefPubMed
Zurück zum Zitat Phelps, G., Kelcey, B., Liu, S., & Jones, N. (2016). Informing Estimates of Program Effects for Studies of Mathematics Professional Development Using Teacher Content Knowledge Outcomes. Evaluation Review, 40, 383–409.CrossRef Phelps, G., Kelcey, B., Liu, S., & Jones, N. (2016). Informing Estimates of Program Effects for Studies of Mathematics Professional Development Using Teacher Content Knowledge Outcomes. Evaluation Review, 40, 383–409.CrossRef
Zurück zum Zitat Pituch, K. A., & Stapleton, L. M. (2012). Distinguishing between cross- and cluster-level mediation processes in the cluster randomized trial. Sociological Methods & Research, 41, 630–670.CrossRef Pituch, K. A., & Stapleton, L. M. (2012). Distinguishing between cross- and cluster-level mediation processes in the cluster randomized trial. Sociological Methods & Research, 41, 630–670.CrossRef
Zurück zum Zitat Preacher, K. J., & Selig, J. P. (2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6(2), 77–98.CrossRef Preacher, K. J., & Selig, J. P. (2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6(2), 77–98.CrossRef
Zurück zum Zitat Texas Christian University [TCU] (2005) Institute of Behavioral Research. The Organizational Readiness for Change: Treatment Staff Version (TCU ORC-S). Available at: www.ibr.tcu.edu. Texas Christian University [TCU] (2005) Institute of Behavioral Research. The Organizational Readiness for Change: Treatment Staff Version (TCU ORC-S). Available at: www.​ibr.​tcu.​edu.
Zurück zum Zitat Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation models. Sociological Methodology, 13, 290–312.CrossRef Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation models. Sociological Methodology, 13, 290–312.CrossRef
Zurück zum Zitat Schwartz, R. (2010). Motivational interviewing (patient-centered counseling) to address childhood obesity. Pediatric Annals, 39(3), 154–158.CrossRefPubMed Schwartz, R. (2010). Motivational interviewing (patient-centered counseling) to address childhood obesity. Pediatric Annals, 39(3), 154–158.CrossRefPubMed
Zurück zum Zitat Spybrook, J., Shi, R., & Kelcey, B. (2016). Progress in the Past Decade: An Examination of the Precision of Cluster Randomized Trials Funded by the U.S. Institute of Education Sciences. International Journal of Research & Method in Education, 39(3), 255–267.CrossRef Spybrook, J., Shi, R., & Kelcey, B. (2016). Progress in the Past Decade: An Examination of the Precision of Cluster Randomized Trials Funded by the U.S. Institute of Education Sciences. International Journal of Research & Method in Education, 39(3), 255–267.CrossRef
Zurück zum Zitat Stapleton, L., Pituch, K., & Dion, E. (2015). Standardized effect size measures for mediation analysis in clusterrandomized trials. Journal of Experimental Education, 83(4), 547–582.CrossRef Stapleton, L., Pituch, K., & Dion, E. (2015). Standardized effect size measures for mediation analysis in clusterrandomized trials. Journal of Experimental Education, 83(4), 547–582.CrossRef
Zurück zum Zitat VanderWeele, T. J. (2010). Direct and indirect effects for neighborhood-based clustered and longitudinal data. Sociological Methods & Research, 38, 515–544.CrossRef VanderWeele, T. J. (2010). Direct and indirect effects for neighborhood-based clustered and longitudinal data. Sociological Methods & Research, 38, 515–544.CrossRef
Zurück zum Zitat Williams, N., & Glisson, C. (2014). Testing a theory or organizational culture, climate and youth outcomes in child welfare systems: A United States national study. Child Abuse and Neglect, 38, 4.CrossRef Williams, N., & Glisson, C. (2014). Testing a theory or organizational culture, climate and youth outcomes in child welfare systems: A United States national study. Child Abuse and Neglect, 38, 4.CrossRef
Zurück zum Zitat Zhang, Z., Zyphur, M., & Preacher, K. (2009). Testing multilevel mediation using hierarchical linear models: Problems and solutions. Organizational Research Methods, 12, 695–719.CrossRef Zhang, Z., Zyphur, M., & Preacher, K. (2009). Testing multilevel mediation using hierarchical linear models: Problems and solutions. Organizational Research Methods, 12, 695–719.CrossRef
Metadaten
Titel
Sample Size Planning for Cluster-Randomized Interventions Probing Multilevel Mediation
verfasst von
Ben Kelcey
Jessaca Spybrook
Nianbo Dong
Publikationsdatum
21.07.2018
Verlag
Springer US
Erschienen in
Prevention Science / Ausgabe 3/2019
Print ISSN: 1389-4986
Elektronische ISSN: 1573-6695
DOI
https://doi.org/10.1007/s11121-018-0921-6

Weitere Artikel der Ausgabe 3/2019

Prevention Science 3/2019 Zur Ausgabe