nach oben

Prevention Science

Erschienen in:

01.10.2015

Maximizing the Yield of Small Samples in Prevention Research: A Review of General Strategies and Best Practices

verfasst von: Cameron R. Hopkin, Rick H. Hoyle, Nisha C. Gottfredson

Erschienen in: Prevention Science | Ausgabe 7/2015

Einloggen, um Zugang zu erhalten

Abstract

The goal of this manuscript is to describe strategies for maximizing the yield of data from small samples in prevention research. We begin by discussing what “small” means as a description of sample size in prevention research. We then present a series of practical strategies for getting the most out of data when sample size is small and constrained. Our focus is the prototypic between-group test for intervention effects; however, we touch on the circumstance in which intervention effects are qualified by one or more moderators. We conclude by highlighting the potential usefulness of graphical methods when sample size is too small for inferential statistical methods.

Our focus on statistical power assumes a traditional null hypothesis statistical testing (NHST) approach to data analysis. We recognize the shortcomings of this approach and its frequent misuse; however, because it remains the primary approach to the analysis of data from prevention trials, it is the approach on which our analysis and recommendations focus. For readers interested in concerns about NHST and potential alternatives, Nickerson (2000) and Harlow et al. (1997) provide balanced, largely nontechnical presentations.

See von Hippel (2013) for potential problems and solutions for use of these methods with small samples.

An informative discussion of the use of covariates to increase statistical power is provided by Dennis et al. (2009).

Information about approaches to data visualization can be found in Young (1996) and a collection of papers edited by Post et al. (2003).

Bacchetti, P., Deeks, S. G., & McCune, J. M. (2011). Breaking free of sample size dogma to perform innovative translational research. Science Translational Medicine, 3, 87. doi:10.1126/scitranslmed.3001628.CrossRef

Brown, C. H., Sloboda, Z., Faggiano, F., Teasdale, B., Keller, F., Burkhart, G., Vigna-Taglianti, F., Howe, G., Masyn, K., Wang, W., Muthén, B., Stephens, P., Grey, S., & Perrino, T. (2011). Methods for synthesizing findings on moderation effects across multiple randomized trials. Prevention Science, 14, 144–156. doi:10.1007/s11121-011-0207-8.CrossRef

Carrig, M., Wirth, R. J., & Curran, P. J. (2004). A SAS macro for estimating and visualizing individual growth curves. Structural Equation Modeling: An Interdisciplinary Journal, 11, 132–149. doi:10.1207/S15328007SEM1101_9.CrossRef

Cohen, J., Cohen, P., West, S., & Aiken, L. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah: Erlbaum.

Collins, L. M., Schafer, J. L., & Kam, C. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6, 330–351. doi:10.1037/1082-989X.6.4.330.CrossRefPubMed

Collins, L. M., Dziak, J. J., & Li, R. (2009). Design of experiments with multiple independent variables: A resource management perspective on complete and reduced factorial designs. Psychological Methods, 14, 202–224. doi:10.1037/a0015826.PubMedCentralCrossRefPubMed

Collins, L. M., Baker, T. D., Mermelstein, R. J., Piper, M. E., Jorenby, D. E., Smith, S. S., Christiansen, B. A., Schlam, T. R., Cook, J. W., & Fiore, M. C. (2011). The multiphase optimization strategy for engineering effective tobacco use interventions. Annals of Behavioral Medicine, 41, 208–226. doi:10.1007/s12160-010-9253-x.PubMedCentralCrossRefPubMed

Curran, P. J., & Hussong, A. M. (2009). Integrative data analysis: The simultaneous analysis of multiple data sets. Psychological Methods, 14, 81–100. doi:10.1037/a0015914.PubMedCentralCrossRefPubMed

Dennis, M., Francis, D. J., Cirino, P. T., Schachar, R., Barnes, M. A., & Fletcher, J. M. (2009). Why IQ is not a covariate in cognitive studies of neurodevelopmental disorders. Journal of the International Neuropsychological Society, 15, 331–343. doi:10.1017/S1355617709090481.PubMedCentralCrossRefPubMed

Dumas, J. E., Lynch, A. M., Laughlin, J. E., Phillips Smith, E., & Prinz, R. J. (2001). Promoting intervention fidelity. Conceptual issues, methods, and preliminary results from the EARLY ALLIANCE prevention trial. American Journal of Preventive Medicine, 20, 38–47. doi:10.1016/S0749-3797(00)00272-5.CrossRefPubMed

Embretson, S. E. (1996). Item response theory models and spurious interaction effect in factorial ANOVA designs. Applied Psychological Measurement, 20, 201–212. doi:10.1177/014662169602000302.CrossRef

Enders, C. K. (2010). Applied missing data analysis. New York: Guildford Press.

Fok, C. C. T., Henry, D., & Allen, J. A (2015). Maybe small is too small a term: Introduction to advancing small sample prevention science. Prevention Science, in press.

Friendly, M. (1995). Exploratory and graphical methods of data analysis [Online short course]. Retrieved from http://www.datavis.ca/courses/eda/. Accessed 6 Aug 2013.

Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576. doi:10.1146/annurev.psych.58.110405.085530.CrossRefPubMed

Hansen, W. B., & Collins, L. M. (1994). Seven ways to increase power without increasing N. NIDA Research Monograph. In L. M. Collins & L. A. Seitz (Eds.), Advances in data analysis for prevention intervention research (NIDA Research Monograph 142, NIH Publication No. 94–3599, pp. 184–195). Rockville, MD: National Institutes of Health.

Hansen, W. B., Tobler, N. S., & Graham, J. W. (1990). Attrition in substance abuse prevention research: A meta-analysis of 85 longitudinally followed cohorts. Evaluation Review, 14, 677–685. doi:10.1177/0193841X9001400608.CrossRef

Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.). (1997). What if there were no significance tests? Mahwah: Erlbaum.

Hawkins, N. G., Sanson-Fisher, R. W., Shakeshaft, A., D’Este, C., & Green, L. W. (2007). The multiple baseline design for evaluating population-based research. American Journal of Preventive Medicine, 33, 162–168. doi:10.1016/j.amepre.2007.03.020.CrossRefPubMed

Hoyle, R. H. (Ed.). (1999). Statistical strategies for small sample research‬. Thousand Oaks: Sage Publications.

Hoyle, R. H., & Gottfredson, N. C. (2015). Sample size considerations in prevention research applications of multilevel modeling and structural equation modeling. Prevention Science. doi:10.1007/s11121-014-0489-8.

Kang, S., & Waller, G. (2005). Moderated multiple regression, spurious interaction effects, and IRT. Applied Psychological Measurement, 29, 87–105. doi:10.1177/0146621604272737.CrossRef

Kratochwill, T. R., & Levin, J. R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15, 124–144. doi:10.1037/a0017736.CrossRefPubMed

Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241–301. doi:10.1037//1082-989X.S.2.241.CrossRefPubMed

Post, F. H., Nielson, G. M., & Bonneau, G.-P. (Eds.) (2003). Data visualization: The state of the art. Boston: Kluwer Academic Publishers. Retrieved from http://www.springer.com/computer/image+processing/book/978-1-4020-7259-8?otherVersion=978-1-4613-5430-7

Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2, 173–185. doi:10.1037/1082-989X.2.2.173.CrossRef

Schafer, J. L. (1997). Analysis of incomplete multivariate data. London: Chapman & Hall.CrossRef

Supplee, L. H., Kelly, B. C., MacKinnon, D. M., & Barofsky, M. Y. (2013). Subgroup analysis in prevention and intervention science [Special issue]. Prevention Science 14(2).

von Hippel, P. T. (2013). The bias and efficiency of incomplete-data estimators in small univariate normal samples. Sociological Methods & Research, 42, 531–558. doi:10.1177/0049124113494582.CrossRef

Wang, R., & Ware, J. H. (2013). Detecting moderator effects using subgroup analyses. Prevention Science, 14, 111–120. doi:10.1007/s11121-011-0221-x.PubMedCentralCrossRefPubMed

Young, F. W. (1996). ViSta: The visual statistics system. Chapel Hill: Thurstone Psychometric Laboratory Research Memorandum 94-1(c). Retrieved from http://forrest.psych.unc.edu/research/index.html

Young, F. W., & Bann, C. M. (1996). ViSta: A visual statistics system. In R. A. Stine & J. Fox (Eds.), Statistical computing environments for social research (pp. 207–236). Thousand Oaks: Sage Publications.

Zand, D., Thomson, N. R., Dugan, M., Braun, J. A., Holterman-Hommes, P., & Hunter, P. L. (2006). Predictors of retention in an alcohol, tobacco, and other drug prevention study. Evaluation Review, 30, 209–222. doi:10.1177/0193841X05281160.CrossRefPubMed

Titel: Maximizing the Yield of Small Samples in Prevention Research: A Review of General Strategies and Best Practices
verfasst von: Cameron R. Hopkin
Rick H. Hoyle
Nisha C. Gottfredson
Publikationsdatum: 01.10.2015
Verlag: Springer US
Erschienen in: Prevention Science / Ausgabe 7/2015
Print ISSN: 1389-4986
Elektronische ISSN: 1573-6695
DOI: https://doi.org/10.1007/s11121-014-0542-7

Springer Medizin

Abstract

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 7/2015

Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation

Designs for Testing Group-Based Interventions with Limited Numbers of Social Units: The Dynamic Wait-Listed and Regression Point Displacement Designs

Why Small is Too Small a Term: Prevention Science for Health Disparities, Culturally Distinct Groups, and Community-Level Intervention

Inflection and Reflection: Design Issues that Shape the Field of Prevention Science

Methods for Multilevel Ordinal Data in Prevention Research

Clustering Methods with Qualitative Data: a Mixed-Methods Approach for Prevention Research with Small Samples