nach oben

Prevention Science

Erschienen in:

22.04.2016

Longitudinal Assessment Design and Statistical Power for Detecting an Intervention Impact

verfasst von: Hanno Petras

Erschienen in: Prevention Science | Ausgabe 7/2016

Einloggen, um Zugang zu erhalten

Abstract

In evaluating randomized control trials (RCTs), statistical power analyses are necessary to choose a sample size which strikes the balance between an insufficient and an excessive design, with the latter leading to misspent resources. With the growing popularity of using longitudinal data to evaluate RCTs, statistical power calculations have become more complex. Specifically, with repeated measures, the number and frequency of measurements per person additionally influence statistical power by determining the precision with which intra-individual change can be measured as well as the reliability with which inter-individual differences in change can be assessed. The application of growth mixture models has shown that the impact of universal interventions is often concentrated among a small group of individuals at the highest level of risk. General sample size calculations were consequently not sufficient to determine whether statistical power is adequate to detect the desired effect. Currently, little guidance exists to recommend a sufficient assessment design to evaluating intervention impact. To this end, Monte Carlo simulations are conducted to assess the statistical power and precision when manipulating study duration and assessment frequency. Estimates were extracted from a published evaluation of the proximal of the Good Behavior Game (GBG) on the developmental course of aggressive behavior. Results indicated that the number of time points and the frequency of assessments influence statistical power and precision. Recommendations for the assessment design of longitudinal studies are discussed.

It is acknowledged that there are several aspects which may influence the statistical power of a covariate effect in a GMM, e.g., number of profiles identified and degree of separation between profiles. For this study, undue emphasis is placed on the first two moments of the covariate impact on the rate of change as well as the reliability of the covariate estimate with respect to study duration and assessment frequency.

Adolph, K. E., Robinson, S. R., Young, J. W., & Gil-Alvarez, F. (2008). What is the shape of developmental change? Psychological Review, 3, 527–543.CrossRef

Arbona, C. (2000). The development of academic achievement in school aged children: Precursors to career development. In S. D. Brown & R. W. Lent (Eds.), Handbook of Counseling Psychology (3rd ed., pp. 270–309). Hoboken: Wiley.

Bloom, P. (2004). Myths of word learning. In D. G. Hall & S. R. Waxman (Eds.), Weaving a Lexicon (pp. 205–224). Cambridge: MIT Press.

Bradshaw, C. P., Schaeffer, C. M., Petras, H., & Ialongo, N. (2010). Predicting negative life outcomes from early aggressive-disruptive behavior trajectories: Gender differences in maladaptation across life domains. Journal of Youth and Adolescence, 8, 953–966.CrossRef

Broidy, L. M., Nagin, D. S., Tremblay, R. E., Bates, J. E., Brame, B., Dodge, K. A., Fergusson, D., Horwood, J. L., Loeber, R., Laird, R., Lyman, D. R., Moffitt, T. E., Pettit, G. S., & Vitaro, F. (2003). Developmental trajectories of childhood disruptive behaviors and adolescent delinquency: A six-site, cross-national study. Developmental Psychology, 39, 222–245.CrossRefPubMedPubMedCentral

Brown, C. H., & Liao, J. (1999). Principles for designing randomized preventive trials in mental health: An emerging developmental epidemiologic paradigm. American Journal of Community Psychology, 27, 677–714.CrossRef

Brown, C. H., Wang, W., Kellam, S. G., Muthén, B. O., Petras, H., Toyinbo, P., Poduska, J., Ialongo, N., Wyman, P. A., Chamberlain, P., Sloboda, Z., MacKinnon, D. P., & Windham, A. (2008). Methods for testing theory and evaluating impact in randomized field trials: Intent-to-treat analyses for integrating the perspectives of person, place, and time. Drug and Alcohol Dependence, 95, S74–S104.CrossRefPubMedPubMedCentral

Catalano, R. F., Kosterman, R., Hawkins, J. D., Newcomb, M. D., & Abbott, R. D. (1996). Modeling the etiology of adolescent substance use: A test of the social development model. Journal of Drug Issues, 26, 429–455.CrossRefPubMedPubMedCentral

Cohen, J. (1969). Statistical power analysis for the behavioral sciences. Hillsdale: Lawrence Erlbaum Associates.

Collins, L. M. (2006). Analysis of longitudinal data: The integration of theoretical model, temporal design, and statistical model. Annual Review of Psychology, 57, 505–528.CrossRefPubMed

Collins, L. M., & Graham, J. W. (2002). The effect of the timing and spacing of observations in longitudinal studies of tobacco and other drug use: Temporal design considerations. Drug and Alcohol Dependence, 68, S85–S96.CrossRefPubMed

Cook, N. R., & Ware, J. H. (1983). Design and analysis for longitudinal research. Annual Review of Public Health, 4, 1–23.CrossRefPubMed

Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2004). Applied longitudinal data analysis. Hoboken, NJ: Wiley.

Griffin, K. W., Botvin, G. J., & Nichols, T. R. (2006). Effects of a school-based drug abuse prevention program for adolescents on HIV risk behaviors in young adulthood. Prevention Science, 7, 103–112.CrossRefPubMed

Hedeker, D., Gibbons, R. D., & Waternaux, C. (1999). Sample size estimation for longitudinal designs with attrition: Comparing time-related contrasts between two groups. Journal of Educational and Behavioral Statistics, 24, 70–93.CrossRef

Kellam, S. G., Brown, C. H., Poduska, J. M., Ialongo, N. S., Wang, W., Toyinbo, P., Petras, H., Ford, C., Windham, A., & Wilcox, H. C. (2008). Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug and Alcohol Dependence, 95, S5–S28.CrossRefPubMedPubMedCentral

Kellam, S. G., Koretz, D., & Mościcki, E. K. (1999). Core elements of developmental epidemiologically based prevention research. American Journal of Community Psychology, 27(4), 463–82.CrossRefPubMed

Kellam, S. G., Mackenzie, A. C., Brown, C. H., Poduska, J. M., Wang, W., Petras, H., & Wilcox, H. C. (2011). The good behavior game and the future of prevention and treatment. Addiction Science and Clinical Practice, 6(1), 73–84.PubMedPubMedCentral

Kellam, S., & Rebok, G. (1992). Building developmental and etiological theory through epidemiological based preventive intervention trials (pp. 162–195). In J. McCord & R. E. Tremblay (Eds.), Preventing antisocial behavior: Interventions from birth through adolescence. New York: Neale Watson Academic Publishers.

Lipsey, M. W., & Hurley, S. M. (2009). Design sensitivity: Statistical power for applied experimental research. In L. Bickman & D. J. Rog (Eds.), The SAGE Handbook of Applied Social Research Methods (pp. 44–76). California: SAGE Publications, Inc.CrossRef

Loeber, R., & Farrington, D. P. (2000). Young children who commit crime: Epidemiology, developmental origins, risk factors, early interventions, and policy implications. Development and Psychopathology, 12, 737–762.CrossRefPubMed

Mason, W. A., Kosterman, R., Hawkins, J. D., Haggerty, K. P., & Spoth, R. (2003). Reducing adolescents’ growth in substance use and delinquency: Randomized trial effects of a preventive parent training intervention. Prevention Science, 4, 203–2013.

Macleod, J., Oakes, R., Copello, A., Crome, I., Egger, M., Hickman, M., Oppenkowski, T., Stokes-Lampard, H., & Davey, S. G. (2004). Psychological and social sequelae of cannabis and other illicit drug use by young people: A systematic review of longitudinal, general population studies. Lancet, 363(9421), 1579–88.CrossRefPubMed

Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annual Review of Psychology, 59, 537–63.CrossRefPubMed

McGrath, J. E., & Tschan, F. (2004). Temporal matters in social psychology: Examining the role of time in the lives of groups and individuals. Washington, DC: APA Publications.CrossRef

Moerbeek, M. (2008). Powerful and cost-efficient designs for longitudinal intervention studies with two treatment groups. Journal of Educational and Behavioral Statistics, 33, 41–61.CrossRef

Moffitt, T. E., & Caspi, A. (2001). Childhood predictors differentiate life-course-persistent and adolescence-limited antisocial pathways among males and females. Development and Psychopathology, 13(2), 355–375.CrossRefPubMed

Mooney, C. Z. (1997). Monte Carlo simulation. Thousand Oaks: Sage Publications.CrossRef

Muthén, B. O., & Curran, P. J. (1997). General longitudinal modeling of individual differences in experimental design: A latent variable framework for analysis and power estimation. Psychological Methods, 4(2), 371–402.CrossRef

Muthén, B., & Shedden, K. (1999). Finite mixture modeling with mixture outcomes using the EM algorithm. Biometrics, 6, 463–469.CrossRef

Muthén, B. (2002). Using Mplus Monte Carlo simulations in practice: A note on assessing estimation quality and power in latent variable models. Version 2, March 22, 2002. Retrieved from www.statmodel.com on 05/15/2015.

Muthén, B. (2004). Latent variable analysis: Growth mixture modeling and related techniques for longitudinal data. In D. Kaplan (Ed.), Handbook of quantitative methodology for the social sciences (pp. 345–368). Newbury Park: Sage Publications.

Muthén, B. and Asparouhov, T. (2002). Using Mplus Monte Carlo simulations in practice: A note on non-normal missing data in latent variable models. Version 2, March 22, 2002. Retrieved from www.statmodel.com on 05/15/2015.

Muthén, B., Brown, C. H., Masyn, K., Jo, B., Khoo, S. T., Yang, C. C., Wang, C. P., Kellam, S., Carlin, J., & Liao, J. (2002). General growth mixture modeling for randomized preventive interventions. Biostatistics, 3, 459–475.CrossRefPubMed

Muthén, L.K. & Muthén, B. (1998-2014). Mplus User’s Guide. Seventh Edition. Los Angeles, CA: Muthén & Muthén.

Muthén, B., Brown, C. H., Hunter, A., Cook, I. A., & Leuchter, A. F. (2011). General approaches to analysis of course: Applying growth mixture modeling to randomized trials of depression medication. In P. E. Shrout (Ed.), Causality and Psychopathology: Finding the Determinants of Disorders and their Cures (pp. 159–178). New York: Oxford University Press.

Nagin, D. S. (1999). Analyzing developmental trajectories: A semiparametric, group-based approach. Psychological Methods, 4, 139–157.CrossRef

Nagin, D., & Tremblay, R. E. (2001). Parental and early childhood predictors of persistent physical aggression in boys from kindergarten to high school. Archives of General Psychiatry, 58, 389–394.CrossRefPubMed

Patterson, G. R., DeBaryshe, B. D., & Ramsey, E. (1989). A developmental perspective on antisocial behavior. American Psychologist, 44, 329–335.CrossRefPubMed

Paxton, P., Curran, P. J., Bollen, K. A., Kirby, J., & Chen, F. (2001). Monte Carlo experiments: Design and implementation. Structural Equation Modeling, 8, 287–312.CrossRef

Petras, H., Kellam, S. G., Brown, H., Muthén, B., Ialongo, N., & Poduska, J. M. (2008). Developmental epidemiological courses leading to antisocial personality disorder and violent and criminal behavior: Effects by young adulthood of a universal preventive intervention in first- and second-grade classrooms. Drug and Alcohol Dependence, 95, S45–S59.CrossRefPubMedPubMedCentral

Petras, H., Masyn, K., & Ialongo, N. (2011). The distal impact of two first grade preventive interventions on aggressive/disruptive behavior in adolescence: An application of latent transition growth mixture modeling. Prevention Science, 12, 300–313.CrossRefPubMedPubMedCentral

Piquero, A. R. (2008). Taking stock of developmental trajectories of criminal activity over the life course. In A. M. Lieberman (Ed.), The long view of crime: A synthesis of longitudinal research (pp. 23–78). New York: Springer.CrossRef

Raudenbush, S. W. (1993). Hierarchical linear models and experimental design. In L. K. Edwards (Ed.), Applied analysis of variance in behavioral science. New York: Marcel Dekker.

Raudenbush, S. W., & Xiao-Feng, L. (2001). Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychological Methods, 6(4), 387–401.CrossRefPubMed

Schlesselman, J. J. (1973). Planning a longitudinal study: II. Frequency of measurement and study duration. Journal of Chronic Disease, 26, 561–570.CrossRef

Schmiedek, F., Lövdén, M., & Lindenberger, U. (2009). On the relation of mean reaction time and intraindividual reaction time variability. Psychology and Aging, 24, 841–857.CrossRefPubMed

Singer, J. D., & Willet, J. B. (1996). Methodological issues in the design of longitudinal research: Principles and recommendations for a quantitative study of teacher’s careers. Educational Evaluation and Policy Analysis, 4, 265–283.CrossRef

van Dulmen, M. H. M., Goncy, E. A., Vest, A., & Flannery, D. J. (2009). Group-based trajectory modeling of externalizing behavior problems from childhood through adulthood: Exploring discrepancies in the empirical findings. In J. Savage (Ed.), The development of persistent criminality (pp. 288–314). New York: Oxford University.CrossRef

Werthamer-Larsson, L., Kellam, S. G., & Wheeler, L. (1991). Effect of first-grade classroom environment on child shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology, 19, 585–602.CrossRefPubMed

Willett, J. B. (1989). Some results on reliability for the longitudinal measurement of change: Implications for the decision of studies of individual growth. Educational and Psychological Measurement, 49, 587–601.CrossRef

Wilson, J. W., & Lipsey, M. W. (2007). School-based interventions for aggressive and disruptive behavior: Update of a meta-analysis. American Journal of Preventive Medicine, 33(2 Suppl.), S130–S143.CrossRefPubMedPubMedCentral

Titel: Longitudinal Assessment Design and Statistical Power for Detecting an Intervention Impact
verfasst von: Hanno Petras
Publikationsdatum: 22.04.2016
Verlag: Springer US
Erschienen in: Prevention Science / Ausgabe 7/2016
Print ISSN: 1389-4986
Elektronische ISSN: 1573-6695
DOI: https://doi.org/10.1007/s11121-016-0646-3

Springer Medizin

Abstract

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 7/2016

Cultural Identity Among Urban American Indian/Alaska Native Youth: Implications for Alcohol and Drug Use

Conceptualizing and Measuring Weekend versus Weekday Alcohol Use: Item Response Theory and Confirmatory Factor Analysis

Planning for Long-Term Follow-Up: Strategies Learned from Longitudinal Studies

Assessing Community Coalition Capacity and its Association with Underage Drinking Prevention Effectiveness in the Context of the SPF SIG

Methodological and Design Considerations in Evaluating the Impact of Prevention Programs on Violence and Related Health Outcomes

Being Blunt About Marijuana: Parent Communication About Marijuana with Their Emerging Adult Children