nach oben

Prevention Science

Erschienen in:

01.06.2010

An Experimental Evaluation of the All Stars Prevention Curriculum in a Community After School Setting

verfasst von: Denise C. Gottfredson, Amanda Cross, Denise Wilson, Melissa Rorie, Nadine Connell

Erschienen in: Prevention Science | Ausgabe 2/2010

Einloggen, um Zugang zu erhalten

Abstract

This study tested the effectiveness of a prevention curriculum, All Stars, as implemented in a year-long school-based after school program and provides an independent replication of the effects of All Stars on targeted mediators and problem behaviors using an experimental methodology. Middle school students (N = 447) who registered for the after school program were randomly assigned to the experimental or control condition. The sample included approximately equal proportions of males and females, was 70% African American, and 59% of the students received subsidized meals at school. All Stars was delivered with reasonable integrity to the program design, although with lower quality than reported in earlier efficacy trials. However, actual student exposure to the program was lower than expected due to low levels of attendance in the after school program. Students who ever attended received an average of 16 h of All Stars instruction. Results showed no differences between the treatment and control students at post-test on any of the outcomes or mediators. Further, no positive effects were found for youths receiving higher dosage, higher quality program delivery, or both. Insufficient time to achieve high quality implementation in the after school context and potential deviancy training are suggested as reasons for the failure to replicate positive program effects.

Some research has examined All Stars effects on mediators not specifically targeted by All Stars and on sexual activity outcomes. We do not discuss these findings as they are beyond the scope of this research.

We conducted power analyses for two-tailed independent t-tests, fixing the Type I error rate (alpha) at 0.05. These power analyses estimated the power available to detect a moderate-sized effect (0.30) and indicated that, using the entire sample, the power to detect differences between the treatment and control groups is 0.99. We calculated power for a number of different outcomes, including aggression (measured continuously) and vulnerability to drug use and last-month frequency drug use (measured as binary outcomes). These power analyses also estimated the minimum effect size detectable with a sample of our size and power fixed at 0.80. Analyses including both a dummy variable for school and the pre-test measure as a covariate will allow us to detect effects of about 0.17.

These students were excluded from outcome analysis because of missing posttest data. They either refused to take the posttest (N = 10), had transferred out of Maryland schools (N = 10), or left more than 40% of the survey items blank (N = 11).

Participants who reported some use at pre-test or failed to report at one or both time periods for cigarettes (37, 9%), alcohol (75, 18%), marijuana (30, 7%), inhalants (40, 10%) and other illegal drugs (11, 3%) were excluded from analyses of initiation for each substance.

Each All Stars lesson is designed to be delivered start-to-finish during one session. However, the ASP sites delivered one lesson over 2 days in two 45-min sessions because the program structure provided extra time.

Subjects were clustered within site. Clustering often violates the independence assumption in OLS regression and may introduce negative bias in standard errors, therefore inflating alpha levels in statistical tests. This negative bias increases in severity as the degree of correlation among clustered observations increases. When the proportion of variance in outcomes that is between groups (the intra-class correlation, or ICC) approaches zero, the consequences of clustering are nil. In this study, very little of the variance in the time 1 measures was between schools. The ICCs ranged from 0.000 to 0.025 and for the most part did not reach nominal levels of statistical significance. The largest proportions of variance between schools were found for disruptive classroom behavior and commitment to abstain from drugs. This suggests that clustering is unlikely to have inflated significance tests statistics that assumed a simple random sample. Nevertheless, we follow the advice of Snijders and Bosker (1999) for handling clustered data when the number of clusters is small (<10), and include dummy variables for schools in all outcome analyses to correct for intercept differences across schools.

The grouping for All Stars was haphazard, depending on the number of students present on each day. The All Stars groups were not systematically recorded.

Hansen (1996) used a three-item response set for each question compared to our four- and five- item response sets. The response sets in Harrington et al. (2001) were not identified.

We considered testing for interactions by the use of certified teachers, following McNeal et al. (2004) who showed that positive effects were found only for the teacher condition. However, in our study, such analyses were redundant with the analyses of interactions by quality because only in sites A, B, and C were lessons led by certified teachers.

The 398 discrete activities included 101 All Stars sessions. However, the number of All Stars sessions in which responses to misbehavior were observed (N = 74) is too small to allow for fine-grained analysis of differences across the five sites.

Several combinations of high v. low implementation schools resulted in significant interactions. The strongest effect appeared in analyses comparing the highest implementation site (Site B) to all others, in a pattern that replicated those reported in Hallfors et al. (2007). To maintain consistency with the analyses by program quality reported earlier, we report here the results of the All Stars by quality interaction test in which quality was measured using a dummy variable contrasting sites A, B, and C vs. D and E.

Response is coded 1 = reinforcing, 2 = neutral, 3 = chastising.

Allison, P.D. (2002). Missing data. Thousand Oaks, CA: Sage.

Catalano, R.F., Berglund, M.L., Ryan, J.A.M., Lonczak, H., & Hawkins, J.D. (2004). Positive youth development in the United States: Research findings on evaluations of positive youth development programs. Annals of the American Academy of Political and Social Sciences, 591, 98–124.CrossRef

Cho, H., Hallfors, D., & Sanchez, V. (2005). Evaluation of a high school peer group intervention for at-risk youth. Journal of Abnormal Child Psychology, 33, 363–374.CrossRefPubMed

Dishion, T.J., Spracklen, K.M., & Andrews, D.W. (1996). Deviancy training in male adolescent friendships. Behavior Therapy, 27, 373–390.CrossRef

Dishion, T.J., McCord, J., & Poulin, F. (1999). When interventions harm: Peer groups and problem behavior. American Psychologist, 54, 755–764.CrossRefPubMed

Durlak, J.A., & Weissberg, R.P. (2007). The impact of after-school programs that promote personal and social skills. Chicago: Collaborative for Academic, Social, and Emotional Learning.

Eysenck, S.B., Easting, G., & Pearson, P.R. (1984). Age norms for impulsiveness, venturesomeness and empathy in children. Personality and Individual Differences, 5, 315–321.CrossRef

Fagan, A.A., Hanson, K., Hawkins, J.D., & Arthur, M.W. (2008). Bridging science to practice: Achieving prevention program implementation fidelity in the Community Youth Development study. American Journal of Community Psychology, 41, 235–249.CrossRefPubMed

Flay, B.R., Biglan, A., Boruch, R.F., Castro, F.G., Gottfredson, D.C., Kellam, S., et al. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151–176.CrossRefPubMed

Gottfredson, G.D., & Gottfredson, D.C. (1992). What about you?. Ellicott City, MD: Gottfredson Associates.

Gottfredson, G.D., & Gottfredson, D.C. (2001). What schools do to prevent delinquency and promote safe environments. Journal of Educational and Psychological Consultation, 12, 313–344.CrossRef

Gottfredson, D.C., & Wilson, D.B. (2003). Characteristics of effective school-based substance abuse prevention. Prevention Science, 4, 27–38.CrossRefPubMed

Gottfredson, G.D., Gottfredson, D.C., Czeh, E.R., Cantor, D., Crosse, S.B., & Hantman, I. (2000). A national study of delinquency prevention in school final report. Ellicott City, MD: Gottfredson Associates (ERIC No. ED 459 409) (available on line at http://www.gottfredson.com).

Gottfredson, G.D., Jones, E.M., & Gore, T.W. (2002a). Implementation and evaluation of a cognitive–behavioral intervention to prevent problem behavior in a disorganized school. Prevention Science, 3, 43–56.CrossRefPubMed

Gottfredson, D.C., Wilson, D.B., & Najaka, S.S. (2002b). School-based crime prevention. In L.W. Sherman, D.P. Farrington, B.C. Welsh, & D.L. MacKenzie (Eds.), Evidence-based crime prevention (pp. 56–164). London: Routledge.

Gottfredson, D.C., Weisman, S.A., Soulé, D.A., Womer, S.C., & Lu, S. (2004). Do after school programs reduce delinquency? Prevention Science, 5, 253–266.CrossRefPubMed

Gottfredson, D.C., Cross, A.B., & Soulé, D.A. (2007). Distinguishing characteristics of effective and ineffective after-school programs to prevent delinquency and victimization. Criminology & Public Policy, 6, 289–318.CrossRef

Gottfredson, D.C., Cross, A.B., Wilson, D.M., Connell, N., & Rorie, M. (2009). A randomized trial of the effects of an enhanced after school program. College Park: MD. Unpublished report available from the authors.

Hallfors, D., Cho, H., Dishion, T., Cuffee, J., & Hartman, S. (2007). The role of teachers in school-based interventions for high risk adolescents: An alternative paradigm for predicting deviancy training. Unpublished manuscript.

Hamilton, L.S., Stecher, B.M., Marsh, J.A., Sloan McCombs, J., Robyn, A., Russell, J., et al. (2007). Standards-based accountability under No Child Left Behind: Experiences of teachers and administrators in three states. Santa Monica, CA: RAND Corporation.

Hansen, W.B. (1996). Pilot test results comparing the All Stars program with seventh grade D.A.R.E.: Program integrity and mediating variable analysis. Substance Use & Misuse, 31, 1359–1377.CrossRef

Hansen, W.B., & Dusenbury, L. (2004). All Stars Plus: A competence and motivation enhancement approach to prevention. Health Education, 104, 371–381.CrossRef

Harrington, N.G., Giles, S.M., Hoyle, R.H., Feeney, G.J., & Yungbluth, S.C. (2001). Evaluation of the All Stars character education and problem behavior prevention program: Effects on mediator and outcome variables for middle school students. Health Education & Behavior, 28, 533–546.CrossRef

Huizinga, D., & Esbensen, F. (1990). Scales and measures of the Denver Youth Survey. Boulder, CO: Institute of Behavioral Science.

Kane, T.J. (2004). The impact of after school programs: Interpreting the results of four recent evaluations. The William T. Grant Foundation Working Paper.

Lauer, P.A., Akiba, M., Wilkerson, S.B., Apthorp, H.S., Snow, D., & Martin-Glenn, M.L. (2006). Out-of-school time programs: A meta-analysis of effects for at-risk students. Review of Educational Research, 76, 275–313.CrossRef

Lipsey, M.W., & Wilson, D.B. (1998). Effective intervention for serious juvenile offenders: A synthesis of research. In R. Loeber & D.P. Farrington (Eds.), Serious and violent juvenile offenders: Risk factors and successful intervention (pp. 313–345). Thousand Oaks, CA: Sage.

McNeal, R.B., Hansen, W.B., Harrington, N.G., & Giles, S.M. (2004). How All Stars works: An examination of program effects on mediating variables. Health Education & Behavior, 31, 165–178.CrossRef

Patterson, G.R., Dishion, T.J., & Yoerger, K. (2000). Adolescent growth in new forms of problem behavior: Macro- and micro-peer dynamics. Prevention Science, 1, 3–13.CrossRefPubMed

Rorie, M., Gottfredson, D.C., Cross, A., Wilson, D., & Connell, N. (2009). Structure and deviancy training in after-school programs. Manuscript submitted for publication.

Scott-Little, C., Hamann, M.S., & Jurs, S.G. (2002). Evaluations of after-school programs: A meta-evaluation of methodologies and narrative synthesis of findings. American Journal of Evaluation, 23, 387–419.

Slater, M.D., Kelly, K.J., Edwards, R.W., Thurman, P.J., Plested, B.A., Keefe, T.J., et al. (2006). Combining in-school and community-based media efforts: Reducing marijuana and alcohol uptake among younger adolescents. Health Education Research, 21, 157–167.CrossRefPubMed

Snijders, T., & Bosker, R. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage.

Tanglewood Research. (2008a). All Stars quality assessments. Retrieved July, 2006, from www.allstarsprevention.com/programs/programsEvaluationQA.asp.

Tanglewood Research. (2008b). All Stars student surveys. Retrieved July, 2006, from www.allstarsprevention.com/programs/programsEvaluationOA.asp.

Wilson, D.B., Gottfredson, D.C., & Najaka, S.S. (2001). School-based prevention of problem behaviors: A meta-analysis. Journal of Quantitative Criminology, 17, 247–272.CrossRef

Zief, S.G., & Lauver, S. (2006). The impacts of after-school programs on student outcomes: A systematic review for the Campbell Collaboration. Retrieved April, 2008, from www.campbellcollaboration.org/doc-pdf/zief_afterschool_review.pdf.

Titel: An Experimental Evaluation of the All Stars Prevention Curriculum in a Community After School Setting
verfasst von: Denise C. Gottfredson
Amanda Cross
Denise Wilson
Melissa Rorie
Nadine Connell
Publikationsdatum: 01.06.2010
Verlag: Springer US
Erschienen in: Prevention Science / Ausgabe 2/2010
Print ISSN: 1389-4986
Elektronische ISSN: 1573-6695
DOI: https://doi.org/10.1007/s11121-009-0156-7

Springer Medizin

Abstract

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 2/2010

An Extended Model of Reasoned Action to Understand the Influence of Individual- and Network-Level Factors on African Americans’ Participation in HIV Vaccine Research

Concurrent Alcohol Use or Heavier Use of Alcohol and Cigarette Smoking Among Women of Childbearing Age with Accessible Health Care

The Effects of Project ALERT One Year Past Curriculum Completion

When Early Crime Prevention Goes to Scale: A New Look at the Evidence

LateNight Penn State Alcohol-Free Programming: Students Drink Less on Days They Participate

Effects of the Communities That Care Model in Pennsylvania on Change in Adolescent Risk and Problem Behaviors