The Relative Importance of Provider, Program, School, and Community Predictors of the Implementation Quality of School-Based Prevention Programs

The original researchers used materials from government agencies, technical assistance providers, professional organizations, program marketers, and scientific literature to develop a list of 20 categories of activities and strategies to prevent and/or reduce delinquency (Gottfredson and Gottfredson 2002). These categories were intended to describe each important aspect of any prevention program or activity. Principals were then asked to name up to five different specific programs or activities for each category; thus, how each program was classified was a principal decision. One program/activity could be listed under multiple categories; in the original study, such multi-component programs made up 17% of the activities identified by principals (Gottfredson et al. 2000). The original researchers then randomly sampled 1 program or activity in each of 14 categories per school and sent the Activity Coordinator survey, which asks for more detailed information about the program, to the individual identified as responsible for the activity. If the same individual was named by the principal multiple times and this person was identified multiple times in the random selection process (as could be the case with multi-component programs), the principal was asked to identify a different person as responsible for one of the activities. If this was not possible, the original researchers re-sampled the activities (Gottfredson et al. 2000). In the current sample, multi-component programs were found in 63 schools (12%). To ensure that the inclusion of these programs did not alter the results, the models were rerun without these schools in the sample. Results were very similar to the results reported here, displaying significant relationships in the same direction and of the same strength.

Generally, all teachers in participating schools were sampled, and a sufficient number of students were sampled to produce an estimated 50 respondents per school. When a student roster containing student gender was available, students were systematically sampled within gender. Otherwise, students were stratified by grade level and sampled.

The final sample of 544 schools is 42.27% of the original 1,287 schools and 64.15% of the 848 schools that responded in Phase One. Student enrollment in these 544 schools ranged from 97 to 2,912, with a mean of 790.31 and a standard deviation of 478.40.

Measurement scales are based on scales developed and copyrighted by Gary Gottfredson (see Gottfredson et al. 2000).

Regression imputation is a standard process for dealing with missing data (Raghunathan et al. 2001). Given the significant correlations among the various indicators of implementation quality (ranging from 0.110 to 0.518, p < 0.01), using three or four indicators to predict a program’s score on one or two indicators fall within the acceptable guidelines of this practice.

As can be seen in Table 2, Conscientiousness appears to have limited variability and a positive skew; this can potentially affect this study’s findings. Self-reported questionnaires designed to collect responses aimed at personal performance can threaten the reliability and validity of outcome measures due to social desirability bias. While these threats have been found to be limited in research, there nevertheless exists the possibility that social desirability can affect outcome measures by producing spurious results, suppressing real results, or moderating the relationships between the independent and dependent variables (Ganster et al. 1983).

Results of exploratory factor analyses from which these factors were created can be seen in Payne et al. (2006).

The following Census variables are markers for the Community Poverty factor: welfare (average household public assistance income), female-headed households (ratio of single females with children under 18 to married couples with children under 18), median income (proportion of households with income below $27,499), poverty (ratio of persons below the 1.24 poverty level to persons above), divorce rate (ratio of persons over 15 years who are married to those who are separated, divorced, or have a spouse absent), and unemployment (proportion of unemployed males/females in the labor force) (Simonsen 1998).

The following variables are markers for the Urbanicity factor: population size (total population), urban level (city level type), and urbanicity (the proportion of people living within an urban area) (Simonsen 1998).

Regression imputation is a standard process for dealing with missing data (Raghunathan et al. 2001). Ten different census variables were used for imputation. For each imputed variable, those census variables with the largest correlations with the variable to be imputed were used. Between 1 and 128 schools required imputation for exogenous variables taken from sources other than the teacher surveys. The two exogenous variables that are taken from the teacher survey (Percentage Teachers African–American and Number of Different Students Taught) were missing data for 221 and 220 schools, respectively. Because imputation was required for a large number of schools, results were examined for possible changes when these two variables were not included in the model; no significant changes were seen.

The generic level one model is Y_ij = β_0j + β_1jX_1ij + … β_kjX_kij + r_ij where Y_ij is the outcome for ith individual in jth school, β_0j is the intercept or the mean level of the outcome for each j school, β_kj is the regression coefficient for the effect on the outcome variable of X_ij, which is a level one predictor of the outcome, and r_ij is the level one error term. This is also known as the within-school equation. The generic level two model is β_0j = γ₀₀ + γ₀₁W_1j + γ₀₂W_2j + … γ_0qW_qj + u_0j, β_1j = γ₁₀ + γ₁₁W_1j + γ₁₂W_2j + … γ_1qW_qj + u_1j, β_kj = γ_k0 + γ_k1W_1j + γ_k2W_2j + … γ_kqW_qj + u_kj where β_0j is the school level mean of the outcome from level one equation, γ₀₀ is the level two intercept or the mean level of the level one outcome for the entire sample, γ_0q is the regression coefficient for the effect on the school level mean of W_qj, which is a level two predictor, and u_0j is the level two error term. As in the level one model, β_kj is the coefficient for a level one predictor of Y_ij, the individual-level outcome. However, in level two, β_kj becomes an outcome itself of a level two predictor W_qj. This is also known as the between-school equation.

Centering adjusts the outcome (either level one or level two) for the particular control variable and the outcome is the predicted value for the program or school with the mean value of that interval variable. Binary variables were left uncentered. Therefore, in the level one equation, the outcome is the predicted value for a program whose value is zero on the binary variable, while the outcome in the level two equation is the predicted value for a school whose value is zero on the binary variable.

It is unlikely, however, that the basic results of our study would change had more of these schools been included. Exploratory analyses of potential biases introduced by the limited response rates examined participating schools that were located in similar communities as the majority of non-participating schools and found results similar to those seen here. Therefore, it seems likely that the inclusion of the non-participating schools would have resulted in actually intensifying the relationships reported in this study. Of course, it is possible that the relationships of interest are not linear in the region of the distribution in which non-participating schools fall, or that some characteristic might alter the relationships established. However, the linear relationships among this study’s measures seem to indicate that, if anything, the results presented here provide conservative estimates of the relationships.