Pupil sample
During 1996 and 1997 two successive cohorts of 13 – 14 year olds participated in a baseline survey (mean age 14 years and two months). These 7,616 pupils who participated (of the 8,430 eligible) were representative of 14 year olds throughout Scotland, in terms of parents' social class and the proportion of one-parent households, using 1991 Census data[
28] This paper also uses data collected in 1998 and 1999 at the first follow-up of the SHARE trial, when the cohorts were aged 15 or 16 (mean age 16 years and one month). By this age 23% had left school. Follow-up data were collected from 5,854 young people giving an overall participation rate of 70%. There was a very different participation rate for those still at school (81%) and early school leavers (39%). A small proportion (2%) refused to participate (Wight et al., 2002). Early school leavers are those that leave school as soon as they are legally old enough to do so (16 years of age). Leaving that early will very rarely have enabled the young person to have sat the level of examinations that would allow access to Higher Education such as University/College.
One school chose not to participate at baseline and is therefore excluded from our analysis (380 pupils). Further exclusions were applied to cases with high levels of missing data (362 pupils), with incomplete data on important covariates (118 pupils), or where we were not certain that sexual intercourse had taken place (68 pupils).
In this paper we have used data from 4,926 pupils for whom we have both baseline and follow-up data (65% of those that originally participated at baseline). These pupils are included in the all statistical models described.
Pupil level measures
The pupil questionnaires at baseline and age 16 follow-up broadly asked the same questions, although the age 16 questionnaire was slightly longer. The questionnaires (and additional information about SHARE) are available on a public domain Internet site [
31].
The questionnaire covered the following topic areas: questions about pupils' sexual experience; socio-cultural variables (e.g. family composition and parental monitoring); attitudinal variables (e.g. attitude to school and self-esteem), aspirational variables (e.g. what the young people think they will be doing in 2 years time); and an indicator of proportion of friends at other schools and proportion of friends who have left school.
For sexual experience, pupils were told that, 'in questions that follow 'sexual intercourse' means: a boy/man putting his penis into a girl/woman's vagina, or 'going the whole way'. The pupils were then asked 'Have you experienced any of the following with a girl/woman [or boy/man]? Then the pupils could tick yes or no to sexual intercourse. As part of the data cleaning process we examined the logical consistency of the answer with reference to ten further questions about first sexual intercourse. All pupils included in this analysis had logically consistent answers (68 were excluded on this basis, see also above).
Process measures
Process data reflect the characteristics of the school, including information on relationships between different dyads within the school (e.g. teacher-pupil, pupil-pupil and teacher-teacher), but also appearance, discipline and layout of the school.
Data on processes within schools were collected through pupil questionnaires, interviews and group discussions, teacher questionnaires and interviews, ethnographic notes, classroom observation and fieldworkers' observations [
32]. Four kinds of data are used in this analysis. First, there are individual-level data from pupil questionnaires about the degree to which the respondent likes school (2 items), and teachers trust and respect pupils (2 items). Second, there are school-level data from questionnaires with sex education teachers (N = 151) about senior management to staff relationships (1 item), staff to staff relationships (2 items) and staff to pupil relationships (2 items). The Cronbach's Alpha for all these items derived independently from pupils and teachers was over 0.7. Focusing solely on staff-pupil relationships, as reported independently by teachers and pupils, yielded a Cronbach's Alpha over 0.9. These Cronbach's Alpha scores indicate that, despite being reported independently by teachers and pupils and through different questionnaire items, there is good internal consistency across the items, demonstrating the validity of this information. These items were included in the factor analysis described below.
The third kind of process information is qualitative, arising from 58 in-depth teacher interviews (conducted with at minimum, the teacher responsible for sex education in each school), observations of lessons with 41 teachers (in 15 schools) and numerous ethnographic notes (from all schools). The interviews covered, amongst other topics, relationships between sex education colleagues, support from senior management, staff-pupil relationships and perceived ethos of the school. Amongst other things systematically noted in the lesson observations were: teacher-pupil relationships, pupil-pupil relationships and pupil behaviour. Two qualitative researchers (including DW) coded these data and then reviewed all the relevant information pertaining to nine aspects of the school, giving a General Score (GS) to each on a scale of 1 (poor), 2 (OK) to 3 (good). The aspects were: pupil morale, relationships between staff and pupils, staff and staff, and staff and senior management, academic focus, organisation of PSE, discipline, school-home relationships and physical environment (Table
1). A second researcher validated this scoring and any discrepancies were discussed with a third researcher (MH) who had also frequently visited the schools until a consensus score was agreed. Inter-rater reliability scores were not calculated, however discrepancies in scoring were usually of only one point, and only in one school were there discrepancies of two points. In establishing consensus scores the researchers drew on their wider knowledge of the teacher, pupils or school in order to better contextualise the recorded observations. It should be noted that the quantity and type of information held about each school varied considerably, and for all but one of the variables there were missing data for some schools.
Table 1
Summary of process data for schools
9 | 1 | 24 | 4.4 | 4 | 4.4 | 4 | 4.5 | 2 | | | (U) 2/3 | Reg. | 1/2 | 3 | 3 | | 1/2 |
15 | 2 | 3 | 3.7 | 3.3 | 3.9 | 3.9 | 4 | 3.2 | 3 | | NU 1/2 | Reg. | 3 | | | 3 | 2 |
6 | 3 | 14 | 3.9 | | 4.2 | | 4.4 | | 1 | | U 3 | Reg. | 1 | | | 1 | |
17 | 4 | 6 | 3.8 | 4.3 | 3.8 | 2.6 | 4.1 | 3.7 | | | NU 3 | Reg. | 2 | 1 | 1 | 3 | 3 |
24 | 5 | 15 | 4.1 | 2.2 | 3.7 | 3.1 | 3.4 | 3 | 2 | 1 | U 3 | Reg./French | | 1 | 1 | | 1 |
10 | 6 | 19 | 3.9 | 4.2 | 4.3 | 5 | 4.2 | 5 | 3 | | U* 1 | Reg. | | 3 | | | 3 |
3 | 7 | 7 | 3.1 | 2 | 3.8 | 3.6 | 4.4 | | 1 | | (U) | 1st French 2nd Maths | 1 | | 1 | | 1 |
14 | 8 | 8 | 3.7 | 4 | 4 | 4.6 | 4.5 | 4.2 | 3 | | U 1 | Reg. | 3 | | | 3 | 3 |
8 | 9 | 16 | 3.8 | 2 | 4 | 2.7 | 4.1 | 3 | | | 3 | Reg./French | | | | 1 | 1/2 |
2 | 10 | 5 | 4.6 | 4.4 | 5 | 4 | 4.5 | 4 | | | NU 3 | Reg./French | 1 | 1 | | | |
1 | 11 | 17 | 4.5 | 4 | 4.2 | 2.6 | 4.2 | 2.3 | 3 | 3 | U 1 | Reg. | | 3 | 3 | 3 | 3 |
22 | 12 | 11 | 4.5 | 4.5 | 4.4 | 3.4 | 4.4 | 3.8 | 2 | | U | English sets | 3 | 2 | 1 | | |
12 | 13 | 21 | 3.8 | 4 | 4 | 3 | 3.9 | 2.3 | | | U | Reg/French | 1/2 | 2 | | | |
23 | 14 | 2 | 3.2 | 4.4 | 3.8 | 4.1 | 4 | 3.5 | | | U* 1 | Alphabetical | 3 | | | 3 | |
21 | 15 | 23 | 3.5 | 3.3 | 4.3 | 3.7 | 4.1 | 2.7 | | 1/2 | U* 1 | Reg. | 3 | 2 | 2 | | 1 |
18 | 16 | 12 | 4.3 | 2.5 | 3.9 | 3.2 | 4.1 | 2.7 | | | U* 1 | Reg./RE/PE | 2 | | | 1 | 2 |
16 | 17 | 19 | 3.9 | 3.9 | 3.9 | 3 | 3.5 | 3 | | | (NU) 2 | Reg./French | 1 | 3 | 1 | | 1/2 |
19 | 18 | 20 | 3.5 | 4.7 | 4 | 3.5 | 3.7 | 3.8 | | 3 | U 3 | Reg./French | 1 | 3 | 3 | 1 | 1/2 |
11 | 19 | 18 | 3.7 | 5 | 4.2 | 4.5 | 4.1 | 4.9 | | | U 1/2 | Maths sets | | 1/2 | | 3 | |
5 | 20 | 1 | 3.1 | 2.6 | 3.5 | 2.5 | 3.3 | 2.8 | 1 | 1/2 | U 1/2 | Reg./French | 1 | 1 | 1 | 1 | 1 |
7 | 21 | 13 | 3.4 | 3.7 | 4.1 | 3.9 | 4.2 | 3 | | | U 1 | English | | 3 | 2 | 3 | |
25 | 22 | 4 | 4.3 | 2.9 | 3.2 | 2.7 | 3.9 | 2.9 | | 3 | U 3 | Reg./French | 1/2 | 3 | | 1 | 1 |
13 | 23 | 22 | 4 | 4.4 | 4.1 | 3.7 | 4 | 3 | | | U* | 1st Eng. sets 2nd not set | | | 1 | 1 | |
20 | 24 | 2 | 3.8 | 3.4 | 4 | 4.1 | 4 | 3.3 | 1 | | U 3 | Reg. | 1 | 1 | 1 | | 2 |
| MEAN | | 3.8 | 3.8 | 4 | 3.6 | 4 | 3.3 | | | | | | | | | |
Fourth, and finally, process data were collected during each class visit to collect survey data. Researchers scored all classes visited according to six variables: staff to pupil relationships, staff to staff relationships, pupil to pupil relationships, discipline, appearance of the school, and layout of the school. The ratings were on a scale of 1 (low) to 5 (high). These "researchers' perceptions" involved scores from around 20 different researchers which were then averaged for each school (Table
1). Data were collected from over 400 school classes which, in the majority of cases, were administered by one researcher, however, for 10% of classes there were two researchers present who each rated the classes. During training the criteria for the ratings were discussed and then new researchers shadowed a more experienced researcher during their first two school visits. During shadowing both researchers were asked to make their rating independently and then these were discussed for training purposes. All the Cohen's Kappa's for inter-rater reliability, for instances where two researchers rated the same school classes, were above 0.7. This indicates substantial inter-rater reliability above chance and good enough to proceed with analysis. Given that the paper is discussing school effects and not class effects, the ratings for each of the classes within a school were averaged. As we have confidence in the inter-rater reliability for each class, averaging to school level should provide an estimate of where each school lay compared to the others.
The school-level process data of the first and second kind are used within the modelling procedures to explain the residuals. The third and fourth kind of data are used externally to the modelling to try and further explain the residuals.
Statistical analysis including preliminary analysis
The main outcome examined in this investigation was a binary indicator of reported experience of sexual intercourse at time of follow-up. This was used in preference to use of the baseline reports of sexual intercourse for several reasons. Baseline data were not available for all pupils and the evidence suggested that some reports were unreliable [
28]. Further, the rates of sexual experience at baseline were low (18% for boys and 15% for girls). To explore 'school effects', a series of models were fitted to the data. These allowed us to examine the results before and after adjustment for pupil and school characteristics. Comparing 'school effects' between models revealed which factors contributed to differences between schools.
School level data were also incorporated in the modelling to ascertain whether they helped to explain the school effect. Data on thirteen variables had been collected at the outset of the trial to facilitate balanced randomization of schools [
33]. Principal components factor analysis was carried out to reduce the dimensionality of these data. This analysis identified 4 factors with eigenvalues greater than 1, accounting for around 80% of the variability. One variable, a composite measure of school ethos, was seen to be contributing roughly equally to 2 of the factors, so was deemed to be a contaminant, and thus removed from the analysis. In the resulting rotated factor solution it was found that the 7 deprivation-related variables -unemployment in school catchment area, deprivation score of local area, pupils' post school destination, proportion receiving free school meals, staying-on rates (S4 to S5 and S5 to S6), and attendance rates – were grouped together in the first factor. The second factor was dominated by the variables denoting access to clinics and the number of placement requests for a school (this picks out the urban/rural areas). Pupil-rated teacher-pupil and teacher-teacher relationships comprised the third factor and a proxy for school size in the fourth. Only factor one, which had high loadings for 7 of the thirteen variables, proved to be significant in explaining residuals in the levels of sexual experience at the school level. All 7 variables contributing to the significant factor were related to the deprivation of the school, with higher values being associated with more affluent schools. Thus only this factor was included as a school-level component at the stage of Model 5 (see below).
A two-level logistic regression model with pupils at level one and schools at level two was used. Computations were carried out using the GLMMIX macro in SAS Version 8.1 [
34]. This was done separately for boys and girls.
The modelling was carried out in stages, adding groups of individual level variables to a basic model as follows:
Model 1 : Basic model: pupils' age in months at follow-up and cohort
Model 2 : Model 1 plus individual socio-cultural variables (who the young person lived with, levels of parental monitoring, amount of personal spending money, mother's qualifications, mother's social class, mother's age, father's qualifications, father's social class, housing tenure, ethnic group, and strength of religious belief). The importance of these factors in predicting sexual experience has previously been demonstrated, for age 14 and age 16 [
28,
29].
Model 3 : Model 2 plus attitudinal variables (self-esteem, attitude to school, pupil-assessed teacher-pupil relationships, and proportion of peers perceived to be having sex).
Model 4 : Model 3 plus aspirational variables (assessment of the following in the future: being in a secure job, living with a partner, being in a training scheme, having a child/children, being at college/university, and being in a steady relationship with somebody. Plus an indicator of proportion of friends at other schools and proportion of friends who have left school).
Model 5 : Model 4 plus factor for school-level deprivation measures.
The following models were also considered in order to assess the influence of individual and school-level socio-economic factors on the outcome variable, independently of pupil attitudes and aspirations:
Model 6 : Model 1 variables plus factor for school-level deprivation measures.
Model 7 : Model 2 variables plus factor for school-level deprivation measures.
School level data and adjustment for missing data due to attrition at follow-up
Follow-up rates of the original cohorts varied by school. This was largely due to between-school differences in percentages of pupils who had left school at the time of follow up. To investigate the sensitivity of our results to this differential follow-up, two approaches were taken. The first was to omit all school leavers from the analysis, on the basis that schools might be having less impact on these early leavers. However, this analysis excluded a substantial proportion of the original cohort, whose behaviour may have been influenced by the school and would only allow generalization of results to those that remained at school. A better approach was to carry out a weighted analysis to compensate for the pupils missing at follow-up, and thus give inferences that could be applied to the whole original sample.
The weighted analysis assumes that data are missing at random, conditional on the variables used to calculate the weights. Baseline data plus an indicator of early school leaving (overwhelmingly leaving school for good) were used to develop a predictor of whether a pupil would participate at follow-up. The variables included in the weighting were: parental monitoring, family composition, spending money, early school leaving, sex (male/female), social class and level of alcohol consumption. This predictor was then used to calculate an inverse probability weight in order to estimate responses that would have been provided by pupils had they all participated at follow-up. Information for those responders in follow-up was used in the same modelling approach described earlier, with the data weighted using the relevant adjustment for each individual. Thus the school-level predictions arising from each model can be thought of as the proportion of sexually active pupils of each gender, adjusted to the levels that would be expected had the non-responding pupils been surveyed. In the results section we will refer to this as the weighted analysis.