Study design & subjects
In a pre-intervention study conducted in February-March 2012 (
t1), municipal health services (MHS) in the Netherlands contacted primary schools to recruit children (9–13 years, mixed gender and ethnicity) by telephone, e-mail, or advertisement in MHS newsletters. In total, 1,447 children from 40 schools participated in this study by completing a specifically developed and pretested compact paper questionnaire [
12]. This pre-intervention study aimed to examine the knowledge, perceived threat, and perceived importance of preventive behaviour in relation to tick bites and their potential consequences. Seventy percent of the children had a good knowledge of ticks and the potential consequences of tick bites. Knowing persons who personally got ill after a tick-bite was associated with a good knowledge score and leads to higher susceptibility and better appreciation of the need for body checks. Perceived severity was associated with a good knowledge score and with knowing persons who got ill after a tick-bite. Based on the results of this study, we concluded that it seemed to be useful to focus in future health education regarding ticks and tick-borne diseases on children besides parents.
In June and July 2012 (
t2), study participants were recruited by contacting children of the two final grades (grade 7 and 8) from the 40 primary schools who had participated in the pre-intervention survey (Table
1). Twenty-five out of the 40 schools involved in the pre-intervention study at
t1, participated again at
t2. We have no information about the reasons for those schools that were lost to follow-up. Children of the 25 schools were randomly assigned per class to either the intervention groups ‘game’ or ‘leaflet’, or to the control group. To ensure that schools were spread out evenly across the Netherlands a cluster randomization sampling design was used. The Netherlands was divided into regions, which allowed sampling of schools per region and then random selection of classes per school.
t1
a
| 40 | 1447 | 25 | 981 |
February-March 2012 Questionnaire 1 | | | |
Game
|
Leaflet
|
Control
|
group
b
|
group
b
|
group
b
|
254 | 328 | 399 |
t2
| 25 | 887 | 25 | 887 | | |
June-July 2012 Questionnaire 2 | | | |
Game
|
Leaflet
|
Control
|
199 (78.3%) | 316 (96.3%) | 372 (93.2%) |
Children either: played the game individually on a personal computer (game group); read a leaflet containing similar information as the game (leaflet group); or received no information (control group). Directly following the intervention (
t2), the children completed questionnaire 2 (Additional file
1: Appendix 1). Ninety percent of the children (887/981) who participated in the pre-intervention study at
t1, participated again at
t2. Absence due to illness was the most important reason for loss to follow-up. Since the children completed the questionnaire anonymously at
t1, they were grouped per class to enable comparative analysis of
t1 and
t2. Questionnaire 2 at
t2 included the same questions as the questionnaire at
t1 (pre-intervention study) complemented with questions about the appreciation of the intervention for the intervention groups.
Intervention materials
We developed an online educational video game
www.teekcontrol.nl, based on the results of the pre-intervention study [
12]. The scenario of the online educational video game
www.teekcontrol.nl is that the player drives around in one’s own neighborhood (selected by entering their postal code) and is then faced with different fictive risky situations for tick bites. An example of such a risky situation is children playing in nature and picking flowers in the bushes. The player has to chase tick bite cases across a map as quickly as possible, and while doing so emit warnings that encourage people to check for ticks. The faster the warning is emitted, the more points they earn. It is possible to play the video game individually, or in a league to become the best tick controller in town. At the end of the game, the children obtain their total score. They get the opportunity to inform their own parents (or responsible adult / guardian / carer) of their game results by sending an automatically generated e-mail about the score. This e-mail also includes information on ticks and LB for the parents. In addition, it is possible to share an automatically generated message via Facebook, Twitter and Hyves (the latter was a now defunct Dutch social network site), indicating you have played the game.
The leaflet was also specifically developed by RIVM for this study and explains to children in simple language and clear pictures what ticks look like, where and how they live, where they bite on the body and when it is important for them to ask parents to (help self-)check for ticks (Additional file
2: Appendix 2). The core lessons of the leaflet and the game are the same, only the mode of delivery differed.
In a similar vein to the pre-intervention study [
12], the game and the leaflet focus on determinants of preventive behavior in accordance with the Protection Motivation Theory [
22,
23]. This theory posits that a ‘threat appraisal’ is formed by an individual based on the perceived likelihood of a particular event (denominated here as ‘perceived susceptibility’) and its perceived severity. In the game we tried to simulate a threat appraisal by allowing the player to drive in one’s own neighborhood and making them face different risky situations with potential exposure to ticks. During the game, new risky situations appear in the game-field. The specific characteristics of these risky situations are also described point by point next to the game-field, thereby allowing players to recognize these situations in reality too. This is the first objective of the game: to teach children to identify risky situations with potential exposure to ticks.
The second objective is to teach children the right coping appraisal. From the pre-intervention study at
t1 it is known that only 18% of the children were, at that time point, routinely checked for ticks by their parents after ‘high-risk outings’. In the game, we tried to influence this coping appraisal by stimulating the players to alert others about the need to perform a tick check as soon as possible after a visit to an area with a high tick concentration. The faster the player reaches a tick bite situation and then emits ‘the tick check alert’, the more points that are earned. Herewith, the right coping appraisal (i.e., request a tick check from a parent after having been in an area with a high potential for exposure to ticks) is rewarded in the game [
24]. In the leaflet these two lessons are elaborated in text and pictures (Additional file
2: Appendix 2).
Questionnaires
The developed questionnaires were pretested among a sample of primary schoolchildren similar to our target group, and amended slightly as a result. Since our subjects are primary schoolchildren, the questions have to be limited in number and easily understood by children. Answers were presented as two or three options, text was limited to short sentences, and images were used when possible. We included the following constructs: knowledge (assessed by asking 7 questions on tick ecology, basic prevention, and tick bites); perceived susceptibility (asking the respondents whether they think they could personally become ill after a tick bite); an additional proxy for perceived susceptibility (asking whether the respondent personally knows someone who became ill after a tick bite); perceived importance of preventive behavior as a proxy for response efficacy (asking whether the respondent thinks tick-checks are important), and actual preventive behavior (asking for the frequency of tick checks performed by the respondent’s parent(s)).
Furthermore, children were asked whether they had been given previous classroom lectures on ticks. Teachers handed out the questionnaires, which were completed in the classroom, and they were returned to RIVM by mail.
This general survey among a sample of healthy children from the general population did not require formal medical ethical approval according to Dutch law [
25].
Analyses
We analyzed whether the game and the leaflet affect knowledge, perception and behavior in relation to ticks, tick bites and LB compared to the control group. Table
2 summarizes the design for this evaluation.
Game |
y
G1
|
y
G2
|
Leaflet |
y
L1
|
y
L2
|
Control |
y
C1
|
y
C2
|
Our main interest was any intervention effect for the game group,
yG2 −
yG1, and for the leaflet group,
yL2 −
yL1. Any observed difference
yC2 −
yC1 in the control group was used to determine whether there are learning effects and therefore mere measurement effects of only completing the questionnaires. In the presence of learning effects we considered a differences-in-differences (DID) design,
1 by adjusting the differences
yG2 −
yG1 and
yL2 −
yL1for the difference
yC2 −
yC1. This gives us an estimate for the intervention effects that is adjusted for learning effects.
For our statistical analysis, we included only children whose school participated both at
t1 and
t2. This selection of individuals makes it more plausible that any differences that may arise are the result of the difference in interventions, rather than any school-specific effects. From the pre-intervention study we concluded that substantial differences in knowledge of ticks exist between schools and therefore children [
12].
Generalized linear mixed models (GLMM) were applied to analyze the intervention effects, whilst taking into account that the data are clustered [
26]. Clusters are present in our data on two levels. First, children within the same class are likely to have a similar knowledge level, and therefore knowledge scores within classes are likely to be correlated. Second, the knowledge level of a child at
t2 depends on the initial knowledge level of this child at
t1.
We chose to dichotomize our responses, and used a GLMM assuming a Bernoulli distribution and logit link (i.e., logistic regression with random effects).
2 The knowledge level response was operationalized as a binary variable, by considering a minimum of 6 out of 7 questions correctly answered as sufficient, and insufficient otherwise (reference category).
3 For the other responses, we defined the classes as follows: knowing other persons with Lyme - “yes” versus “no”/“don’t know” (reference), becoming ill after a tick bite (susceptibility) - “yes” versus “no”/“don’t know” (reference), importance of checking - “very”/“somewhat” versus “not important” (reference), and frequency of checks - “very often”/“sometimes” versus “not at all” (reference). For the questions related to knowing other persons with Lyme, and becoming ill after a tick bite, “no” and “don’t know” were pooled together because they both reflect, to varying degrees, the fact that respondents could not confirm the question. For the importance and frequency of checks, the categorization was chosen to see whether the intervention affected the proportion of individuals that take the risk of Lyme disease serious.
To address the sensitivity of our findings to this categorization, we also performed the analysis using an alternative categorization: becoming ill after a tick bite - “yes” versus “no” (leaving “don’t know” out), knowing other persons with Lyme - “yes” versus “no”, importance of checking - “very” versus “somewhat”/“not important”, and frequency of checks - “very often” versus “sometimes”/“not at all”. In most cases, we found that the results did not alter - with the exception of the check frequency, which will be discussed in the results below.
Three types of GLMM models were applied: (1) a model to estimate treatment effects in t2 versus t1, (2) a model to estimate the difference in intervention effects between the game and leaflet groups, and (3) a model to estimate the intervention effects conditional on the covariate values (“knowing somebody with Lyme”, and “having had classroom lecture on ticks”). The rationale behind these models is as follows. In Model 1, we examined whether there exists any intervention effect at all for each group, without taking potential confounding into account. The assumption is made here that if a true effect exists, then this should already become apparent in this model because confounding over time (e.g., children might know more persons with Lyme disease in t2 than in t1) within a group is relatively modest. And if such an effect is found, we applied Model 2 to determine whether this effect remained after adjusting for confounding over time, and whether this effect was sustained between groups after adjusting for differences in confounders between groups (e.g., one group might have had more classroom lectures on ticks than another). In particular, we were interested whether the game and leaflet groups performed better than the control group, and whether the game and leaflet groups performed differently from each other. Furthermore, based on an effect in Model 1, one might wonder whether this effect differs between subpopulations (e.g., the effect may be smaller for children who received classroom lectures on ticks), and if this difference exists, whether the difference varies by intervention group. To further describe these models, we introduce some mathematical notation. First, we define the intervention group Z as the group having been exposed to a specific intervention; Z can either be the game group, leaflet group or control group (i.e., no intervention at all). Let Z
ij
be the group to which individual i in class j belongs. Furthermore, let p
ij
be the probability of a success (e.g., a high knowledge score), T
ij
the point of time and x
ijk
the k
th
covariate respectively. T
ij
= 0 refers to the baseline measurement, where all individuals are unexposed (because the leaflet was developed specifically for this study and not available anywhere). T
ij
> 0 refers to the time points in which the game and leaflet groups become exposed to the intervention, but the control group remains unexposed. T
ij
can be either 1 or 2. Note that we did not consider random effects on individual level in these models, for simplicity. It was not feasible to include models that take into account such random effects. Between-class variation was considered more important than between-person variation based on examination of the data.
Model 1: Treatment effects t2 versus t1
$$ \log \left(\frac{p_{ij}}{1-{p}_{ij}}\right)=\left({\alpha}_0+{a}_{0j}\right)+\left({\alpha}_1+{a}_{1j}\right){T}_{ij}, $$
Where a0j and a1j are a random intercept and random slope for classes, respectively. α1 provides the intervention effect for an individual (which can be interpreted as changes in p
ij
on a logit scale per individual between t2 and t1). This model was fitted separately per intervention group Z, and for all outcomes.
Model 2: Differences in intervention effects between intervention groups
$$ \log \left(\frac{p_{ij}}{1-{p}_{ij}}\right)=\left({\beta}_0+{b}_{0j}\right)+\left({\beta}_1+{b}_{1j}\right){T}_{ij}+{\beta}_2{Z}_{ij}+{\beta}_3{G}_{ij}{T}_{ij}+{\displaystyle \sum_k{\beta}_k}{x}_{ijk}+{\displaystyle \sum_k{\beta}_{k+m}}{x}_{ijk}{T}_{ij} $$
Here, β1 provides the intervention effect for the reference group (game group), β2 the difference between intervention groups at baseline, and β3 the difference in intervention effects between intervention groups at t. Here, we are mainly interested in β3. β
k
and βk + mrepresent the influence of confounding covariate k, but are not of interest. This model was fitted across all intervention groups. This model was considered for all outcomes. Covariates used were “knowing somebody with Lyme”, and “having followed classes on tick bites”.
Model 3: Conditional intervention effects
$$ \log \left(\frac{p_{ij}}{1-{p}_{ij}}\right)=\left({\gamma}_0+{c}_{0j}\right)+\left({\gamma}_1+{\gamma}_{ij}\right){T}_{ij}+{\displaystyle \sum_k{\gamma}_k}{x}_{ijk}+{\displaystyle \sum_k{\gamma}_{k+m}}{x}_{ijk}{T}_{ij} $$
Here, γk + m, gives the intervention effect conditional on x
k
. This model is fitted separately per intervention group Z. This model was considered for all outcomes. Covariates that were considered are “knowing somebody with Lyme”, and “having had classroom lecture on ticks”.