Assessing the effects of interventions for Aedes aegypti control: systematic review and meta-analysis of cluster randomised controlled trials

In 2013 we carried out a systematic search for articles published between January 2003 and June 2013 assessing the impact of chemical control, biological control and/or community mobilization as strategies for Aedes aegypti vector control. We searched the Medline, Ovid, BVS, LILACS, ARTEMISA, IMBIOMED and MEDIGRAPHIC databases. The search terms we used were “dengue”, “Aedes aegypti”, “chemical control”, “biological control”, “community-based”, “community mobilisation”, “social mobilisation”, “community empowerment”, “effectiveness” and “vector control”, and their Spanish and Portuguese equivalents. We updated the search in November 2016 to cover articles published up to the end of October 2016. We also reviewed the references listed in identified publications and included additional studies found in these lists, limiting our search to publications in English, Spanish or Portuguese.

Figure 1 is a flow chart of the studies identified and finally included in the systematic review and meta analysis. The first search in 2013 produced a list of 588 articles. In 2015, we added a further 27 studies and in 2016 we added a further 233 studies identified by a new electronic search and a manual search (total 848 articles). Two reviewers (VA and LA), working independently, reviewed the title and abstract of these articles. They excluded 749 articles: 590 because they clearly did not meet the inclusion criteria, and 159 because they were further publications of the same studies.

The pre-established inclusion criteria were:

The reviewers read the full text of the remaining 99 candidate articles and excluded 81 of them as not meeting the inclusion criteria. Five of the excluded studies did not measure impact on entomological indices, and 76 were not CRCTs. This systematic review includes all 18 remaining articles; 10 of these had the necessary information for calculation of the entomological indices to allow us to include them in the meta-analysis.

We extracted data from the 18 articles using a format developed by consensus among study team members. Two reviewers extracted the data independently and then resolved discrepancies by consensus. We assessed methodological validity of the studies using the Cochrane approach for assessing the risk of bias [28]. This includes an assessment of how the studies handled and reported: random sequence generation, blinding of participants and personnel, blinding of outcome assessment, handling of incomplete data, and selectiveness of reporting. We graded each paper for each domain as having low, unclear or high risk of bias, and then calculated an overall risk of bias.

We defined intervention effectiveness for each of the entomological indices (HI, CI and BI) as the difference between the intervention group and the control group at the last point of measurement. For each type of intervention (chemical control, biological control, community mobilisation) we performed a meta-analysis using a random effects model to estimate global intervention effectiveness for each entomological index (HI, CI, BI), estimating the combined overall Risk Difference (RD) and its 95% CI. The model took into account inter- and intra-study variability by weighting [29]. We carried out the analysis using the open-source software CIETmap [30] and the “meta” package of the statistical language R [31].

We performed the DerSimonian and Laird Q test [32] to assess the level of heterogeneity, with the null hypothesis of non-heterogeneity. We derived p-values for this test by comparing the Q statistic with the α-percentile of a χ² distribution with k-1 degrees of freedom (where k is the number of studies).

For each type of intervention, we measured each study’s influence on the overall estimated intervention effectiveness by replicating the meta-analysis for each of the three entomological indices, eliminating one of the included studies from the analysis at each step. We then quantified the differences in the overall results [29, 33].

We assessed publication bias using a funnel plot, which shows the sample size of each study next to the detected effect size. We used the Begg and Egger statistical test [34, 35] and considered p < 0.10 to be a statistically significant indicator of publication bias.

Among the eight CRCTs categorised as chemical control interventions, five tested the effect of insecticide-treated window and door screens or curtains: one as a single intervention [41], two combined with insecticide-treated water container covers [37, 42], and two combined with temephos or spinosad treatment of productive water containers [40, 43]. One trial tested the impact of insecticide-treated bed nets as a single intervention [38] and one tested the impact of temephos applied to water containers as a single intervention [36]. Ocampo et al. reported on a trial of lethal ovitraps and Bacillus thuringiensis israelensis (Bti) briquettes, alone or in combination, together with an initial education and clean-up campaign and regular household visits. Since education/clean-up and visits alone was also the ‘control’ condition, we categorised this as a chemical intervention of the deltamethrin lethal ovitraps [39]. Three trials had a staged intervention: in Guatemala deltamethrin-treated window and door nets were replenished and supplemented with temephos treatment of productive containers after 17 months [40]; in Colombia, deltamethrin treated container covers supplemented deltamethrin treated window and door nets after eight months in about half the clusters [42]; and in Mexico, researchers added spinosad treatment of productive water containers to cypermethrin treated door and window screens after 14 months.

The number of clusters randomised to intervention and control status varied widely, from just one very large intervention and one very large control cluster in Brazil [36] to 22 intervention and 66 small control clusters in Thailand [41]. The largest number of households to receive the intervention was also in Thailand, although the researchers only measured entomological indices in half of these [41]. The duration of follow up varied from six weeks to 18 months after the start of an intervention. In the three studies with two-staged interventions, the last measurements of entomological indices were at six weeks [40, 42] to 10 months [43] after the second intervention. For interventions beginning at single time point, the last measurements were at between four months [39] and 18 months [41].

Measured impacts of the interventions varied considerably. The temephos trial found no effect; the BI and CI were slightly lower in control than intervention clusters at most time points [36]. In the trials concerned with insecticide-treated window and door screens or curtains, three found an impact on pupal densities and other indices mainly after addition of the second intervention of treating productive containers [40, 43] or of adding treated container covers [42]. The trial of treated door and window nets alone found that the impact on BI at six months, when 71% of households used the nets, was not maintained at 18 months, when only a third of households used the nets [41]. In the report of the trial of treated window and door nets in Mexico and Venezuela, with added treated container covers in Venezuela only, the authors found a reduction in entomological indices in all clusters, not different between intervention and control clusters, and attributed this to a spill-over into the nearby control clusters. The authors of the Haiti treated bed nets trial also attributed the fall in indices in all clusters, with no difference between intervention and control clusters, to a spill-over effect. The trial of deltamethrin lethal ovitraps and Bti, alone and in combination, with education and household visits as the control condition, found no difference in entomological indices between the intervention clusters and the control cluster. The authors postulated this could be because the initial education and clean-up followed by repeated visits were in themselves an intervention as effective as the interventions being tested [39].

Only two of the CRCTs measured the impact of chemical interventions on dengue infection as well as on entomological indices, with inconclusive findings. In the trial of deltamethrin treated window curtains and container covers in Venezuela, Kroeger et al. reported that positive adult dengue IgM serology at eight months was lower in intervention than control clusters, with borderline statistical significance [37]. In the trial of treated bed nets in Haiti, in all clusters there were fewer individuals positive for dengue IgM at 12 months; the authors considered the lack of difference between intervention and control clusters reflected a spill-over effect.

Only one study of a biological control cluster trial met the inclusion criteria. Kittayapong et al. in Thailand of using either copepods or Bti (the households had a choice) in household water containers to control breeding of the dengue vector [44]. The intervention also included community mobilisation meetings and recruitment of eco-health volunteers (EHVs) from among existing community health volunteers. The EHVs visited households to deliver the biological control materials and educated household members on elimination of vector breeding sites. Public services cleaned up communal spaces in the communities. Although there was also an element of community mobilisation, we categorised this trial as primarily of the biological control methods. The study compared 10 intervention clusters with 10 control clusters, with measurements of vector indices up to six months. The HI, CI and BI were significantly lower at follow up than at baseline in all clusters, but not so in control compared with intervention clusters. The PPI was significantly lower in intervention than control clusters at all time points after the baseline.

We categorised nine CRCTs as primarily trials of community mobilisation and participation, seven from Central and South America [45, 46, 49‐53] and two from Asia [47, 48]. One trial from Mexico measured the impact of an educational intervention at household and community level and a chemical intervention (space spraying with malathion and temephos applied to household water containers), alone or in combination, compared with a control cluster with neither intervention [45]. Common features of the complex interventions included: engagement of local stakeholders in discussions of the problems and planning of activities; involvement of community members in prevention and dissemination activities; household visits to support their efforts to reduce dengue breeding sites; educational programmes at household and community levels; partnerships with local services; and efforts to improve local services such as garbage collection. Four trials involved schools and schoolchildren [47, 50, 51, 53] or elders [50]. Two noted the importance of involving women [47, 53]. Specific activities included: distribution of locally made covers for water containers [47], promoting composting of biodegradable waste [48], and collecting small waste items from around houses [52].

In all trials, the routine government dengue control activities continued in the intervention as well as control clusters, so the measured impact was of the community mobilisation in addition to the routine prevention activities. In the trial from Ecuador [51], the analysis was complicated by a change in the government programme midway through the intervention: from a programme based on temephos in water and insecticide space spraying to use of a biolarvicide (Bti) and education for source reduction.

The included CRCTs varied in size, from a very small study in Mexico with three intervention clusters and one control cluster and a total of 187 households [45], and a small study in Sri Lanka with four intervention and four control clusters and 1593 households [48], to a study in Cuba with 16 intervention and 16 control clusters and a total of 19,170 households [46], and a trial in Nicaragua and Mexico with 75 intervention and 75 control clusters and a total of 18,838 households [53]. Length of follow up varied from five months [45, 50, 52] to 24 months [49]. Some trials reported only measurements at baseline and follow up [45, 50‐52], while others made one or more measurements in between [46‐48, 53]. One trial in Cuba relied on monthly measurements by the government vector control programme [49].

The reported impacts of the CRCTs varied but were broadly positive, with a significant impact on at least one entomological index. Four studies found all the measured indices were significantly lower in the intervention than control clusters at the last follow up [46, 47, 50, 53]. The trial from Sri Lanka with a focus on solid waste management found a significant impact on BI at 15 months and on PPI at all time points [48]. The trial in Cuba that used figures from the routine government surveillance found significantly lower BI in intervention clusters at all time points [49]. The Ecuador trial of the elementary school education programme and the clean patio safe container programme detected a significant impact on PPI only at 12 months, but only when clusters without full implementation were excluded. This trial was complicated by the change (probably improvement) in the government programme in the control sites midway through the intervention [51]. The Uruguay trial reported a non-significant difference between intervention and control cluster in favour of the intervention; low vector densities in the sites reduced the power of the study to detect significant differences [52]. The small complicated trial from Mexico compared an educational intervention, with or without malathion spraying, with a control cluster. It found a significant impact of the education programme only on a specific index (positive containers per household); this impact was less marked when the education intervention was combined with malathion space spraying [45].

Only one CRCT of community mobilisation measured the impact on dengue infection. The trial in Nicaragua and Mexico found a significant impact on childhood dengue infection (assessed by dengue antibodies in paired saliva samples) and on self-reported dengue cases in households [53].

We assessed six studies in the meta-analysis as having a “low risk of bias” and four as having an “unclear risk of bias”, because they did not report some of the information needed to assess elements of the risk of bias (Table 3).

Eight of ten articles in the meta-analysis provided the necessary data to calculate the combined effectiveness for all three entomological indices (HI, CI and BI). One study provided information for only two indices (HI and BI) and one provided information only for the HI. Table 4 summarizes the data for the Aedes aegypti indices in the last measurement for each study’s intervention and control groups, with calculated intervention effectiveness estimates (RD and 95% CI). In every trial of community participation, the estimated intervention effectiveness was positive, showing a decrease in the HI, CI and BI; the higher 95% CI limit for these estimations is 0.03 (for the HI and BI), from the study by Basso et al. in Uruguay [52].

The overall intervention impact assessments for the Household Index were −0.01 (95% CI -0.05 to 0.03) for chemical control, and −0.10 (95% CI -0.20 to 0.00) for community participation (Fig. 2). None of the confidence intervals for impact on HI from the studies of community participation interventions included unity, reflecting a consistently significant impact on this index. The single CRCT of biological control reported an impact of −0.02 (95% CI -0.07 to 0.03) on the HI.

For the Container Index, community participation interventions again showed the most consistent impact. The overall intervention impact assessments for CI were 0.01 (95%CI -0.01 to 0.02) for chemical control interventions, and −0.03 (95%CI -0.05 to −0.01) for community participation interventions (Fig. 3). The single CRCT of biological intervention reported an impact of −0.02 (95%CI -0.04 to −0.01) on the CI.

The estimated combined intervention impact of chemical control on the Breteau Index was 0.01 (95% CI -0.03 to 0.05), while that of community participation was −0.13 (95% CI -0.22 to −0.05) (Fig. 4). The impact on BI of the single biological control trial was −0.08 (95% CI -0.15 to −0.01).

Unlike earlier systematic reviews [14‐25], our review only included CRCTs. Our meta-analysis required data for calculating classic Aedes aegypti entomological indices. This limited the number of studies eligible to be included but it meant that the quality of the included studies was relatively good. None of the 18 studies included in our systematic review was considered to have a high risk of bias, although 10 had an “unclear” risk of bias, mostly due to lack of information in the reports. Other limitations of our meta-analysis are the heterogeneity of intervention duration, the small number of clusters in some of the studies, and the variable cluster size, all of which could affect the intervention effectiveness estimates. The sensitivity analysis, however, showed stability of the global effectiveness estimates.

The grouping of different kinds of interventions together into the broad categories of chemical interventions, biological interventions, and community mobilisation interventions in the meta-analysis could lead to the effectiveness of a particular intervention being under-estimated because it is over-shadowed by poor performance of other interventions in the same broad group. We do not believe this is likely in our study. The four community mobilisation studies all showed positive impacts, albeit of varying magnitude. And among the five chemical intervention trials, three were of treated window and door curtains or nets, and one was of treated bednets, with only one being a different type of intervention (lethal ovitraps).

We were not able to include the main chemical control methods used in government Aedes aegypti control programmes – temephos in domestic water containers and peri-domestic insecticide spraying – in our meta-analysis because we did not identify any CRCTs with details of impact on entomological indicators. In the descriptive review, we included one CRCT of temephos use, with no significant impact on entomological indices [36], and a CRCT that studied both and education programme and ultra-low volume malathion spraying and temephos application, and found that the ULV spraying reduced the effectiveness of the educational intervention [45].

Table 3 and Figs. 2- 4 are based on the last measurement point comparing intervention and control sites in each trial. It is possible that this missed some useful impact for some of the interventions. In the trials reported by Lenhart et al. [38] and Vanlerberghe et al. [41] the difference between intervention and control clusters was greater in earlier measurements than later measurements; the authors attributed this to spill-over effects or reduced coverage of the treated materials over time.

Cluster trials, assessing community effectiveness, unlike household or container based trials, take account of community level dynamics. In this real life setting, our review shows chemical control was less effective than community mobilisation, for all three entomological indices.

Depositing temephos in water storage containers is the mainstay of most centrally managed Aedes aegypti control programmes in Latin America and elsewhere [6]. A recent systematic review of the effectiveness of temephos for dengue vector control concluded there was evidence of impact on entomological indices of Aedes aegypti when temephos use was evaluated as a single intervention; effectiveness varied considerably depending on factors such as frequency and method of application and usually did not persist for more than three months. The effect of temephos was less in studies where temephos was part of a combined intervention, as it is almost everywhere in Aedes aegypti control programmes [21]. Most of the studies in the temephos review by George et al. were not CRCTs [21]. The single CRCT of the use of temephos alone included in our systematic review reported no impact of temephos on entomological indices [36]. In Guatemala, the use of temephos together with deltamethrin treated window and door nets had an impact on some, but not all, entomological indices [40].

Outside the research context, Aedes aegypti control almost everywhere implies complex interventions and cluster dynamics. Community mobilisation implies changes in human attitudes and behaviour, which in turn has multiple effects: people might be motivated to control breeding sites and to cover water containers, to work together on communal vector breeding sites like cemeteries, and they might also be motivated to remove pesticide from water containers. From a centrally managed programme, it would be difficult to foresee the exact mix of interventions to suit every community. Centrally managed awareness and education programmes are thus a weak basis to achieve community commitment to and ownership of interventions. Sustainable community engagement includes local evaluation of evidence and co-designing interventions that best suit their local conditions and culture [54]. This community authorship, rather than interventions being imposed or advised from outside, seems to underwrite the success of the Camino Verde intervention in Mexico and Nicaragua [55].

The cost implications of multi-faceted programmes for vector control need further study. Countries using temephos and insecticide spraying as key elements of national vector control programmes already carry the expense of centralised programming and logistical structures, and the vertical management and huge numbers of local personnel required to achieve monthly or bimonthly household visits. These countries are paying for vector control that, judging by the relentless increase in dengue risk and recent explosive zika and chikungunya epidemics, does not work very well. A central concern in adding community engagement efforts is how much this would add to effectiveness and acceptability, in relation to the added cost. The cost of adding community engagement might also be offset if it helped to support uptake of a dengue vaccine as that becomes a real public health option.