Background
National malaria control programmes (NMCPs) must ensure that all people living in malaria transmission areas are protected through the provision, nightly use and timely replacement of high quality long-lasting insecticidal nets (ITNs) and where appropriate, the additional application of indoor residual spraying (IRS) [
1]. While it is assumed that all ITNs that have World Health Organization (WHO) prequalification listing last for 3 years, several ITN products are available that may vary in price as well as performance under local conditions [
1‐
7]. Because ITNs are the primary means of malaria control, their durability, measured through physical integrity and bioefficacy against anopheline mosquitoes, needs to be accurately assessed in order to inform NMCPs of the most cost effective products and the correct interval for net replenishment campaigns [
8].
Any ITN product is expected to retain its insecticidal activity (bioefficacy) for a minimum number of 20 standard washes or 3 years of use under field conditions as defined by the WHO [
9]. However, the durability (years of functional life) of both existing and new net products under development is a crucial consideration. Despite mass distribution of ITNs, currently fewer than 50% of people living in malaria endemic areas are covered by one of the core malaria interventions: either ITNs or IRS [
10]. Maximizing ITN access through the provision of the most long-lasting, and cost-effective products remains a critical concern, particularly as a number of countries have shown an increase in malaria in the past year (2016/2017) as investments in malaria control have plateaued [
10].
For products within new intervention classes e.g. dual active ITNs, an Evidence Review Group (ERG) report to the WHO Malaria Policy Advisory Committee (MPAC) recommended specific guidance on the assessment of non-inferiority of products within a class [
11]. A non-inferiority trial of an intervention aims to demonstrate that the test product is not worse than the comparator/reference by more than a pre-specified margin [
12], known as the non-inferiority margin. For ITNs this margin relates to mortality or feeding inhibition. In recognition of the importance of ITN durability, the WHO recommended that once sufficient test and active comparator ITNs from large-scale field trials have been collected over 3 years of field use, a second set of two non-inferiority trials should be conducted to ensure that the test product continues to be non-inferior to the comparator/reference product for up to 3 years on both mosquito mortality and blood-feeding inhibition endpoints [
13]. While this guidance recommended that non-inferiority trials should be conducted in experimental huts it was acknowledged that alternative methodology for non-inferiority testing including the ambient chamber test or the tunnel test should be explored.
The standard means of ITN bioefficacy evaluation is through cone bioassays, WHO tunnel tests and experimental hut evaluations [
14]. The cone test is a contact assay where mosquitoes are held in proximity to the ITN and mosquito knockdown (KD60) and 24-h mortality are recorded after 60 min and 24 h, respectively. The tunnel test uses a live animal as a bait (rabbit or guinea pig), so mosquitoes are able to exercise host-seeking behaviour, and ITN efficacy is assessed by measuring mosquito mortality and blood feeding inhibition [
15‐
17]. Experimental huts are small scale field (phase II) testing assays used to evaluate ITNs that meet laboratory (phase I) testing criteria [
8,
18]. Huts are built in areas with high densities of target mosquito species and are designed to resemble small local housing but have features to retain mosquitoes that enter huts such as window traps and baffles [
19]. Volunteers sleep underneath the ITNs and wild mosquitoes attempt to feed and interact with the ITNs in the same way as they would in local homes. Both mortality and feeding inhibition are key outcome parameters, which translate to personal and community protection from malaria [
20].
However, all assays have some limitations, which need to be considered when assessing bioefficacy of ITNs. WHO cone tests may underestimate the induced mortality of irritant insecticides, as mosquitoes do not settle on treated nets [
21]. Indeed, comparatively higher mortality is often measured in experimental hut studies of ITNs where mosquitoes make repeated contacts with treated nets as they try to feed on human volunteers sleeping under nets. In the WHO tunnel test, the live host used as bait is not the preferred host for the strongly anthropophilic Afro-tropical vector
Anopheles gambiae sensu stricto (s.s.) [
22] and may overestimate feeding inhibition. Alternatively, mosquitoes must be reared by feeding them on small mammals to select them for a preference to these non-preferred hosts, which is both expensive and of animal welfare concern. Experimental hut bioassays are the gold standard for ITN and IRS evaluation, but wild mosquito populations are often seasonal and have high temporal heterogeneity requiring substantial replication to ensure adequate power to detect true effect differences between products [
23].
Therefore, presented here is the first evaluation of a new standardized semi-field assay: the Ifakara Ambient Chamber Test (I-ACT) assay. The assay was used to evaluate the bioefficacy of whole ITNs that were returned from the field in a longitudinal durability study. This study measured the bioefficacy of used (field-aged) ITNs using the I-ACT assay and standard WHO durability testing bioassays (cone and tunnel tests). The proportion of nets passing WHO criteria by standard methods and I-ACT was compared. The aim was to demonstrate the utility of this new assay for measuring bioefficacy of different ITN products and to explore its applicability for non-inferiority testing of new ITN products [
11]. Further work comparing the I-ACT and experimental hut evaluations of ITNs will be reported separately.
Discussion
This is the first ITN durability study to compare bioefficacy of ITNs using standard WHO bioassays (cone and tunnel tests) with data collected from whole nets tested using the I-ACT. A large numbers of three brands of nets returned from the field were evaluated in I-ACT to measure their protection to users sleeping underneath them in the presence of natural wear and tear. The I-ACT allowed high throughput (48 nets per brand per year) to give precise estimates of overall product efficacy. Data from the I-ACT for the deltamethrin nets PermaNet® 2.0 and Netprotect® that function through rapid knock down and mortality largely agrees with the standard WHO methods (cone/tunnel tests). However, for Olyset® that functions through the prevention of feeding a greater proportion of nets passed using I-ACT than standard WHO methods (cone/tunnel tests).
It was observed that each of the three net brands showed lower efficacy measured by standard WHO bioassays compared to efficacy measured by I-ACT. This could be due to: (1) duration of exposure (3 min vs 12 h), (2) surface area of treated fabric presented to the mosquitoes (both standard WHO methods use 20 cm
2 samples versus a whole net in the I-ACT) and (3) number of tarsal contacts with the ITN resulting in exposure to different dose of insecticide due to the presence of a human host under the net for the I-ACT. In cone test experiments, mosquitoes are exposed to tested ITN for only 3 min which may not allow the tested mosquitoes to exercise natural host-seeking behaviour with multiple contacts over the net surface resulting in a higher cumulative dose of insecticide. This has also been measured by other authors in studies to understand behavioural and physiological changes in mosquitoes in relation to responses to insecticides. A series of studies by Angarita-Jaimes and colleagues using a novel video-tracking system to quantify the behaviour of nocturnal mosquitoes attacking human hosts in the laboratory and in field observed that, both
An. gambiae s.s. and
Culex quinquefasciatus showed multiple contacts with bed nets when a human host was present [
30], and this host seeking activity is lower for treated nets than in untreated nets [
31,
32]. However, the I-ACT study demonstrated that these contacts were sufficient to kill or inhibit feeding among the majority of pyrethroid susceptible mosquitoes used in this study.
In addition, the findings add to existing data that shows that the cone test underestimates the bioefficacy of Olyset
® that contains Permethrin, a contact irritant pyrethroid [
15,
17,
33,
34]. During cone tests, permethrin causes mosquitoes to minimize contact with the netting fibres and they may sometimes rest on the sides of the cone or cotton plug on the cone and avoid the insecticide and demonstrate frequent take offs from the net [
28].
The tunnel test was developed as a consequence of the need to measure feeding inhibition of permethrin-treated nets [
35] and has also shown some use in evaluating products that fail cone tests including chlorfenapyr products as it allows mosquitoes to exhibit flight and host seeking feeding behaviour in a natural simulated condition [
36]. However, as with cone test, the tunnel test has some limitations. The overall pass rate (using 12 or 24-h mortality and blood feeding inhibition), as measured following both WHO 2005 and 2013 criteria, was lower compared to that measured by I-ACT. A possible explanation for this observation is that, the baits used in tunnel tests are rabbits that are not the preferred bait for
An. gambiae s.s. that feeds almost exclusively on humans [
22,
37]. Therefore, during standard tunnel test experiments, mosquitoes may be less responsive to non-preferred bait and remain in the releasing chamber throughout the exposure time without interacting with the ITN sample resulting in a lower cumulative exposure to insecticide. Additionally, using a whole net in the I-ACT killed more mosquitoes possibly due to the large surface area of insecticide available for mosquitoes to interact with. It is known from repellent testing that use of a non-preferred bait will overestimate repellent efficacy [
38]. However, a similar number of PermaNet
® 2.0 and NetProtect
® passed the combined WHO tests and the I-ACT whereas fewer Olyset
® passed combined WHO tests indicating that the WHO tests are conservative and therefore unlikely to pass a product that is of low efficacy. As the I-ACT is a less conservative test it may have use for early screening of new insecticide treated nets including those with irritant insecticides or those that function through a mode of action other than rapid knockdown before more costly experimental hut tests.
The overall percentage of Olyset
® nets that passed tunnel test following WHO 2013 guidelines was marginally lower than when following WHO 2005 guidelines [
25]. This suggests that reinstating the WHO 2005 pass/fail efficacy criteria may be justified to avoid missing products that are efficacious during early testing. This will also align tunnel test holding times with those of cone bioassays and experimental huts (24 h). It may also be justifiable when testing some products to hold mosquitoes for even longer than 24 h as some authors have done to measure the effects of slow acting insecticides [
39,
40]. This simple pairwise test between the two guidelines demonstrated the usefulness of exploring the impact of holding time on the outcome of product tests.
The mode of action of insecticides used on ITNs is an important consideration when selecting bioassays. New products with modes of action different from pyrethroids (which are fast acting and neurotoxic) are coming to market and there is a need for a suitable means to bioassay them. An example is chlorfenapyr, which acts by disrupting metabolic respiratory pathways (oxidative phosphorylation) in the cells of mitochondria and that require the conversion of the active compound through metabolism [
41]. The conversion is optimal at night and is maximized when mosquitoes are metabolically active i.e. during the active part of their circadian rhythm and flying during host seeking [
42]. Cone tests are usually conducted during the day and take 3 min exposure time with no bait involved. Findings from two well conducted studies by Oxborough et al. and Ngufor et al. observed extremely low levels of mortality caused by chlorfenapyr compared to pyrethroids when assessed by cone test, but excellent effect against resistant mosquitoes when tested in experimental huts [
39,
43]. These data again suggest that cone test may be best suited for fast acting non-irritant insecticides [
35] and there is a need to be open to exploring new bioassays for new mode-of-action products. The higher pass rate of I-ACT compared to standard WHO tests may be useful when conducting “quick and dirty” tests for new products to avoid early “kill” of promising products because they are failing to pass bioefficacy criteria in phase I laboratory tests when they may prove highly efficacious in gold standard experimental hut tests (Phase II).
Ifakara Ambient Chamber Test may be useful in evaluating new products that function through either mortality or feeding inhibition. Tests are conducted at times when mosquitoes are metabolically active, and using the preferred host of Afro-tropical malaria vectors. The advantage of using the I-ACT is that nets are evaluated using mortality and feeding inhibition using just one test rather than having to perform the cone (for mortality) followed by the tunnel test bioassays (for feeding inhibition or mortality at night). Regarding the issue of precision in outcome measure estimates, the durability study performed here in the I-ACT used 30 mosquitoes per chamber per night of experiment and allowed large numbers of nets to be evaluated without exhausting the insectary which is always a concern when product testing. It is important to assess a large number of nets in durability studies to allow a sufficient sample of nets to be returned from the field to capture the large heterogeneity in product performance i.e. fabric integrity and insecticidal content, and using a random sampling framework that is large enough to avoid sampling bias such as the Hawthorne effect [
44].
When the efficacy of ITNs was compared using standard WHO assays and I-ACT, it was seen that most of the tested nets were extremely effective against mosquitoes and passed WHO criteria of feeding inhibition and/or mortality using the pyrethroid susceptible
An. gambiae s.s. (Ifakara) strain even after 3 years of use with natural damage and insecticide depletion from the field. This has also been shown by other research in Tanzania [
34,
45,
46]. Many of the tested nets were damaged. The median hole surface area was 459 cm
2 in Olyset, 295 cm
2 in Permanet and 152 cm
2 in NetProtect in year 3, which means that most surviving nets were in the “damaged” category, but remained highly protective.
In addition, a simple non-inferiority test was conducted using WHO criteria to evaluate the effect of difference between products for mortality and feeding inhibition. Olyset
® was used as the reference product (first in class or active comparator) against which the two other brands (second in class, test product or innovator product) were compared since it is the standard of care in Tanzania. PermaNet
® 2.0 and Netprotect
® were non-inferior compared to Olyset
® on the feeding inhibition endpoint and superior to Olyset
® on the 24-h mortality endpoint when measured in the I-ACT. The WHO passes a product based on a combination of mortality and feeding inhibition, and based on these criteria, PermaNet
® 2.0 and Netprotect
® were non-inferior to Olyset
® based on data for three-year durability. This was also seen with WHO bioassays: Olyset
® demonstrated lower mortality and similar feeding inhibition to PermaNet
® 2.0 and Netprotect
® when tested using cone tests and tunnel tests. Estimates of efficacy from the sample of 144 nets per brand were very precise and a 10% effect difference in mortality could be observed. However, it is unlikely that 144 nets per brand could be cost effectively evaluated in experimental huts. A comparison study between Ifakara experimental huts and the I-ACT using 24-h mortality and feeding inhibition outcome measures is in progress (Moore et al., pers. commun.) and will show how I-ACT and gold standard experimental huts compare for non-inferiority evaluation of ITNs. This is important since experimental huts are used to measure entomological correlates of the epidemiological effectiveness i.e. the public health benefit of interventions [
47].
Therefore, the I-ACT could prove useful for testing insecticidal materials that can provide a high throughput option for evaluating
functional bioefficacy of ITNs i.e. the true protection as a function of damage and bioavailability of insecticide in durability studies. Functional bioefficacy i.e. incorporating insecticidal effectiveness has also been suggested by WHO’s Malaria Policy Advisory committee to be included for net durability assessment [
48]. While the methods presented here may not be useful for operational durability monitoring they may be useful for consideration in WHO “Phase 3” community field assessments of ITNs.
In this new assay, recapture of released mosquitoes is 99% so 30 mosquitoes were consistently “captured” every night I every chamber which is unlikely to be the case in standard experimental hut studies [
45,
49‐
57]. Experimental hut studies rely on wild mosquitoes entering the hut, and the nightly number of mosquitoes captured is highly variable and consequently substantial replication is required to obtain adequate precision to estimate true effect differences between products [
23]. As mosquito densities fluctuate due to seasonality in rainfall it is useful to have a whole net assay that is not dependent on field populations of mosquitoes that may limit the windows of opportunity to conduct tests with adequate mosquito densities to achieve power. Whole net bioassays where the interaction between insecticide and fabric integrity is measured are important for selecting between products or ranking their durability [
48]. Bioassays that assess only the insecticidal bioefficacy of a net sample may favour poor quality nets that tear easily reducing user protection and consequently user acceptance, which will eventually lead to the user discarding the net [
58].
The experimental hut bioassay that simulates domestic conditions and allows nets to be tested against wild mosquitoes is the definitive test of ITN efficacy [
43]. This study had several limitations. Firstly, I-ACT uses laboratory-reared mosquitoes, which means it relies on laboratory strains that may have different resistance mechanisms to those locally or limited genetic diversity. Secondly, the I-ACT test is a more expensive infrastructure to establish compared to small WHO cones and WHO tunnel glass chambers, requires space and it is immovable. The assay must be conducted in climate-controlled chambers or in areas with suitable ambient conditions to conduct the tests. In contrast standard WHO cones and tunnel chambers which can be taken anywhere and tests conducted provided the environment is set to standard conditions for conducting tests. The I-ACT needs to be compared to experimental hut tests, but it did agree well with findings of standard WHO methods using pyrethroid susceptible mosquitoes. Evaluations of ITNs with pyrethroid resistant strains as well as using dual active ITNs will be reported in subsequent publications.
Based on the data here presented, the overnight I-ACT may be a bridge between the lab and the field. Data agreement with standard WHO testing methods was excellent, with high sensitivity and specificity. It allows mosquitoes to host seek during the active phase of the circadian rhythm, and have multiple contacts with treated netting in a more realistic way. It uses the preferred human host but allows laboratory-reared mosquitoes to be used. This improves safety for human volunteers because laboratory-reared mosquitoes are disease free and allows sufficient numbers of mosquitoes to be released to reach the power needed to conduct precise comparisons of product performance.
Authors’ contributions
Conceived and designed the experiment DJM, SJM, JDM. Performed the experiments DJM, JDM, WSN, ZMM. Analysed the data DJM. Contributed to the analysis SJM, JB. Wrote the manuscript DJM, SJM. Critically revised the final manuscript LML, SM, EM, WNK, ZMM, JB, HJO, SJM. Drew the diagrams JDM, SA. All authors read and approved the final manuscript.