Methods
Study area
The study was carried out between 2000 and 2005 in Kyenjojo District in west Uganda, a hilly area with altitudes between 1,350 and 1,550 meters. Annual rainfall ranges from 1,200 to 1,600 mm, mean temperature is 21.3°C, and average relative humidity about 70%. Malaria is meso- to hyperendemic with
Plasmodium falciparum prevalence rates in asymptomatic children age 2–9 years between 45 % and 68% [
15,
16]. Main vectors are
Anopheles gambiae sensu strictu and
An. funestus with Entomological Inoculation Rates (EIR) estimated around 7 infective bites per person per year [
17]. Study site was in Kirongo Parish, Nyantungo Subcounty and involved five villages: Bucuni, Bwendero, Kasunga, Kidomi and Kyakahuli. These had previously participated in a study on insecticide treated curtains showing excellent cooperation. In addition, village health workers with intensive experience in surveys and field work were available.
Study design and sample size
This was a prospective study with mosquito nets as the unit of observation. Two types of nets, LLIN and ITN, i.e. conventionally treated nets, were randomly distributed to households with users as well as field staff initially blinded with respect to the type of net. In intervals of 6 or 12 months a sample of nets was randomly selected out of the pool of study nets for testing with bioassay (mortality and knockdown rate of mosquitoes) and chemical residue as the principle outcome measures. Two distinct phases of the study can be distinguished each testing a different product of LLIN and comparing it to ITN. Time of follow-up for LLIN was 39 and 36 months respectively and that for ITN was 12 months.
The necessary sample size, i.e. number of nets to be sampled at each time point was calculated based on the main outcome variables. It was found that a sample of 40 nets for each type (LLIN and ITN) would be sufficient. Assuming one measurement for each of the nets per time-point, an alpha error of 0.05, power of 80%, a standard deviation of 8.0 (taken from previous deltamethrin studies) this sample size was sufficient to detect a decline of 8 mg/m2 or more between time points as statistically significant. Similarly, it allows the detection of a difference of at least 15%-points between time points or type of nets in vector knockdown or mortality rates as significant.
While this study was designed before the publication of the WHOPES guidelines for phase III field testing of LLIN [
18] its approach is in keeping with these recommendations.
Tested products and net treatments
Two versions of the LLIN PermaNet® (Vestergaard-Frandsen) were tested. The first generation is a multifilament polyester net treated with a target dose of 50 mg/m2 of deltamethrin using a coating technology to enable wash resistance and create a reservoir of insecticide. The first generation test nets were part of a shipment of 10,000 nets delivered to the Commercial Market Strategies (CMS) project in Kampala in November 2000 for social marketing and were produced in September 2000. Nets were stored in the CMS warehouse in bundles of 50 nets. Nets for the study were identified at random by first selecting 46 bundles from the shelves in the warehouse and then from each bundle 10 nets giving a total of 460 LLIN.
For the second generation PermaNet® no large scale shipment was available and 270 test nets were directly received from Vestergaard-Frandsen in August 2002. These multi-filament polyester nets were part of routine production with a target dose of 55 mg/m2 of deltamethrin. At the time of production (July 2002) the company did not yet use batch numbers on the labels so these are not available.
ITN for phase one (test of first generation PermaNet
®) were nets identical to the LLIN with respect to netting material but were sent from the factory untreated. The treatment of these 150 nets was done in the CMS warehouse in Kampala using a 1% suspension concentrate of deltamethrin (K-Othrine, Bayer Environmental Science), one of the WHOPES recommended insecticides for net treatment [
19]. Based on a surface area of the nets of 13.1 m
2 and a target dose of 25 mg/m
2 33 ml of insecticide solution and 400 ml of water were used per net. A solution was prepared in a bucket for two nets at a time and nets soaked for two to three minutes under continuous kneading. After excess water had dripped off nets were dried flat on plastic sheeting in the shade of the warehouse.
For the second phase of the study (test of second generation PermaNet
®) the ITNs comprised one group of the original nets re-treated after 15–18 months of field use and one group of new, multifilament polyester nets (Siamdutch Netting Company, Thailand) which were exchanged for other conventionally treated nets. Net treatments were done in the field using the wetable tablet version of the same WHOPES recommended deltamethrin [
19] (KO-Tab, Bayer Environmental Science) containing 360 mg deltamethrin. Nets were treated by experienced and supervised field staff using basins and 400 ml of water and 1 insecticide tablet according to manufacturer's instructions. Nets were dried flat on the grass and as much as possible in the shade.
All nets, LLIN and ITN, were white, rectangular nets of 75 denier and a size of 160 × 150 × 180 cm (width, height, length).
Field procedures
From each type of net (first and second generation LLIN, ITN) 10 randomly selected nets were kept for baseline analysis while the remaining nets, 450 first generation LLIN, 140 ITN and 260 second generation LLIN were prepared for distribution to households. For the first phase of the study all existing labels on the nets were removed in order to allow blinding and an identification number, printed with wash resistant ink on a piece of polyester band, stitched on the net. The numbers had been previously randomly allocated to the two groups (LLIN and ITN) so that no identification of net type was possible purely by the ID number. After the ID number labels had been fixed to the nets they were re-sorted by ID number thereby mixing the nets at random. Based on a household list from the five villages net numbers were randomly allocated to households according to available sleeping places to ensure equitable distribution. No specific instructions regarding use or washing were given to the net users.
The procedure for second generation PermaNet® was similar with the difference that labels were not removed since at the time only LLIN were distributed. Random allocation to households was done based on "vacancies" according to the master list of nets which was kept by the principle investigator and recorded the status of each net (type, ID number, household number, time of distribution, time of sampling, replacement or removal).
Jointly with the initial net distribution a household survey of all 294 participating households was carried out in December 2000 by experienced field staff following one day of training on the specific questionnaire. In addition to the number of persons and children living in the household the pre-coded questionnaire included information on education and occupation of head of household, physical condition of the house (roof, windows, wall materials, and eaves), family assets and ownership of animals and land.
Between 11 and 15 days after the net distribution a random sample of 116 households was re-visited and a short questionnaire designed to capture any adverse effects of the insecticide on the net users, their duration and severity.
Throughout the study net surveys were carried out among households to collect data from all active study nets on net use (who and frequency of use), perception of net effect, washing habits (method of washing, type of water and soap used) and number of washes since last survey. The physical condition of each study net was assessed with respect to where the net was found (hanging or not) and number, size and position of holes. Three categories of hole sizes were used: up to the diameter or length of a coin, hand width or larger than a hand width. To mark the position of a hole the net was divided into numbered areas (upper and lower part of each side) and the code for the hole's location entered into the questionnaire. A detailed list of the time intervals of the surveys and the number of nets seen is given in the annex [see Additional file
1]. A total of 13 surveys were carried out for phase one nets and seven for phase two nets. For logistic and operational reasons the frequency of surveys per year reduced over time from four in the first, three each in the second and third and two each in the fourth and fifth. During the course of the study the questionnaire for net follow-up was simplified as the information obtained did not change over time. The first change was made in April 2002 (survey 5) when assessment of the hole's position and the information on who used the net were ended and the second in October 2004 (survey 11) when questions on the method of washing and net perception were ended.
Sampling and sample preparation
For each time point a random sample of 40 nets per net type were selected from the net master list together with 2 possible replacements in case the selected households could not be reached or the net had been lost to follow-up since the last visit. Nets were then collected by the field team and each household received a LLIN as a replacement to insure continuous protection of the family. These replacements nets, however, did not have ID numbers and were not included in net follow-up surveys. Details of collected samples at various time points are presented in the annex [see Additional file
2].
Using templates of 30 × 30 cm (bioassay) and 10 × 10 cm (chemical residue) samples were cut out of the nets, marked with the date and the net ID number, packed in aluminium foil and stored at room temperature until transport to the respective laboratories for analysis. Samples for bioassay and chemical residue were always taken from the same spot (long side near the ID number mid-way between top and bottom) immediately next to each other. Generally one sample was taken per net per time point with the following exceptions in order to allow assessment of intra- and inter-net variability. At baseline and 6 months of the first phase a second sample was taken from the short side of the net for bioassay and chemical residue. These locations were termed sites one (standard sample) and two respectively. At base line of the first phase two additional samples for chemical residue were taken from each site immediately next to each other and these were termed position one and two respectively. Therefore, there were 2 samples (site 1 and 2) per net for bioassay at baseline and 6 months while for chemical residue there were 4 samples at baseline (position 1 and 2 each for site 1 and 2) and 2 at 6 months (site 1 and 2). For the second phase additional samples were only taken at baseline and only for chemical residue (site 1 and 2).
After the first samples had been taken the ten baseline nets for the first generation LLIN were kept outside the package exposed to air and dust but not sun and were not used or washed. Samples from these nets were taken 11, 27, 39 and 60 months after the unpacking and sent for chemical and bioassay analysis.
Chemical residue
All samples were analysed at the laboratory of the Pesticides Research Department of the Walloon Agricultural Research Centre in Gembloux, Belgium (WHO Collaborating Centre) using the MEREPERMA methodology which has been ISO accredited (ISO 17025). Surface area and weight of each 10 × 10 cm sample was measured and the sample then introduced into a 100 mL Erlenmeyer flask. Deltamethrin was extracted from the sample by heating under reflux for 60 minutes with 40 mL xylene. After cooling to ambient temperature the extract was quantitatively transferred into a 50 mL volumetric flask. The flask was filled up to volume with xylene. A 10 times dilution was achieved in xylene. The final extract was then analysed for determination of deltamethrin by Capillary Gas Chromatography with 63Ni Electron Capture Detection (GC-ECD) using an external standard calibration. For each sample two chromatographic injections were performed and the mean reported as g/kg deltamethrin and then transformed to mg/m2 based on the surface area of that sample. The analytical method was validated for the determination of deltamethrin residues in conventionally and long lasting treated nets. Specificity, repeatability (precision), linearity of the detector response, recoveries (accuracy) and limit of quantification were determined. The accuracy of the method was determined concurrently with the analysis of samples from 2001 to 2005 by spiking untreated mosquito net samples (which had already been extracted) with know amounts of deltamethrin. The mean recovery varied between 95% and 101 % depending on level of deltamethrin concentration (n = 242) with a Relative Standard Deviation (RSD) between 7% and 11% for fortification levels ranging from 0.3 mg/m2 to 100 mg/m2. The acceptable limit is 90–110 % with a RSD < 15 %. Therefore, the accuracy and precision of the analytical method was found to be excellent.
A total of 115 samples from both first and second phase of the study which had been taken immediately next to the sample for GC-ECD analysis were also analysed at the Vestergaard-Frandsen Quality Control Laboratories in Hanoi, Vietnam using a method where insecticide determination is done by normal phase High Performance Liquid Chromatography with UV Diode Array Detection (HPLC-DAD) using an internal standard. The principle of this method was proposed in 2006 for adoption by the Collaborative International Pesticides Analytical Council (CIPAC). In brief, net samples are cut into small pieces of < 2 × 2 cm and deltamethrin is extracted into solution using a mixture of solvents iso-octan plus 1,4 dioxan with 0.15% HPLC grade water (80/20, v/v). Dibutyl phthalate is added as the internal standard. The extraction bottle is sonicated in a water bath set at 80°C and then shaken vigorously for at least 15 minutes. A proper volume of solution is filtered through 0.45 micrometer membrane syringe filter into a vial. A volume of 5 μL of filtered solution is injected into a normal phase isocratic HPLC equipped with PDA/UV detector and deltamethrin is quantified using an internal standard calibration curve. The method was shown to be suitable for deltamethrin with repeatability (same net sample) of 1.8% (RSD, n = 7), reproducibility (multiple samples over time) of 11.6% (RSD) and recovery of 99.7% (95% CI 98.6%–101.6%) from samples in which deltamethrin content was added at an exact amount by weighting method (n = 5).
Bioassays
Bioassays for the first study phase were carried out by the Laboratoire de Lutte contre les Insectes Nuisible, Montpellier, France (WHO Collaborating Centre) using WHO standardized procedures [
20]. For the tests 2–4 day old, unfed female
Anopheles gambiae s.s.(Kisumu strain) and
Culex quinquefasciatus (S-lab strain) were used, the latter were only tested for the first 12 months. Both species have been well established in culture and are known to be pyrethroid sensitive. The tests were conducted using the standard WHO plastic cones and a three minute exposure time. Five mosquitoes were introduced into cones at a time. Tests were carried out at 25°C ± 2 under subdued light. After exposure, females were grouped into batches of 10 or 20 in 200 mL plastic cups and maintained at 28°C ± 2 and 80% ± 10% relative humidity with honey solution provided. For each sample tested, a total of 50 mosquitoes were used (Inter Quartile Range 50–51, range 40–62, ten cones). Proportion of mosquitoes knocked down at 60 minutes (KD60) was calculated. Percentage mortalities were recorded after 24 h.
Bioassays for the second phase of the study were carried out at the entomology laboratories at Centers for Disease Control (CDC), Atlanta, USA (WHO Collaborating Centre). Tests were carried out with Anopheles gambiae s.s. Exposure time and method were identical, however, only 40 mosquitoes were used per test.
For all bioassays unexposed controls were run to validate the tests results and results were discarded if mortality among control was > 5%.
The definitions of effectiveness of nets based on bioassay results followed recommendations by WHO (Pierre Guillet, personal communication) and were as follows:
Minimal effectiveness: KD60 ≥ 75% or functional mortality ≥ 50%
Optimal effectiveness: KD60 ≥ 95% or functional mortality ≥ 80%
Data analysis
Data were entered using EpiInfo 6.04 software (WHO/CDC, 1997) and then transferred to Stata 8.2 (Stata Corporation, Texas, USA, 2005) for further data management and analysis.
From the household data a socio-economic index was calculated using principal component analysis considering education, ratio household members per bed, physical condition of house, assets (radio, vehicles), animals and land possession. Only the first component was used to build the index. Households were then divided into wealth quintiles for further analysis.
After appropriate data preparation cumulative washes per net were calculated until the net was censured, i.e. sampled or otherwise lost to follow-up. Similarly, a hole index was calculated for each net and time point which was constructed by multiplying the number of holes with the hole category (1–3 and increasing with size) and then calculating a mean over all nets in the sample, including those with no holes.
For the analysis of net performance (chemical residue, bioassay) the mean of all samples per net was calculated. For the expression of the central tendency of sample measurements of all outcome variables mean, geometric mean or median was chosen after evaluation of the distribution of values within the sample.
For the assessment of between-net variability of insecticide concentration the standard deviation of deltamethrin residue was expressed as percent of the sample mean (coefficient of variation). Intra-net variability was expressed as the difference of samples of the same net to the mean expressed as percent of the mean and then averaged over the sample. For the analysis of statistical differences of inter-intra net variations of outcomes ANOVA was used.
Statistical analysis was generally done in two steps, first univariate analysis was carried out considering all co-variables of interest and tested using Chi-squared test for categorical and t-test or Kruskal-Wallis test for continuous variables depending on the validity of the assumption of normal distribution of values. In a second step multivariate analysis (linear or logistic regression models as appropriate) was used to verify any associations found in the univariate analysis.
Ethical considerations and approval
This study was conducted according to the principles of the Declaration of Helsinki and the international guidelines of biomedical research involving human subjects. It was reviewed and approved by the Ministry of Health, Uganda, WHO Roll Back Malaria Project, Geneva and Gesellschaft für Technische Zusammenarbeit, GTZ, Germany.
Discussion
Two possible approaches can be taken to study the field performance of a LLIN product under "real life" conditions. First, a cohort design can be used where a number of nets are randomly distributed to households and repeated measurements taken from the same net over time until a defined endpoint is reached. This design, which was applied by Lindblade and colleagues in Kenya [
12], has the advantage that it follows the same net over time but the disadvantage that invasive measurements that require cutting of samples are limited to time of failure and bioassays have to be done on-site requiring an insectary within the study area. The second design option uses multiple random samples of nets with only one measurement per net. This approach which needs a larger number of nets but has no restrictions with respect to number of samples cut from each net for chemical and bioassay tests, is currently recommended by WHOPES [
18] and has been applied in this study. By selecting the first generation PermaNet
® for the study randomly from a large batch of LLIN in the warehouse and distributing them randomly among households each random, cross-sectional sample of nets is representative of the original shipment of 10,000 nets to Uganda. In case of the second generation nets there was no large shipment to select from so that each time point represents the overall shipment from the factory. With the exception of the baseline nets only one sample per net was taken in a standardized fashion, i.e. from the same location on the net. This implies that the mean for all sampled nets per time point can be interpreted as a valid estimate of chemical residue, vector mortality or knockdown rate, while the result of an individual net sample has to be interpreted with caution as it will depend on the level of within-net variation of the insecticide and can not be taken as the true average of that net. To obtain an reliable estimate for a specific net a composite sample of at least 5 locations on the net has to be taken as described in the WHO specifications for deltamethrin LN [
21] or better 12–14 as is practiced by the VF quality control laboratories (Phan, personal communication).
In the first generation PermaNet
® an unexpected drop in performance was observed after only 6 months at which time the rates for the outcome variables were not significantly different from conventionally treated ITN. However, thereafter chemical residue as well as bioassay results remained quite stable over more than 20 months with 15–45% of nets still showing optimal effectiveness. Even after 39 months of field use 5% of the net samples still had more than 4 mg/m
2 deltamethrin and 18% showed either a mosquito knockdown rate of ≥ 75% or mortality of ≥ 50% (minimal effectiveness). While this is not sufficient to fulfil WHOPES phase III criteria [
18], it is clearly more than what can be expected from conventionally treated nets which have been reported to reach good performance up to 15 months [
22] but never up to 27 months. The cause for the observed pattern of performance can not be explained by any of the variables in the data set, however, some potential causes can be excluded: excessive washing, use of more aggressive (alkaline) soaps and also the impact of sun light as none of the samples showed the R-isomer of deltamethrin typical for UV exposure (data not shown). The fact that these nets did not show any loss of activity even after 5 years when they were not used or washed (Table
5) and the bimodal distribution of results at the time points between 6 and 27 months suggest that a certain proportion of nets did loose the coating with the insecticide quickly under the stress of every day use and/or washing but the remaining 20% to 30% of nets continued to perform as LLIN at least up to 27 months and some up to 39 months. This explanation would also be consistent with the statement of the manufacturer that until early 2001 some problems in the manufacturing process existed that could have resulted in parts of a production batch having a lower quality and that between June 2000 and August 2002 continuous improvements on the production quality had been implemented (VF personal communication). This also implies that published data on the performance of the first generation PermaNet
® has to be seen in relation to the time of manufacture of tested nets. The first report on field evaluated PermaNet
® comes from Burkina Faso [
10] where after 12 months mean delamethrin (n = 11) was 3.7 mg/m
2 with a mortality of 54% and after 18 months (n = 5) 1.6 mg/m
2and 7% respectively. According to Kroeger and colleagues [
23] the nets used in that study were an early version of the product manufactured in the first two months of 2000 while the PermaNet
® tested in Columbia by Kroeger and his team [
9,
20] were produced in the second part of 2000 at about the same time as the first generation nets used in this study. In the Columbia study six PermaNet
® were washed 20 times by local women during five months and then used by these families for another 2.5 years. After three years mean deltamethrin content (n = 4) was 9.6 mg/m
2 and mortality 88% [
23]. Asidi
et al. [
24] tested one PermaNet
® in late 2000 after five washes using an experimental hut design and found no difference to conventionally treated nets. Similarly, Graham
et al [
13] reports on five wash and experimental hut trials of first generation PermaNet
® in three countries with nets being delivered between January 2000 and April 2002 and found mortality on the LLIN after 15 to 21 washes above 90% but median time to knockdown and blood feeding inhibition only significantly better than conventionally treated nets in one trial. In contrast, Gimnig and co workers [
25] had good results in a field trial in Malawi using nets produced late 2001 with mortality 42.5% after 24 months of field use (n = 25) and 60% of the first generation PermaNet
® still having minimal effectiveness, very similar to results reported in this study. This varying performance seems to be in keeping with our hypothesis of inconsistent production quality of the nets between early 2000 and late 2001 with some batches or part of batches being better than others which in conjunction with the small sample sizes of the quoted studies would explain the variations in results. In contrast, tests on first generation PermaNet
® from first half of 2002 – just before the finalization of the second generation PermaNet
® (VF personal communication) – tested in a field trial in Kenya showed excellent performance after two years with mean mortality of 72% and 82% of the nets functional with mortality rates at all times greater than 50% [
12]. These same nets washed in the laboratory 20 times under standard WHO conditions still showed 55% mortality, 72% knockdown and a chemical residue of 18.5 mg/m
2, very close to what was found for the second generation PermaNet
® in this study.
To date there are only two studies published on the second generation PermaNet
® which was officially launched as PermaNet 2.0
® in April 2003. Both, however, were not phase III field trials but rather phase I/II washing trials. In the first study in Pakistan [
13] PermaNet 2.0
® performed significantly better after 20 washes in the field than a conventional ITN with mortality 81.8%, knockdown 79.5% and a deltamenthrin residue of 24.1 mg/m
2 when net swatches were washed and 13.1 mg/m
2when the whole net was washed. After 30 washes mortality dropped to 43.2% but knockdown rate remained high with 77.3%. Mean deltamethrin content at this time was 5.8 mg/m
2. In the second study second generation PermaNet
® were used as a control in the laboratory study of another product [
26]. Here a mortality rate of 100% was found after 30 washes with 18.7 mg/m
2 deltamethrin remaining on the net after these washes. Although these results are not directly comparable they indicate already the strong performance of the improved product. This is very similar to results from this study where only the 24 months sample showed slightly reduced figures for mortality (73.9%) and knockdown (92.4%), with a resulting proportion of nets with optimal effectiveness of 71.0 % (WHOPES phase III criteria 80%). However, the confidence interval for this sample did include the 80% mark and three of the nets had been washed immediately before collection which might have resulted in the poorer performance in the bioassay. Also, the deltamethrin content decline showed a continuous, more or less linear decline with no unusual drop at 24 months supporting the hypothesis that this lower value was a statistical outlier rather than a systematic decline in performance. At 36 months 90.0% (95% CI 76.3–97.2) of the second generation PermaNet
® had either mortality ≥ 80% or knockdown ≥ 95% and therefore fulfilled the WHOPES criteria for phase III LLIN evaluation [
18]. The nets tested in this study were produced in July 2002, a few weeks before the finalization of the production details for the PermaNet 2.0
® brand but according to the manufacturer did not differ significantly from that product (VF personal communication).
The declared deltamethrin loading dose for the first generation PermaNet
® was 50 mg/m
2 and a mean of 43.6 mg/m
2 (95% CI 36.0–51.3) was found which compares well with other reported results of 44.9 [
12], 47.1 [
10], and 48.3 mg/m
2 [
11]. In contrast, the baseline deltamethrin content for the second generation PermaNet
® was rather high with a mean of 67.1 mg/m
2 (95% CI 60.0–74.4), above the declared 55 mg/m
2but still within the allowed upper limit of 68.7 mg/m
2stated in the WHO specifications for deltamethrin on long-lasting (coated) insecticidal nets [
21]. When the content was expressed in g/kg, the units of the actual measurement, the baseline value for the second generation PermaNet
® was 2.1 g/kg (95% CI 1.9–2.4) which is closer to the expected value of 1.8 g/kg and clearly within the required limits of 1.35–2.25 g/kg. This difference between mg/m
2 and g/kg results indicates that the actual, measured mass of net per unit was slightly higher than the 30 g/m
2 stated in the specifications and, in fact, it was found to be 31.7 g/m
2 from measurements taken according to the relevant ISO norm 3801 [
27]. The range of deltamethrin content on the baseline nets was 52.5–80.4 mg/m
2 or 1.5–2.5 g/kg but since only two samples were taken per net this is a reflection of within net variability and – as outlined above – can not be interpreted as mean insecticide level for those nets. Other studies have also found rather high baseline insecticide values for the second generation PermaNet
®. Yates et al. [
26] using HPLC found a mean baseline deltamethrin concentration of 66.7 mg/m
2 and Graham
et al [
13] 86.3 mg/m
2 when they tested net swatches before washing but 55.3 mg/m
2 on the samples from nets prepared for washing.
Interestingly, a systematic difference was observed between results from the analytical methods used by the WHO collaboration centre (GC-ECD) and the VF quality control laboratories (HPLC-DAD, CIPAC) with the latter systematically lower than the former (Figure
5) and the difference reaching 0.30 g/kg at the 2.0 g/kg level. This would imply that a mean deltamethrin content of 2.1 g/kg found in the gas chromatography for the second generation PermaNet
® at baseline would correspond to 1.8 g/kg or 55 mg/m
2 in the methodology used by VF. It is concluded, therefore, that the baseline deltamethrin content of second generation PermaNet
® in this study was within the specified target dose with either method of determination. Furthermore, even if the initial deltamethrin loading had been slightly lower the result in the three year follow-up would not be significantly different as at 36 months on average well above 15 mg/m
2deltamethrin would have remained on the nets which was shown to correlate with high bioassay results.
The difference between the analytical procedures was quite significant at higher deltamethrin content and needs further evaluation as both methods are frequently used in the testing of LLIN. It is not likely that the difference was caused by the fact that samples were not exactly identical although taken immediately next to each other on the net. Such a difference by intra-net variability of insecticide would be expected to go in both directions and would average very close to zero. A direct comparison of the two protocols, HPLC-DAD according to #333 CIPAC and GC-ECD according to ISO 17025 at the WHO Collaborating centre in Gembloux, Belgium (n = 11, two measurements per sample) by independent technicians showed almost the same linear relationship and systematic error at higher levels of insecticide as in our reported data with 2.0 g/kg in GC-ECD corresponding to 1.75 g/kg in HPLC-DAD (Pigeon, unpublished data). This means that the difference is also not caused by the execution of the protocol by either laboratory in our study but is a true, systematic difference in the methodologies. However, it is at this point not clear at which part of the protocol the deviation occurs: sample preparation, extraction or determination of insecticide content. Both analytical methods have been validated using samples with known deltamethrin content and shown to have a very high recovery rates of 99.7% (95% CI 98.6–101.6) in the case of the HPLC-DAD (CIPAC) method (Phan, unpublished data) and 98.2% (97.1–99.4) for the GC-ECD method (Pigeon, unpublished data). However, at a deltamethrin loading of more than 30 mg/m2 recovery in the GC-ECD method was 101.2% (99.8–102.6). Repeated re-extracting of those samples consistently failed to show any remaining deltamethrin indicating that all insecticide was captured. The higher values measured in GC-ECD at high deltamethrin levels could possibly be explained by slight dilution variability. On the other hand, the sample preparation could also contribute to the systematic difference as in the HPLC-DAD (CIPAC) methodology the 10 × 10 cm sample is cut up into many small pieces before extraction which could lead to a mechanical loss of insecticide. The GC-ECD protocol is the more universally applicable method for insecticide determination in LLIN as it can be used for LLIN with coating as well as incorporation technology and is able to reliably detect even very small amounts of insecticide. The HPLC-DAD (CIPAC) protocol has the limitation that it can only be done with LLIN samples using coating technology but is very suitable for assessment of baseline loading dose of the LLIN and is, therefore, used by manufacturers for quality control purposes. As both methods will continue to play a significant role in the testing of new LLIN products the systematic difference described here clearly needs further study.
The initial batch of ITN also lost insecticide very quickly with a median deltamethrin content of only 0.7 mg/m
2 after six months of use and a vector mortality and knockdown rate of 8.5% and 31.6% respectively. Although reported results from deltamethrin treated conventional ITN after washing and/or field use vary considerably [
11‐
13,
23,
25,
28], this appears lower than expected. However, when the same nets were re-treated after 15 months they showed a chemical residue of 3.1 mg/m
2, mortality of 40.8% and knockdown of 76.9% six months later and there was no difference to new polyester nets treated at the same time. These results are very similar to what was found by Gimnig in Malawi [
25]. A possible explanation for the initial rapid loss could be the presence of some warping oil on the nets at the time of dipping or a poor quality of polyester that did not allow the insecticide to attach to the fiber. A second group of re-treated conventional nets had even better results with deltemethrin of 1.4 mg/m
2, mortality 44.8% and knockdown 59.4% after 12 months. One possible explanation is a cumulative effect with insecticide remaining from the previous treatment [
29] or a slightly better persistence of the deltamethrin tablets compared to the liquid formulation [
28]. Nonetheless, these results are within the reported limits of performance as mortality of > 70% has been described with deltamethrin treated nets even after 15 months of fields use [
22] and deltamethrin concentrations as low as 1.5 mg/m
2 [
28].
Nets in our study were used regularly and washed mainly with cold water in a basin and without rubbing on stones, similar as has been described from Tanzania by Erlanger and colleagues [
30]. With very few exceptions locally made soap bars were used rather than industrial detergents. The alkalinity of this soap was found to be moderate with ph 9–10 which has also been found in other studies [
13,
30] and can be considered favourable with respect to the potential to degrade deltamethrin compared to the industrial detergents. The washing frequency in our study was low with initially 2.2 washes per year and then declining in phase two to 1.8 washes per year. This is significantly lower than the 1.0 wash per month reported in Uganda from a descriptive household interview study [
31]. However, as Miller
et al have demonstrated in Tanzania reported washing does not correlate well with actually observed washes [
32] the latter being significantly lower than the former. The washing frequency in our study areas was not that much different from some of the rates reported from other countries: Gambia 1.9 washes/year [
28], Columbia 1.2 [
23]. Other studies reported washing rates between 3.6 and 5.6 per year [
5,
12,
30] but none in the range of 12 washes per year. It is difficult to say to which extent results on longevity of insecticidal effects observed in this study would have been different if the washing frequency would have been higher. But it might be useful for the WHOPES evaluation process to require studies in settings with differing washing habits.
In the washing study of PermaNet 2.0
® by Graham and colleagues [
13] in Pakistan 72.1% of the baseline insecticide had been lost after 20 washes. This is slightly more than the 58.2% loss found in this study for second generation LLIN after three years. On the other hand, the loss was 59% after 20 washes in the laboratory washing study by Yates at al. [
26]. This indicates that the loss observed in this study over three years would be equivalent to somewhere between 15 and 20 washes without field use. However, the mean number of washes actually observed for our tested nets was 4.5 with a range of 0–9. This suggests that under every day use in the field washing alone is not the only determinant of insecticide loss. This finding was further explored using a linear regression model with number of washes and time of field use as key variables and found that, indeed, more insecticide (75.6 %) was lost through every day use than by washing (24.4 %). It is not exactly clear whether physical handling of the nets or environmental factors or a mixture of these causes the loss and this issue will require further study. It is, however, quite clear from our results that at least for LLIN using coating technology, maximum number of washes obtained for a LLIN product in the laboratory without loss of functionality can not be translated into the same number of washes under field conditions. This emphasizes the need for phase III trials under field conditions in the process of WHOPES evaluation. The situation may be different for LLIN using incorporation technology [
33].
A highly significant correlation was found between the chemical residue and bioassay results for samples taken from the same location on the net. A very similar relationship has been described by Yates and colleagues between chemical residue and median time to knockdown [
26]. Adams et al. [
34], on the other hand, did not find a statistically significant correlation between insecticide content and bioassay results possibly because none of the insecticide levels were below 3 mg/m
2 and most corresponding mortality values above 90% and all above 50%. At least for
Anopheles gambiae s.s. deltamethrin levels as low as 1–3 mg/m
2 can produce knockdown rates between 75% and 95% and mortality above 50%, something that several authors have described previously [
23,
27,
29,
30,
34].
Culex quinquefasciatus responded less sensitive to deltamethrin with bioassay results rapidly declining at insecticide levels below 15–20 mg/m
2. This is thought to be due to a generally higher tolerance of this species to tarsal contact insecticides [
22].
Based on a total of 578 observations, it could be established that using a cut-off of ≥ 4 mg/m
2 and ≥ 15 mg/m
2 deltamethrin on a net sample can distinguish nets with minimal and optimal performance in the bioassay with
An. gambiae with a positive predictive value of > 95%. Sensitivity was rather low and this test, therefore, not suitable to identify an indivudual net that fails the effectiveness standards. But it would be possible to use chemical residue as a proxy to identify nets that continue to provide protection in situations where taking of large samples for bioassays is not possible. One must keep in mind, however, that this test would then need to assume that the insecticide found on or in the fibre also is bio-available to the mosquito which may not always be the case, particularly in LLIN that use the incorporation technology [
11,
12].
When investigating the longevity of protection one can expect from a LLIN product under programmatic conditions the duration of the insecticidal effect is only one part of the equation. The duration of the physical integrity of the netting material and the resulting useful life is equally important and in contrast to the former, there is very little published literature on the latter. Ritmeijer et al. [
35] report 65% of fine-mesh, 75 denier polyester nets with any holes after two years in a leishmaniasis programme in Sudan and Spencer and co-workers [
36] found 78% of 75 denier polyester nets with any holes 12–15 months after distribution in a refugee camp in western Uganda. Both these findings are within the results from this study (Figure
2). In a cross-sectional survey in Tanzania [
30] 86% of polyester nets were found with any holes but the age range of the nets was 0.5–5 years and the fibre strength ranged from 40 to 100 denier.
Reports on the severity of damage are difficult to compare as each study used a different definition. They ranged from 45% severely damaged (> 7 holes larger 2 cm) in the cross-sectional net survey [
30] to 33.2% with > 5 holes after two years in Sudan [
35] to 28% with at least one hole of 40 cm
2 after 12–15 months in Uganda [
36]. In order to provide a more standardized and continuous measure of the physical net condition a hole index was used which was constructed in analogy to the classical spleen index in malariology which combines the frequency of any hole with the size of these holes divided into three categories. It was demonstrated that the physical condition of the 75 denier polyester nets linearly deteriorated in the first 3 years but decay then seemed to accelerate somewhat with evidence that severely torn nets with a hole index of 30 or more were significantly more likely to be stored away rather than used.
Interestingly, the rate of hole acquisition (proportion with any hole) over time was slower in the second phase of the study after families had used nets already for two years. This trend continued in the current third phase of LLIN testing in the very same families where after 12 months only 34% of nets had any holes compared to 50% in phase two and 71% in phase one and after 18 months the figures were 44%, 64% and 80% respectively. This strongly suggests that net users learn to handle the net better or more carefully with increasing user experience and emphasizes the importance of a communication campaign to support the development of such a net culture.
While it is reasonably easy to monitor the physical condition of a net, it is very difficult to infer from these findings on the average useful life of a net under "real life" conditions as the continued use of a torn net will depend on many factors such as availability of a replacement, presence of other nets in the household that can be shared or perceived pressure to keep the net in a study or project setting. In this study 83.5% of the originally distributed polyester nets were still present after five years when those nets sampled or replaced at earlier times were not considered [see additional file
2]. This is similar to what has been reported for polyethylene nets in Tanzania where seven years after distribution 97% of households that received one or more nets still had at least one net in spite of the observation that 93% of the nets had any holes and 55% had 6–15 holes of 2 cm or more which would be equivalent to a hole index of 15–45 [
33]. Although these polyethylene nets (equivalent to ~180 denier) are likely to be stronger than a 75 denier polyester net neither observation is likely to be equivalent to a realistic average useful life estimate due to the study/project situation. A household survey design may be a better tool to approach the useful life of nets and based on one such study is likely to average around 3 years for polyester nets of medium quality [
37].