Background
Malaria mortality and morbidity have declined significantly in many parts of the world in the last 2 decades [
1]. Despite this achievement, the World Health Organization estimated that there were still 438,000 deaths due to malaria in 2015, and that ~70 % of deaths occurred in children less than 5 years old [
1]. Recently, Bhatt et al. estimated that 663 million clinical cases have been averted by malaria interventions from 2000 to 2015 in sub-Saharan Africa, largely as a result of insecticide-treated bed nets and indoor residual spraying [
2]. However, the emergence of resistance in
Anopheles vectors to current insecticides, and shifting behavioural patterns among vector groups are a great concern for future malaria control [
3]. In addition, the spread of resistance to artemisinin-based combination therapy among Southeast Asian parasite strains have been reported [
4]. Novel interventions are, therefore, important extensions to the current range of control tools, and may be necessary to achieve elimination in many currently endemic areas [
5]. Transmission-blocking vaccines that interrupt human-to-mosquito transmission by targeting the sexual, sporogonic, or mosquito stages of the parasite are called SSM-VIMT. SSM-VIMT have the potential to reduce malaria transmission from humans to mosquitoes for whole populations, and could be an important supplement to traditional controls in countries striving for malaria elimination [
6‐
8].
SSM-VIMT are designed to elicit anti-parasite or anti-mosquito antibodies in vaccinees, and the antibodies block parasite development in the mosquito vector when ingested with gametocytes: the sexual-stage, transmissible form of the malaria parasite. While several different assays can be applied for SSM-VIMT development [
9], the standard membrane-feeding assay (SMFA) is considered the “gold standard” for determining the impact of test factors on gametocyte infectivity to mosquitoes (either measured by a reduction in oocyst intensity or in prevalence of infected mosquitoes). The SMFA has broad utility, and has been employed to evaluate the functionality of vaccine or whole-parasite induced antibodies in animal studies and human clinical trials [
7,
8,
10‐
12], as well as antibodies induced by natural exposure to malaria infection in endemic settings [
13‐
16]. Furthermore, increasing interest in transmission-blocking drugs has made the SMFA a useful assay for malaria drug development [
17‐
19].
While there are variations in SMFA methodology among different investigators, the assay is generally conducted by feeding a blood meal containing a mixture of cultured
Plasmodium falciparum gametocytes and test (or control) antibodies to
Anopheles mosquitoes through a membrane-feeding apparatus. Approximately 1 week later mosquitoes from the test and control groups are examined to enumerate the oocyst-forms of parasites that, if present, can be visualized in the epithelium of the mosquito’s midgut by mercury-bromide staining. A recent study qualified SMFA following the International Conference on Harmonisation (ICH) Harmonised Tripartite Guideline Q2(R1) using an anti-Pfs25 monoclonal antibody (mAb) with a single protocol [
20]. The study concluded that the range (the levels of transmission-blocking activity in which the analytical procedure has a suitable level of precision and linearity) of SMFA performed with their method was when there was more than ~80 % inhibition in oocyst intensity. However, there have been no direct studies, which assess inter-laboratory variation in % inhibition in SMFA.
Modern vaccine development relies heavily on collaborative research efforts and product development partnerships that involve multiple laboratories around the world [
21]. Though highly standardized SMFAs have been performed in separate facilities, there is no assurance that SMFA data derived from experiments performed by different investigators are comparable. To enable such comparison, SMFA performance using the same test antibodies was assessed at two laboratories, TropIQ Health Sciences (TropIQ, using methodologies adapted from Radboud University Medical Center [Radboudumc], Nijmegen, The Netherlands) and the Laboratory of Malaria and Vector Research (LMVR, USA). Variation between the laboratories was assessed using a mouse mAb, a rat mAb, and human polyclonal antibody (pAb), all tested at several concentrations in two or three independent assays. When conducted under controlled conditions in different laboratories, the SMFA can provide informative comparable data for SSM-VIMT development.
Discussion
This is the first study to evaluate inter-laboratory variation of the SMFA, the “gold standard” assay for
Plasmodium gametocyte infectivity, using a range of transmission reducing test antibodies (mouse and rat mAbs, and human pAb). In this study, regardless of antibody type or the targets recognized by the antibodies tested, inter-assay and inter-laboratory variation decreased with increasing inhibition of oocyst formation. Conversely, variation became larger in both laboratories with decreasing percent inhibition. These results are in line with previous observations that precision of the SMFA increases with higher percentage reduction, and that levels of inhibition below 80 % need to be interpreted with caution when low numbers of replicate data are available [
29]. Across the range of antibodies and concentration tested, inter-laboratory variation did not appear to be greater than the inter-assay variation observed within a laboratory. However, inter-assay variations were larger at lower levels of inhibition, therefore, it is possible that inter-laboratory variations could not be detected, if any, in the lower range. Since the error in %TRA estimate changes gradually (larger error with lower %TRA) as expected from the zero-inflated negative binomial model [
20] and shown in this study, it may be difficult to establish a specific level of %TRA at which the inter-assay or inter-laboratory variations are acceptable in all situations. For example, in assessing samples from a clinical study it may be justifiable to set up more stringent criteria than that in a novel candidate discovery study. However, this study showed that at antibody concentrations that led to a >80 % reduction in oocyst numbers, the differences in best estimates of %TRA from multiple feeds between the two laboratories were less than 5 % points.
The SMFA is critical to SSM-VIMT development and there are preceding studies that provide several guidances to optimize intra-laboratory precision when performing SMFA. Medley et al. suggested that at least 50 mosquitoes (ideally 100) need to be dissected to accurately assess transmission-blocking activity [
28]. Another study by van der Kolk et al. concluded that if all samples are not tested in a single experiment, data from different assays should be compared only when the mean intensity in the control is at least 35 oocysts per mosquito [
13]. Churcher et al. simulated how many mosquitoes were required for dissection per feed to ensure that a reported percent inhibition value had within 10 % error from the true efficacy [
29]. For example, if the mean oocyst number in the control is 5, it is estimated that ~200 mosquitoes need to be dissected to accurately report 80 % inhibition, whereas if the mean oocyst number is 100, approximately 50 mosquitoes need to be dissected.
At the present time, it is practically challenging and resource intensive to only regard SMFA experiments with a mean control oocyst intensity of 35 oocysts per mosquito as valid. In terms of the mosquito supply and maintenance, amount of test material required, and the labor intensiveness of dissection, it is also impractical to routinely perform the SMFA with 200 mosquitoes per COM (to be ready if the mean oocysts number in the control were to be 5 oocysts), To some degree, these difficulties can be overcome using one of several modified methods recently published [
19,
25,
30], i.e. higher throughput SMFA using transgenic parasites that allow (semi-) automated detection of infection and may eliminate the need for mosquito dissection. However, the modified assays are not necessarily applied to all laboratories at present. Therefore, the comparison study was conducted using the SMFA methods usually performed in many laboratories; i.e. by dissection of 20–30 mosquitoes per feed. With these study conditions, SMFA showed high precision only at higher levels of inhibition. The modifications of the assay discussed above (e.g., dissecting 200 mosquitoes per COM, only utilizing SMFA data when mean control oocyst intensity is >35) should expand the range of % inhibition where the precision of the assay is acceptable. However, application of such modifications for every single assay is impractical at present. The aim of this study was to evaluate the inter-assay and inter-laboratory variations using methods which can be applied in many laboratories routinely. The minor differences between the two laboratories in methods did not cause any measurable “laboratory” effect on percent inhibitions. The present work stems from observations from a preliminary study where results with 4B7 mAb from two additional laboratories were compared. While there was an unfortunate loss in stability of the antibody during preparation and shipment, the results showed similar trends in percent inhibition between the labs despite several differences in methodology (Yimin Wu, Rhoel Dinglasan, personal communication). The approach used here will be useful to determine whether any difference in SMFA procedure affects SMFA readouts.
Because %TRA plateau at low and high concentrations of the test antibody (which gives 0 %TRA and 100 %TRA, respectively), the SMFA cannot discriminate between doses that fall within these upper or lower plateaus. The results of this study indicate that precision of the SMFA is very high at (near-) saturated antibody concentrations, where the sensitivity to changes in antibody concentrations is low. To reveal any differences in assay sensitivity between the two laboratories, IC
50 values observed for the rat 85RF45.1 and human polyclonal samples were compared. For all antibodies tested here, the dose–effect relationship fitted well to a Hill equation, concordant with the notion that the interaction between an antibody and a ligand can be described by equilibrium kinetics [
31]. The differences in IC
50 values between laboratories were not larger than the inter-assay differences observed within a single laboratory. The SMFA data from this study also fitted well with a linear regression when doses and %TRA were transformed appropriately [
11]. While the dose effect was significant, the test site did not significantly influence the outcome. Compared to analyses that evaluate single doses, the full dose response analyses takes into account all data points. This increase in number of observations is likely to increase precision of the measure of inhibition. Moreover, an understanding of the dose–effect relationship will guide predictions of the minimally required titer to achieve the desired reduction in transmission in human SSM-VIMT studies and campaigns. However, to demine the dose–effect relationship, each test sample needs to be tested at multiple concentrations. Therefore, the best balance between throughput of assay and precision/sensitivity needs to be optimized on a per study basis.
This study showed large variation in baseline oocyst intensities between different feeds (ranging from 4 to 62 mean oocysts in this study). Standardization of the average number of oocysts in assay controls would benefit from a gametocyte fertility marker, which is lacking at the moment. Nevertheless, the data indicate that the relative reduction in oocyst intensity (%TRA) can be measured reproducibly at higher levels of inhibition when oocyst intensity in the controls differed between 4 and 62. A recent study conducted at LMVR (repeat assays with 4B7 mAb and simulations using a zero-inflated negative binomial model) also supports that %TRA is independent from control oocyst intensity in SMFA (mean control oocyst intensity in the study ranged from 0.1 to 73.7) [
26]. This study reconfirms that an inter-assay variability in the SMFA decreases with increasing percentage inhibition and that variability was demonstrated to be similar between different laboratories. While a weaker inhibitory activity reported from any laboratory needs to be interpreted carefully (unless tested many times or with many mosquitoes), results are very reproducible in higher percentage inhibition. The SMFAs that are performed at the two test sites described here give similar results in terms of precision and sensitivity. This is important for the screening and development of transmission reducing antibodies or compounds, as it alleviates the necessity of performing head-to-head analyses in the same assay(s) at the same laboratory, and thereby increases the flexibility in multi-center vaccine development programs. While additional tests are likely to be required when other laboratories participate in multi-centre SMFA evaluations, the study design and findings presented in this manuscript will provide guidance for the additional testing, thus supporting future SSM-VIMT development.
Authors' contributions
KM, WS, KMK, GG; Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article; EL, MM, CL and KD; Conception and design, Analysis and interpretation of data, Drafting or revising the article. BD and LZ; Acquisition of data, Analysis and interpretation of data. TB and RWS: Drafting and revising the article. All authors read and approved the final manuscript.