Background
According to the World Health Organization (WHO), investment in malaria strategies needs to increase to meet the new global malaria targets for 2030 [
1]. There is a substantial and growing body of evidence demonstrating the effectiveness of malaria control interventions in reducing malaria morbidity and mortality, as well as the benefits of malaria control for health systems [
2‐
10]. However, less is known about the micro-economic impact of such interventions. Effective malaria control can reduce the incidence of malaria, reducing disruption to household economic activities and also reducing household resources allocated to malaria care-seeking [
11]. As a result, investment in malaria control could lead to lower rates of work and school absenteeism, improved worker productivity, higher household income, and greater spending on key household commodities. Country-level evidence on the impact of malaria control scale-up strategies on micro-economic outcomes would provide a more holistic picture of the benefits of malaria control and better inform ministries of health and finance, donors, private sector actors, researchers, and other stakeholders to make decisions about allocating resources for malaria.
Studies using experimental and quasi-experimental approaches have explored the impact of malaria control on certain key micro-economic outcomes, especially education. However, many of these studies use a short-term follow-up period and are not necessarily generalizable to national level, meaning there is a lack information on the impact of malaria control over a longer period of time at the country level. Notably, a prospective randomized controlled trial (RCT) conducted in a rural area of Zambia’s Southern Province, found a statistically significant increase in the productivity of farmers (a nearly 15% higher value of agricultural output, given free insecticide-treated nets (ITNs) compared to farmers with access to partially subsidized ITNs through loans [
11]). Another RCT in India concluded that offering micro-loans for ITNs led to a reduction of 2 days in lost work or school absenteeism and a decrease of 269 rupees (31%) in medical costs due to malaria over 6 months, relative to the control group [
12]. An earlier RCT in Kenya found that ITNs correlated with a reduction of $0.25 in health care expenditure over a 2-week period for children under 5 years old, as well as a statistically insignificant reduction of 0.5 days in household time lost due to caring for sick children [
13]. A recent retrospective study in Uganda looked at the long-term, micro-economic benefits of a malaria eradication programme more than a half-century ago in southwestern Uganda. The authors concluded that this programme boosted primary school completion for females, improved educational attainment by half a year for both sexes, and increased the likelihood of male wage labour by nearly 40% [
14]. Other studies explore the effects of malaria control interventions on educational outcomes [
15‐
19]. In general, the evidence related to educational outcomes has been mixed.
This study complements existing literature through its national scope, by employing a multi-year study period and by utilizing commonly available malaria and micro-economic data in Zambia to assess the relationship between malaria control scale-up and micro-economic outcomes. The authors considered several potential evaluation methods to answer this research question. A RCT was eschewed for a number of reasons, including costs, political feasibility given the need to vary access to the malaria control intervention by treatment group, and an extended time frame for study. Instead, a quasi-experimental design, drawing on retrospective data, was chosen. The authors purposely chose to use secondary, retrospective data to provide an example of how readily available, high-quality, low-cost data might be used to answer policy-relevant questions using a quasi-experimental design. After considering several quasi-experimental approaches, the authors opted for using both a fixed effects model, based on a difference in differences (DD) approach, and a generalized propensity score (GPS) model because they best suited the data available (one pre- and one post-scale-up time period) and highlighted the effects of scale-up of the intervention (done using measurable criteria such as malaria incidence).
The study data covered the period 2006–2010, when the Government of the Republic of Zambia and cooperating partners substantially scaled up malaria control efforts. The authors tested whether the scale-up of malaria control activities in Zambia is associated with food consumption, total household consumption, schooling attendance, and agricultural production. This study provides an illuminating example of a resource-conscious approach that marries commonly available data with a quasi-experimental design and has potential relevance for countries and researchers interested in assessing the micro-economic effects of malaria control efforts. Ultimately, this study places malaria control strategies within the broader context of poverty alleviation, serving to inform stakeholder decision-making. The authors proceed by discussing the methods employed for this study, the analytical results, and discuss the lessons learned and conclusions from this research endeavour.
Discussion
In this study, the authors used a quasi-experimental design and drew upon commonly available secondary data on malaria coverage and micro-economic outcomes to provide robust evidence of the relationship between malaria control coverage and micro-economic outcomes at national level in Zambia, a country that has scaled-up malaria control efforts and reduced the incidence of malaria. Reliable, country-specific evidence would contribute to a better understanding of the return on investment from scaling-up malaria control efforts and achieving the objective of malaria elimination, especially as governments and donors face competing demands on limited resources.
The substantial scale-up of malaria control efforts in Zambia from 2006 to 2010 combined with the availability of data coinciding with this period provided a unique opportunity to use secondary data and a rigorous analytical approach. The authors observed improvements in coverage of malaria control efforts in parallel with improvements in micro-economic outcomes, such as food spending, during the study period. The authors also found that food spending in 2006 was positively correlated with ITN and/or IRS coverage in 2006. These positive correlations suggest that either higher ITN and/or IRS coverage achieved by 2010 caused higher food spending by 2010 (i.e., a positive micro-economic impact), or that areas that tend to have higher malaria control coverage tend to be areas with higher food spending to begin with. The quasi-experimental approaches attempted to control for this second association, which represents a selection effect and confounds the relationship between malaria control scale-up and micro-economic outcomes.
When exploratory OLS analyses of the correlation between the scale-up over time of malaria control efforts and micro-economic outcomes was performed, the authors found that districts targeted earlier for malaria control scale-up (which are called Phase 1 districts in this study) had higher rates of school attendance and more years of education, but lower food spending by 2010 than those targeted for later efforts. One interpretation is that Phase 1 districts, by virtue of being targeted earlier and over a longer period, had better economic outcomes by 2010, as proxied by educational attainment. However, the lower food spending by 2010 suggests that the authors may also be capturing some possible selection bias; Phase 1 districts may have had worse outcomes to begin with, such as lower food spending, and therefore were targeted first. While these exploratory OLS results (along with the initial correlations identified) do not provide conclusive evidence suggesting that malaria control efforts were targeted to specific areas, they further highlight the need for quasi-experimental approaches to generate unconfounded results that allow for causal inference related to the impact of malaria control efforts.
For the GPS model, the authors failed to identify statistically significant relationships between ITN ownership and/or IRS receipt and the primary (total household spending on food) or secondary micro-economic outcomes. For food spending, the confidence intervals were such that positive associations between malaria control coverage and micro-economic outcomes greater than 14 or 34% could be rule out, depending on specification. To put this into context, for households with average food spending among the lowest quintile for food spending in 2010, an increase of 14 or 34% in food spending would not move them out of the lowest quintile; indeed, it would require a 49% increase in average food spending of those in the lowest quintile to move them up to the next quintile.
For the secondary outcome, educational attainment, the confidence interval around the GPS coefficient rules out an effect greater than half a year of schooling. The authors can compare this upper bound with results from other studies focusing on educational attainment. A retrospective study in Uganda of a malaria eradication programme showed that educational attainment increased by a half a year for both women and men [
14]. Another retrospective study found that national malaria eradication campaigns in Sri Lanka and Paraguay increased completed schooling by approximately 0.1 years [
17]. These studies suggest that the bounded estimate is within the range of other studies, if there is in fact a positive impact of malaria control coverage on educational attainment. That said, other studies have also found no effect on educational attainment [
15,
18], and the authors similarly cannot rule out that there was no impact on educational attainment.
For agricultural production, the GPS results ruled out an effect greater than a 57% increase in maize production. By comparison, Fink et al. identified a 15% increase in agricultural output as a result of providing free bed nets. The results clearly include this effect but are too large to make any meaningful conclusions. In addition, there still remains evidence of imbalance in certain analyses, specifically for agricultural production, even after controlling for the GPS.
As with the GPS model, the fixed effects model did not find statistically significant relationships between malaria control scale-up and either primary or secondary micro-economic outcomes. Confidence intervals were similarly wide, making it impossible to rule out positive, negative or null changes in these outcomes.
One factor that may explain the absence of detectable effects in this specific example is the resurgence of malaria in certain provinces of Zambia over this time period. Malaria parasite infection prevalence declined rapidly from 2006 to 2008 in nearly all provinces of Zambia, but then increased again in six of nine provinces by 2010, according to MIS data [
34]. These trends indicate that malaria control programmes may not have maintained as high levels of coverage in the second half of the study period, possibly due in part to reduced funding for commodities and delays with procurement and implementation, which led to ITNs not being delivered [
35]. Despite summary statistics showing large increases in coverage during this period, these figures may mask lower changes in prevalence if malaria control scale-up was not effectively implemented in practice. Additionally, reported use of ITNs used in the treatment measure may well be higher than actual use due to desirability bias, which would attenuate the relationship between the malaria control and micro-economic outcomes.
While the results are inconclusive, the approach taken in this study can yield lessons for key stakeholders, including donors, governments, and other researchers interested in investigating the link between malaria control scale-up and micro-economic outcomes. These lessons illuminate the challenges encountered in using secondary data, including selection effects, limited sample size, data limitations, and other potential confounders, and suggested approaches for addressing them.
Using low-cost, reliable, representative and readily available data offers clear benefits in answering policy-relevant questions using rigorous analytic approaches. Specifically for researchers and stakeholders conducting malaria-related research, there is ready access to standardized malaria indicator data on both ITN distribution and IRS available for 30 countries in Africa through MIS. Many countries have multiple years of data from the MIS, allowing for longitudinal analyses and nationally representative findings. In addition, programmatic data on net distribution and spraying may also be available, along with micro-economic data from integrated living standards surveys (like the LCMS) on outcomes such as household consumption, agricultural production and education. Use of these existing, secondary data, coupled with a quasi-experimental approach, provides a resource-friendly alternative to stand-alone data collection efforts. In addition, if there is sufficient variation in the intervention and it is possible to identify similar comparison groups, a quasi-experimental approach can provide convincing causal evidence, in the absence of a randomized experiment.
The first lesson from the study relates to the challenges of using nationally representative data, which may not be representative at lower geographic units, such as sub-district. The authors discovered that conducting the analysis at the district level significantly reduced the study’s statistical power. There were 72 observations, the total number of districts in Zambia in 2010, for the analysis. The power calculations showed that this model could detect a 22% increase in food spending, yet the upper bound of the confidence intervals from the GPS results showed that the authors were able to reject an effect greater than 14–34%, depending on specification. The confidence intervals for agricultural production are even larger. Other studies, such as Fink et al. [
11], identified a 15% increase in agricultural production which suggests that this study was under-powered to detect these positive impacts. One potential solution would be to conduct the analysis at a lower geographical level, which would increase the statistical power but may introduce other challenges. For example, analysing these data at a lower geographic unit runs the risk of environmental bias, meaning that the randomly selected sampling unit within a given area may have better or worse outcomes by chance than the average in the area.
Moreover, while there are clear benefits from using low-cost, readily available, national-level, secondary data, a key consideration relates to the feasible unit of analysis that can be used. This is especially important when multiple sources of data must be merged into a single analytic dataset, and when some of the sources are only available at the district level or higher. For example, some data, such as NMCC data on net distribution, was only available at the district level. The sample sizes required can be large, as the simulations demonstrated for Zambia over the period of study. Smaller sample sizes would be suitable if the predicted effect is assumed to be large; in this case, only limited evidence of effect size was available at the time of the study design.
The second lesson relates to whether using a quasi-experimental design drawing on secondary data can sufficiently and convincingly eliminate potential selection effects. The OLS regressions found evidence of potential selection bias, meaning that provinces which were targeted earlier for malaria control scale-up had lower food spending by 2010 but also ended up having better educational outcomes. These contradictory findings point to potential selection effects (i.e., provinces targeted earlier were those with poorer outcomes such as lower food spending) combined with the potential impacts of malaria control interventions (i.e., provinces targeted earlier had better educational outcomes after scale-up). These potential selection effects are a prime reason for selecting quasi-experimental designs. In the GPS analyses, despite mitigating selection effects by controlling for baseline characteristics used to generate the propensity score, the authors still found evidence of imbalance (meaning that there still remain unobservable factors that differentiate districts). Factors, such as political considerations that may have influenced district selection for scale-up efforts as well as ease of implementation of malaria control efforts, are challenging to control for analytically and are not available through these secondary data sources. By complementing the GPS approach with the fixed effects model, the authors attempted to control for time-invariant characteristics at the district. Yet, the large confidence intervals highlight that this study still had limited statistical power. The simulation findings indicate that the GPS model is not a viable choice for examining the relationship between malaria control and food expenditures in Zambia.
The third lesson highlights the potential benefits of using primary versus secondary data, which can influence factors such as the authors’ ability to control for selection bias as discussed. With primary data collection, researchers have more control over the data collected, both at baseline and follow-up. There would no longer be a concern with merging data sources at different levels of disaggregation or representative populations. In the case of malaria, it would also be more useful to have data with a very short recall period since the micro-economic impact of a malaria episode may not necessarily be identified through measures aggregated across a year. For example, data on days of work lost, school days missed, food spending, and household financial coping strategies in the last month would be much more useful in identifying an impact on household’s economic well-being. The available schooling measure in the LCMS (whether a child is attending school in general) is a crude approximation that is not likely to capture the effect of a malaria episode on days of school missed. While developing survey instruments that collect the necessary variables would provide the exact measures needed, primary data collection is much more costly than relying on secondary datasets. These costs need to be weighed against the reliability of estimates that can be produced using existing secondary data.
One recommendation is to add additional survey questions to large-scale household surveys, such as the MIS, the LCMS, the Living Standards Measurement Studies, and the Demographic Health Surveys. For example, the LCMS in Zambia could add a small set of questions on malaria coverage and therefore permit an examination of the research question of this study at the household level. Alternatively, the MIS could incorporate micro-economic outcomes into its survey instrument. The challenge though is that certain variables such as food consumption and agricultural production require large modules to collect reliable estimates. High-quality implementation data would also be helpful. In this study, the authors intended to use data from the NMCC on ITN distribution by district to look at the impact of donor and national government’s malaria control efforts. However, when these data were correlated with the MIS ITN ownership data, the authors found a very low correlation and determined that more reliable estimates would come from the MIS data. Nonetheless, implementation data from the NMCC, especially if available at a lower geographical area, would be a low-cost, alternative data source for this type of study.
The fourth lesson relates to the selection of research methodology. Where there are relative merits of using a retrospective, quasi-experimental approach compared to other research designs, there are accompanying challenges as well. One of the reasons the authors chose a retrospective, quasi-experimental design was the potential challenge of randomizing access to malaria control interventions. One important ethical concern of donors and implementers is the prospect of withholding effective malaria control measures for an extended time period. However, randomized experiments seek to overcome this limitation in various ways, such as varying price for nets. There are clear benefits from using a randomized experiment, in terms of causal attribution. Evaluators can conclusively attribute any microeconomic effects to the malaria control interventions, as in Fink et al. [
11] where the impact on farmer productivity can be directly attributed to having access to free bed nets. However, other factors, such as programmatic feasibility of varying access to an intervention, particularly over longer periods of time, may make a randomized experiment less feasible or desirable. In this case, using retrospective, commonly available data over a longer period of time allowed a national scope, over a multi-year period, which may yield different results than smaller scale randomized experiments implemented in a small geographic area.
Authors’ contributions
AC, AL, SN, and MA were involved in the study conception and design. SN travelled to the field to acquire data, with assistance from HK, BB and BH. AL and SN cleaned data. AC, AL and SN analysed and interpreted data, and drafted the manuscript. BJ and LO provided methodological advice. All authors provided critical revisions of the manuscript. All authors read and approved the final manuscript.