Consultative meeting that examined alignment and discrepancies between health facility and household survey data in Malawi

Malaria is a significant public health problem in Malawi that is the most frequently reported disease among children under age 5 at both outpatient departments and village clinics. Each year, approximately six million malaria cases account for 30% of all outpatient visits at health facilities, 34% of inpatient hospital admissions, and 2967 malaria-related hospital deaths, according to Malawi’s 2018 Health Management Information System (HMIS). In Malawi, the entire population is at risk of malaria infection and the incidence of malaria was 393 per 1000 in 2018, according to HMIS data.

The transmission of malaria in Malawi is perennial with the peak number of cases occurring between January and May, which is after the beginning of the annual rains. High burden areas are located in the warmer, more humid low-lying areas (lakeshore, Shire river valley, and central plain), while there is lower risk in the highland areas of Rumphi, Mzimba, Chitipa, and the Kirk Range [1]. Plasmodium falciparum is the most common species of malaria, which accounts for 98% of infections and all cases severe disease and death [2]. In the 2017 Malaria Indicator Survey (MIS), the prevalence of malaria parasites (by microscopy) in children under age 5 ranged from 11% in the Northern region to 26% in the Central and Southern regions, with the national average of 24% [3].

In the national Malaria Strategic Plan (MSP) for 2017–2022, Malawi, through the National Malaria Control Programme (NMCP), aims to reduce the incidence of malaria from 386 per 1000 population in 2015 to 193 per 1000 by 2022; and to reduce malaria deaths from 23 per 100,000 population in 2015 to 12 per 100,000 by 2022 [4]. The NMCP plans to achieve these goals by implementing efficacious interventions, such as long-lasting insecticide-treated nets (LLINs), prompt and effective case management, and intermittent preventive therapy in pregnancy (IPTp).

Specific objectives related to LLINs, case management, and IPTp in the 2017–2022 MSP are achieving universal LLIN access, which is one net for every two individuals in a household; ensuring that 95% of suspected malaria cases are tested and 100% of confirmed cases are treated with the recommended anti-malarial drugs, such as artemisinin-based combination therapy (ACT); and increasing the uptake of three or more doses of IPTp in pregnant women from 12% in 2014 to 60% by 2022. Other MSP objectives include promoting social behaviour change and communication that enhances community uptake of interventions; minimizing ACT stock-outs; strengthening surveillance, monitoring and evaluation (M&E), operational research; and improving data quality and accuracy.

The District Health Information Software 2 (DHIS2) (https://​www.​dhis2.​org/​) platform was implemented in Malawi to strengthen data management at the district and national levels by improving data capture and storage for programme and service delivery units in the public health sector. The platform was introduced with technical support from the University of Oslo, in collaboration with the University of Malawi College of Medicine, in 2011 and was adopted by the Ministry of Health and Population through the enactment of the Health Information Systems Policy in 2015.

In addition to robust routine surveillance data through DHIS2, Malawi has also conducted household surveys with the technical assistance of The Demographic and Health Survey (DHS) Program in Rockville, MD, USA, which is funded by the United States Agency for International Development (USAID). The most recent surveys include two Malaria Indicator Surveys (MIS), conducted in 2014 and 2017, and a DHS in 2015–2016. These nationally representative surveys gather data on indicators of the prevalence, treatment, and control of malaria.

A recent analysis examined the use of malaria M&E data to track progress and refine the targets in the MSP [5]. This study found that although targets had not been achieved, future MSP targets were set at the same or a higher value. The authors attribute this to the insufficient use of data. The availability of a digital routine surveillance platform and multiple household surveys in Malawi presents an opportunity for the NMCP to use complementary data sources to evaluate progress towards the MSP targets [4, 6]. Since there is little evidence in the literature that these data sources are being used together for strategic decision making [7], a Malaria Data Consultative Meeting was planned to determine how to leverage these data sources for programmatic support.

The aims of the meeting focused on the analysis of malaria indicators from the facility-based DHIS2 data and the household-based data from the 2014 Malawi MIS, the 2015–2016 Malawi DHS, and the 2017 Malawi MIS. The meeting was held in mid-July before the MSP review at the end of July. Meeting participants reviewed the 2017–2022 Malawi MSP to identify opportunities for complementary information from household survey data and to refine and inform DHIS2 data being used to evaluate MSP targets. This review led to the creation of case studies that would be used to examine situations when household and DHIS2 data sources provided discrepant results and when these sources could be used together to provide a more complete picture of existing malaria transmission and control.

The stated aims of the meeting were that, by the end of the meeting, participants would be able to:

The first part of the Malaria Data Consultative Meeting included a 3-day data exploration meeting designed for malaria data experts to explore the available indicators and produce graphics that would highlight similarities and differences between the DHS/MIS and health facility data. During this meeting, participants: (i) reviewed malaria M&E data sources; (ii) discussed the strengths and weaknesses of these data sources; (iii) reviewed the main household and facility-based malaria indicators; (iv) identified indicators from DHS/MIS surveys that complement the health facility data; (v) checked health facility-based data for external consistency with the DHS/MIS data; (vi) compared indicators to targets identified in the 2017–2022 MSP; and (vi) drafted three case studies that compared DHS/MIS and health facility data. In total, 13 individuals participated in the data exploration meeting (6—NMCP, 2-Ministry of Health (MoH), 2—Kuunika Project, 1—Central Monitoring Evaluation Division (CMED), 1—The DHS Program, and 1—Malaria Atlas Project).

The Malaria Data Consultative Meeting then included a 2-day presentation of results. This meeting was designed for a larger group of stakeholders who would discuss the findings of the data exploration meeting and comment on the case studies. The first day of the results presentation included oral presentations of the three case studies and a roundtable discussion, while the second day included a structured writing workshop focused on assembling the first draft of the meeting report. A total of 23 individuals participated in the results meeting.

A list of 45 core indicators was assembled from the Malawi MSP and the World Health Organization (WHO) Guidance for Malaria Programme Managers [6, 8]. The selected indicators were metrics gathered from routine DHIS2 or household surveys that are commonly used to evaluate the success of malaria control programmes focused on vector control, chemoprevention, case detection, diagnostic testing, and treatment. Key meeting activities included reviewing the list of metrics and identifying which indicators could be measured with both sources. There was a small amount of overlap, which is to be expected given that the data sources are not designed to replicate indicators, but rather to complement each other. For the six indicators found in both data sources, participants explored how the metrics could be disaggregated by geography (region or district) or population (pregnant women or children under age 5). Three indicators that were strong candidates for the case studies, based upon the number of data collection years in the DHIS2 and household surveys were distribution of LLINs to pregnant women through antenatal clinics (ANCs), diagnostic testing rates in children under age 5, and uptake of two or more doses of IPTp by pregnant women. The indicators were restricted to children under age 5 and pregnant women for comparability between data sources, but these key risk groups were also highlighted in the MSP targets.

After the presentation of each case study, there was an opportunity for discussion and feedback from the stakeholders present for the second portion of the data consultative meeting in Lilongwe. Participants identified additional data that would complement the results of the case studies and recommended next steps based on the MSP. This was programmatically relevant given the upcoming review of the MSP.

Some issues arose in the three case studies. The stakeholders reiterated that rather than comparing the absolute values of the indicators from the two data sources, the similarities (or discrepancies) in the trends should be the preferred focus. It is not possible to have the exact same denominator for each indicator (Tables 1, 2 and 4), which makes it difficult to compare values rather than trends. In addition, although routine data sources are subject to quality issues such as reporting completeness and the influence of commodity stockouts, survey data are also prone to recall or respondent bias. Finally, survey data may be able to capture the impact of the private sector more effectively than routine surveillance. Although some private sources of care are captured in DHIS2 (Christian Health Association of Malawi [CHAMs] facilities), many are not. Thus, it is not possible to quantify the percentage of care that is sought but is unavailable.

It was emphasized that although net use is included in the MSP, distribution is not. Completeness of distribution could be a benchmark so that access as well as use is assessed. Although net distribution at ANC and birth includes key risk groups, it would also be beneficial to organize other routine distribution points such as schools.

Both the DHIS2 and household survey data show improvement in the testing of suspected cases over time. According to the DHIS2 data, the MSP targets have been met. When all ages (instead of only children under age 5) were examined, testing rates exceeded 100%. This may reflect data quality issues that need to be addressed. It is also important to report testing rates and the status of commodity stocks, so that the root causes of low rates can be determined. Children under age 5 were the focus of the case study because it allowed for a more direct comparison with the household survey data. However, confirmation of all suspected cases should be a priority, as outlined in the MSP. There may be additional data sources to consider when assessing this metric, such as Logistics Management Information Systems (LMIS) or Multiple Indicator Cluster Surveys (MICS).

Assessment of IPTp showed some surprisingly low rates of uptake given that the indicator reflected two or more doses of SP during pregnancy and the recommendation is now three or more. However, many participants emphasized that IPTp doses are inherently dependent on the number of visits made to ANC and many women do not go to ANC until their second trimester of pregnancy, which limits the number of doses a woman can receive. Next steps include improving community engagement to break down barriers to IPTp uptake such as myths or superstitions associated with early ANC attendance and institutional barriers such as commodity availability (SP/Fansidar), as well as increasing access to care points by taking the services closer to communities, which the NMCP is currently piloting. A key point from the stakeholder discussions was that community awareness is a key factor in determining the levels of the indicators and that there needs to be greater focus on other determinants of health such as socioeconomic issues and education.

Consistent evaluation of planned targets is needed as countries work to control malaria. Malawi has assembled a wealth of data that can serve as benchmarks to programmatic success. The adoption of the DHIS2 platform for routine surveillance of data has made facility and clinic-based data more readily accessible and reliable. Multiple household surveys conducted with standardized methodologies provide consistent indicators over time. Data from DHIS2 and household surveys have different strengths and weaknesses. Routine surveillance is subject to errors in reporting and completeness and is dependent on commodity stocks, but includes data from the entire country, which can be aggregated at subnational levels and followed longitudinally. Survey data are collected at a single point in time, are subject to recall bias, and do not provide subnational estimates, although the survey design allows surveys from different years and locations to be compared by a range of outcomes and predictors. The case studies presented in the meeting illustrate that routine data and household survey data can be used together to produce a more complete, unbiased picture of malaria in Malawi.