The score of integrated disease surveillance and response adequacy (SIA): a pragmatic score for comparing weekly reported diseases based on a systematic review

Sub-Saharan Africa has the highest total morbidity and mortality burden of infectious diseases in the world [1]. In 2012, global morbidity in the region was estimated at 74,000 per 100,000 inhabitants. This number is nearly twice that of the Eastern Mediterranean or South-East Asian regions (40,779 and 40,341 per 100,000 inhabitants, respectively) [2]. In addition, approximately half the mortality from all types of infectious diseases in the world occurs in Sub-Saharan Africa [3, 4]. While emerging non-transmittable diseases have expanded in the region in the last two decades, infectious diseases remain a major public health concern [3, 4]. By the end of the 1990s, most sub-Saharan African countries had seen major public health events or outbreaks. Huge epidemics occurred in the DRC, including the large cholera outbreak in the Rwandan Hutu refugee camp in Goma in 1994 [5], the 1000 cases of poliomyelitis reported in Mbuji-Mayi in 1995 [6, 7], and the largest ever monkey pox outbreak that took place in Sankuru in 1996–1997 [8]. Overall, the growing epidemic risk in sub-Saharan African countries is a source of major international concern.

In 1998, an Integrated Disease Surveillance and Response (IDSR) strategy was initiated under the guidance of the WHO Regional Office for Africa (WHO AFRO) to prevent and control these multiple epidemic emergencies [9, 10]. This strategy works by reinforcing national public health surveillance and response systems in the region. According to the IDSR technical guidelines, the specific objectives of the strategy are to: 1) integrate vertical disease surveillance systems for effective and efficient use of resources; 2) improve the flow and use of information for detecting and responding to public health threats; and 3) improve country capacity to detect and respond to priority public health events [10, 11]. The IDSR strategy, which relies on systematic and continuous data collection and reporting by health care facilities, has eight functions: identification, notification, analysis and interpretation, epidemic investigation and confirmation, preparation, response, circulation of information and evaluation, and improvement of the system [11‐14]. Depending on the health specificity of each country, WHO AFRO recommends IDSR surveillance of a number of priority transmittable diseases (weekly reporting) and non-transmittable diseases (monthly reporting) [11]. This timely continuous epidemiological surveillance improves the availability and use of data on the leading causes of illness, death, and disability in the region [15]. As such, the IDSR strategy contributes to high-level decision-making in the area of public health in participating countries.

In addition to contributing to disease outbreak prevention and control, IDSR morbidity data are increasingly relevant to epidemiological research. In particular, they are now being used to determine the spatial and temporal dynamics of various diseases [16‐18]. They thus constitute an alternative to other techniques for generating health data in low and middle-income countries (e.g., demographic health surveys, STEPS surveys, household surveys, etc.). While these techniques can produce reliable data, they are indeed very costly to implement [19]. In the DRC, more and more epidemiological studies rely on the large amount of data that have been produced since the 1999 implementation of the IDSR strategy.

It has been demonstrated, however, that the IDSR strategy results in social and spatial discrepancies (differences) in disease distribution between reported cases (reported morbidity) and field reality (actual morbidity). In the particular context of the DRC, this is likely because IDSR morbidity data reporting is based on a syndromic approach [19‐22]. Unfortunately, these discrepancies cast doubt on the validity of epidemiological studies using IDSR morbidity data. Because the true value and accuracy of these data are difficult to evaluate [19, 20, 22], there is a pressing need to develop a method for validating them before they can be used for research or public health purposes.

While some approaches have been proposed to assess the quality of IDSR morbidity data, they focus on a limited number of diseases. These approaches include: 1) a method for assessing the “relevance and validity” of IDSR morbidity data on chickenpox, hepatoma, anaemia, malnutrition, and measles [20]; 2) a method for constructing, comparing and spatializing selected indicators and indices for analyses of malaria based on IDSR morbidity data [19]; or 3) a method using a tree model scenario to estimate the proportion of lost cases of monkey pox in the DRC health system due to IDSR data reporting [23]. To our knowledge, no method applicable to all diseases monitored by IDSR surveillance and assessing the discrepancies between “reported morbidity” and “actual morbidity” has been published.

The aim of this article is to propose a method for assessing the level of adequacy of IDSR morbidity data in reflecting the actual morbidity of the 15 weekly reported diseases monitored by IDSR surveillance in the DRC.

When the SIA was applied to the 15 weekly reported diseases, SIA values ranged from 4 (rabies) to 19 points (cholera). The mean score was 10.7 points (Table 3). Agreement between items in each dimension ranged from 0.03 to 0.47 (Kappa coefficients). All numerical code values proposed for each item were used at least once.

After discretization, 3 categories of diseases were identified as follows (Fig. 4):

The application of the SIA to the 15 weekly reported diseases monitored by IDSR surveillance in the DRC yielded 3 categories or types: high score or good adequacy (value > = 14; usable data), moderate score or fair adequacy (value > = 8 and < 14; usable data after adjustment), and low score or low or non-adequacy (value < 8; non-usable data).

Overall, our examination of studies from a wide range of disciplines, as well as of IDSR technical guides and reports of outbreak investigations, gave a strong foundation to this study. The SIA is the outcome of sound interdisciplinary work, combining the expertise of specialists in the fields of integrative health geography, public health, epidemiology, infectious diseases, and medical anthropology. It also draws on 20 years of experience and feedback from field practitioners responsible for care structures in rural areas of the DRC.

The SIA was constructed using a pragmatic and robust assessment method. The agreement between items varied from low to moderate, indicating a low redundancy of items in each dimension. All the code values assigned to SIA items were used at least once. The sensitivity analysis confirmed the robustness of the score: neither the score ranking nor the classification of the different diseases were modified when iteratively removing one item at a time, or when modifying the item code values. However, for some Type-2 diseases, the score ranking was slightly modified, even as the classification remained unchanged.

According to the PRISMA checklist, there should be no bias within or across studies included in systematic reviews. Our systematic review was not concerned by this requirement because no meta-analysis was included. Only qualitative studies were screened for determinants that can induce discrepancies between reported morbidity and actual morbidity.

Other methods have been proposed to study the adequacy of reported morbidity in reflecting actual morbidity. However, these methods focused on a single disease or on a limited number of diseases [19, 20, 23]. To our knowledge, the SIA is the first method that can apply to all 15 weekly reported diseases monitored by IDSR surveillance in the DRC. As such, it allows for the extensive assessment of the quality of data collected on a wide range of diseases, including neglected diseases that are not integrated into major global strategies (like pertussis or plague).

Our study found coherence in the value of the score obtained for each of the diseases. The SIA confirms the low quality of data produced on diseases (such as malaria) monitored using syndromic surveillance [22, 29]. Moreover, it allows for analysing data on certain pathologies (neonatal tetanus, acute respiratory infections, and rabies) that are less frequently investigated. Unlike previous attempts at assessing data quality (usually data on a single health problem or disease) with tools using binary variables (presence/absence, positive/negative, or yes/no) [20], the SIA can quantify the adequacy of data in reflecting the actual morbidity of each monitored disease. It is therefore one of the only tools available to pragmatically assess the quality of IDSR morbidity data. This can be of great relevance to the various users and actors involved in the implementation of surveillance and response strategies like the IDSR.

Nevertheless, this study has limitations that must be acknowledged. Our literature review may have missed some relevant determinants that could have been included in the score. However, only currently available and accessible information in the DRC were used to construct the SIA. It is therefore unlikely that an additional determinant would have affected the consistency and validity of the score.

Another limitation of our study is that the SIA was applied to weekly reported diseases that are monitored using the syndromic approach. The code values of 3 items (number of epidemics reported each year during the last 5 years; PPV; and proportion of health zones affected by epidemics of each of the 15 diseases) would likely change if the data on these diseases concerned biologically confirmed cases. It is therefore also likely that the score ranking of the different diseases would change if the SIA were applied to diseases not monitored using the syndromic approach.

In our study, cholera had the highest score among all Type-1 diseases. The good adequacy of IDSR cholera data in reflecting actual cholera morbidity has already been demonstrated in a study by Bompangue et al. This study examined the spatial and temporal dynamic of cholera outbreaks using IDSR morbidity data: it identified lacustrine sanctuary areas as the site of emergence of all cholera outbreaks in the DRC [16, 17]. These findings have led to significant advances in the understanding of the mechanisms involved in the spread and recurrence of cholera outbreaks. They have also prompted the DRC Ministry of Health to adjust the national strategy for cholera control by targeting cholera sanctuary areas [18]. The success of this adjusted strategy may be seen as a retroactive validation of the quality of IDSR data on this disease. This is in line with our findings, whereby IDSR data on cholera have a high level of adequacy in reflecting actual cholera morbidity, as calculated using the SIA.

Similarly, the SIA classification of measles as a Type-1 disease corroborates the findings of a study by Fasin et al. [20]. Of the five diseases or health conditions analysed in this study (chickenpox, hepatoma, anaemia, malnutrition and measles), only data on measles morbidity had a good level of adequacy in reflecting actual measles morbidity. This pathology was also the only one to show both satisfactory relevance and quality [20]. These findings are unsurprising because in low and middle-income countries, where measles prevalence remains high, health workers are well-equipped to recognize and produce a clinical diagnosis of the disease and diagnosis is oriented in the absence of prior vaccination [30].

Adjusted data on monkeypox, a Type-2 disease as per the SIA, have been used in a study conducted in the DRC by Hoff et al. This study using IDSR morbidity data on monkeypox proposed two adjustment methods: a comparison of monkey pox with two control diseases (acute flaccid paralysis and neonatal tetanus), and a scenario tree model to estimate the proportion of potentially lost cases in the DRC health system [23].

Our study indicates that IDSR morbidity data on yellow fever, malaria, typhoid fever, and rabies (Type-3 diseases) should not be used for epidemiological research or public health purposes, due to their low level of adequacy in reflecting actual morbidity. Until 2014, the clinical and presumptive diagnosis of malaria based solely on fever likely induced many false positives. Several infectious diseases can cause fever, which may explain the large gap between supposed and actual cases of malaria [22, 29, 31]. The same applies to yellow fever and typhoid fever, which can be confused with several other pathologies when they are syndromically diagnosed. One of the main reasons for the misdiagnosis of rabies, and for the limited use of data on the disease, may be its long incubation period. It is indeed difficult for health workers to link a dog bite that occurred 1 to 3 months earlier to recent symptoms. The clinical diagnosis of rabies can also be difficult if the specific signs of hydrophobia or aerophobia are not present [32, 33]. Lastly, bat-acquired cases of rabies may be missed because health care providers fail to inquire into the history of bat bites [34].

Our study found that the quality of IDSR morbidity data is highly variable from one disease to another. If confirmed, these findings could prompt a revision of the IDSR strategy as it is applied in low- and middle-income countries. The different algorithms of disease surveillance could be reconsidered and improved, especially those that rely on a syndromic approach. Lastly, the SIA could be applied: 1) to monthly reported diseases; and 2) to weekly reported diseases in other low- and middle-income countries that are not monitored using a syndromic approach.