Background
Despite a reduction of childhood mortality by 53% since 1990, almost six million children under 5-years died in 2015 [
1]. High child mortality rates are observed especially in sub-Saharan Africa, where 83 out of 1000 live births die, mostly due to acute respiratory infections, diarrhoeal disease and malaria [
2]. Timely diagnosis and treatment of diseases are life-saving, yet provision and access to health-care services are still limited in many areas of sub-Saharan Africa [
3]. Long distances to health care facilities have a strong influence on health-seeking behaviour and many deaths could be prevented if adequate treatment was initiated earlier [
4,
5].
To improve diagnosis and treatment for children living in rural and remote areas, innovative strategies are needed. mHealth (mobile Health), the use of mobile devices to support public health measures, offers great potential to enhance communication between patients and the professional health care system [
6,
7]. Mobile phone targeted health interventions have increasing potential in many developing countries, since the number of mobile phone subscriptions is constantly growing and the mobile phone network is also expanding to rural areas [
8].
The idea of this study was the development of an mHealth tool to support guardians of children living in areas with limited access to health care. To achieve this goal, the Integrated Management of Childhood Illnesses (IMCI) guidelines [
9] were used to develop a clinical algorithm to assess disease symptoms in sick children that could be implemented into an interactive voice response (IVR)-system applicable in automated health hotlines. The transfer of information would rely on audio files - rather than on text messages - which could be useful for the multitude of illiterates in sub-Saharan Africa [
10]. At the beginning of the development and repeatedly every 6 months during the consecutive process of testing, data analysis and writing of the manuscript (last search on 13.09.2017) we conducted a systematic literature search on PubMed to identify studies that had evaluated mHealth tools in African settings.
So far, the systematic literature research has not revealed any studies using algorithm based IVR-systems for disease detection in children [
7,
11,
12]. Some studies have been conducted in which the IMCI guidelines were translated into mHealth interventions to assist health care workers. In one study by Ginsburg et al. a mHealth application, integrating a digital version of the IMCI guidelines with a breath counter and a pulse oximeter, was developed to support health care workers in the diagnosis of pneumonia in children in Ghana [
13]. The authors concluded that their tool had “the potential to facilitate prompt diagnosis and assessment” [
14]. In another study by Rambaud-Althaus et al. the authors developed a new algorithm for the management of children under 5 years of age living in resource poor settings. It included point-of-care tests and clinical predictors for acute illnesses to improve the rational use of antimicrobials [
15]. In a subsequent controlled non-inferiority study in Tanzania they concluded that the use of their algorithm “improved clinical outcome and reduced antibiotic prescription” [
16]. However, these studies aimed at health care workers, whereas our study directly targets guardians of sick children.
This study was part of the electronic Health Information and Surveillance System project (eHISS) that aimed at conceptualizing and piloting a mobile phone based tool to collect individual disease information and to provide corresponding treatment recommendations. Health information from participating populations and the simultaneous collection of spatio-temporal data on the incidence of fever, diarrhoea and respiratory distress were envisaged for monitoring potential disease outbreaks. This manuscript presents the development and evaluation of an algorithm based tool to identify symptoms of common childhood diseases and to provide basic treatment recommendations, focussing on testing its medical correctness. User experiences with the finalized tool were evaluated in another study [
17].
Discussion
This study describes the development and evaluation of an algorithm-based IVR-tool for guardians of sick children in sub-Saharan Africa. It was shown that, compared to the subsequent examination by attending physicians, the tool performed well in identifying the three leading disease symptoms fever (agreement = 83.5%, kappa = 0.59), cough (agreement = 82.3%, kappa = 0.64), and diarrhoea (agreement = 84.4%, kappa = 0.57). The detection of vomiting was moderate (agreement = 76.4%, kappa = 0.42). Additionally, the tool was able to give out appropriate treatment recommendations. Merely seven out of 237 patients (3.0%) were not sent to the hospital by the tool, although physicians applied causal treatment for specific illnesses (e.g. prescribed antibiotics). These results indicate that the tool can provide basic treatment recommendations without a medical health care professional nearby, although disease assessment was not achieved in a reliable way. The discrepancy between adequate treatment recommendations and inadequate disease assessment may be explained by the registration of multiple disease assessments but only one treatment recommendation. Guardians were not able to answer the more specific questions correctly after confirming a symptom, but they seemed to be able to estimate the need for treatment. This was shown by the tool’s good performance in deciding whether or not a child has to be presented at a hospital (agreement = 92.8%, kappa = 0.28). Thus, this tool could contribute to timely diagnosis and treatment and may be effective in lowering childhood mortality in areas with limited access to health care services.
Success and failure of this tool are strongly dependent on the guardians’ ability to correctly assess their children’s health status. A systematic review by Geldsetzer et al. evaluated the guardians’ recognition of childhood illnesses in developing countries. Due to low values of sensitivity for the recognition of fever, diarrhoea and pneumonia, the authors concluded that “survey data based on these reports should not be used for disease burden estimations” [
21]. In comparison our results show considerably higher values of sensitivity for fever and diarrhoea, but slightly lower values of specificity. Generally, guardians detected symptoms of common childhood illnesses with higher sensitivity but lower specificity compared to other studies [
21]. Our results indicate that guardians of sick children can contribute to disease surveillance, at least by identifying their children’s symptoms.
In contrast, the results for disease assessment generally revealed inadequate performance of the tool. A possible reason may be that questions were too imprecise to achieve reliable assessment corresponding to the physicians’ findings. It was considered whether more questions could improve the assessment, but concluded that more than 15 possible items would only protract the questioning and, thus, lower user satisfaction. Moreover, one has to acknowledge the structural difference in the diagnostic process of a machine vs. a human being. The tool heavily depended on the data which were only received by asking questions; finally, data were collected in a binary yes/no form. In contrast, the physician can get additional specific clues by examining the patient and can put information into perspective.
A systematic literature search was conducted to identify mHealth studies that aimed at improving child health before the development of the algorithm. None of the reviewed publications used an algorithm-based IVR-tool for the detection of symptoms or the assessment of a child’s health status. Most of the mHealth interventions focused on improving patient follow-up and medication adherence as well as data collection/transfer and reporting [
11,
22‐
25]. The targeted group for decision-support-systems was medical personnel [
12,
16,
26‐
28]. The communication channels primarily used were short messaging services (SMS) [
29‐
31], whereas IVR-systems were not used at all [
7,
11,
32].
Given that this is the first study using an IVR-tool operating on a medical algorithm to receive health information and process them into a health assessment, it had several limitations that should be considered in future studies. First, the IMCI guidelines proved to be difficult when compared to the physicians’ notes so that some questions, especially the danger signs, could not be measured adequately. Reasons might be the low prevalence (e.g., for convulsions and unconsciousness, which were fast-tracked through the OPD), the lack of the guardians’ medical knowledge of danger signs (as reported in other studies [
16]) or the fact that the physicians assessed a child’s health status in a different way than the IMCI. Future studies should use structured questionnaires to be filled out by the physicians during consultation to record clinical assessment and improve comparisons between the findings made by the tool and the physician.
Second, not all characteristics of the IMCI guidelines could be implemented into the algorithm. One of its key components is the assessment by the community health worker: inspecting the child (e.g., for chest indrawing), measuring the body temperature and counting the breaths (“ask, look, feel”). We could not transfer these tasks onto guardians without medical experience or otherwise the data would be unreliable and result in biased information. Thus we had to reduce the complexity of the IMCI guidelines in such a way that they could be understood by the guardians and correct data could be obtained. Naturally, this happened at the expense of the diagnostic performance, which was anticipated during the development of the algorithm. However, since we wanted to avoid recommending home care in severe illnesses, we designed the algorithm to detect symptoms as sensitive as possible and accepted the compromise between symptom detection with high sensitivity on the one hand and “overestimating” a child’s condition on the other hand.
Third, the study was conducted in a children’s OPD of a well-equipped district hospital, which implicated a higher prevalence and severity of diseases compared with everyday life. Due to this instance, the tool’s performance could be appropriately evaluated for the treatment recommendations A and B, but the prevalence of C-cases was too low. Therefore, our research group initiated another study in fieldwork to evaluate our tool in the domestic setting, where more C-cases were suspected.
The study results indicate the ability of the tool to detect symptoms of common childhood diseases and to give suitable treatment recommendations. Our research group further found that our tool was applicable and accepted by Ghanaians in focus group discussions [
17]. Furthermore, the project group developed a software application for space-time surveillance and model-based analysis within the eHISS system. Provided its adequate performance in the domestic setting, the panel of clinicians and epidemiologists could use the gathered information to update and optimize the tool. In its final version, the eHISS tool is envisaged to provide people with health information and give health authorities and health policy makers reliable data on prevailing disease syndromes.