Epidemic surveillance in a low resource setting: lessons from an evaluation of the Solomon Islands syndromic surveillance system, 2017

Infectious diseases place a significant burden on global health accounting for an estimated 16% of deaths and tens-of-millions of healthy years of life lost, primarily in low-middle income countries [1‐3]. Despite this, many developing countries lack the resources, systems and infrastructure required to implement comprehensive multi-faceted early warning surveillance strategies for infectious diseases of outbreak potential; and hence, rely on relatively rudimentary syndromic surveillance methods for their detection [4‐6].

Solomon Islands (SI) (Fig. 1) is a Pacific island nation of 653,000 people [7] located in the south-west Pacific Ocean, approximately 1800 km north-east of Australia [8, 9]. Most of the population (78%) reside in rural areas and maintain village-based lifestyles [10]. With a per-capita gross national income of just USD1,561 and a United Nations Human Development Index ranking of 156 of 188 nations, SI is one of the least developed countries in the world [9, 11].

In 2011, while recovering from 6 years of civil unrest that resulted in health sector fragility [6] and recognising vulnerability due to the lack of a formalised routine outbreak early warning detection mechanism [12], the Solomon Islands Government (SIG) implemented a simple syndrome-based outbreak detection strategy known as the SI Syndromic Surveillance System (SI-SSS). Syndromic surveillance can be defined as ‘the real-time (or near real-time) collection, analysis, interpretation and dissemination of health-related data to enable the early identification of the impact (or absence of impact) of potential human or veterinary public health threats that require effective public health action’ [13]. The SI-SSS’s design is based on the broader Pacific-wide Pacific SSS (PSSS) [14]. The objectives of the SI-SSS are “to accurately detect outbreaks in the community quickly so that responses can be initiated promptly, and health impacts minimised” and “to support SI’s compliance with IHR (2005) obligations” The SI-SSS is described in Additional file 1.

The national health authority, the Ministry of Health and Medical Services (MHMS), faces complex challenges in delivering health care to the population. Challenges include a fragmented, fragile and under-resourced health sector with multiple and competing health priorities; the need to respond to the pressures of rapid urbanisation while also delivering care to remote communities; high vulnerability to natural disasters; and increasing rates of non-communicable diseases [15‐18].

Infectious and parasitic diseases remain marked contributors to the health burden in SI, causing an estimated 8% of all deaths [17] and more than 10% of disability-adjusted life years lost in 2015 [19]. While no log of outbreak events is available, it is understood that epidemics are common with a number of outbreaks challenging the country in recent years, including rubella in 2012/13 [20]; a rotavirus outbreak associated with > 4000 cases in 2014 [21]; a measles outbreak with 4563 cases in 2014 [16]; a Zika virus outbreak in 2015 [22, 23]; a cluster of meningococcal disease in 2015 [24]; and dengue outbreaks in 2013 [25] and again in 2016–17 [26] associated with more than 7000 and 12,000 suspected cases respectively.

In 2017, as part of an effort to strengthen the SI-SSS’s performance the MHMS requested a review of the system be undertaken. In this paper we present the findings of the first evaluation of the SI-SSS and the first exploration of a national experience within the broader PSSS [27]. We aim to determine how well the SI-SSS is performing in terms of its objectives and identify opportunities for improvement.

Drawing on seminal guidance produced by the United States Centres for Disease Control and Prevention [28, 29], the World Health Organization (WHO) [30, 31] and the European Centre for Disease Control [32] a mixed-method evaluation research frame was developed. The following system attributes were considered: outbreak detection sensitivity and false alert rate; timeliness; stability; data quality; representativeness; simplicity; acceptability and usefulness. These terms are defined in Additional file 2.

We interviewed 34 key informants including 12 (35%) nurses responsible for data collection, 18 (53%) MHMS staff with system administration/support and response decision-making roles, and four (12%) staff of agencies external to the MHMS that support the SI-SSS. Nurse respondents were drawn from seven of the ten sentinel health facilities that contribute to the SI-SSS; MHMS respondents included four senior executives, four SSS administer-managers, four laboratory staff and six associated public health program managers. Respondents from supporting organisations included two WHO and two bilateral donor agency staff. The median time respondents had been involved in the SI-SSS was 3 years (IQR: 1–3.5).

We report the findings of the first evaluation of the SI-SSS and explore the function of a national experience of the broader PSSS. We found that the SI-SSS provides an accepted, stable and highly valued surveillance mechanism suited to the resource-constrained context of SI, a point highlighted by others as important to maximise implementability and sustainability [5, 6, 27, 42‐44]. The SI-SSS is the primary source of information on which MHMS decision makers rely for the detection and monitoring of outbreaks. The system has made a notable contribution to addressing indicator-based surveillance core capacity requirements of the IHR (2005).

Fast and accurate detection of outbreaks are key attributes of a functional outbreak early warning surveillance system. Further, in resource-constrained settings, maintaining a high predictive value positive is important to ensure that the limited resources available for response are used appropriately. The SI-SSS’s outbreak detection mechanism is dependent on the application of a single algorithm which had not previously been validated. We found that the system was moderately successful at detecting large and explosive outbreaks and poor at detecting smaller and protracted events. The implications of these findings for SI (and other states that use similar approaches) is that reliance on an untested syndromic algorithm for the timely detection of small-moderate events may be ill-advised, and development of a broad suite of complementary surveillance activities, including event-, in-patient- and laboratory-based surveillance, is required. Further, we found the approach to be inappropriate for syndrome categories with small baseline counts as variation within these baselines were often larger than the size of the simulated outbreak, leading to a failure of detection. These issues are – in part – a symptom of the small patient throughput of health facilities in SI and therefore needs to be accounted for in the interpretation of information generated by the system. Adoption of a conservative static threshold approach to surveillance signal generation from such data may be more appropriate.

To further explore both the feasibility and performance of statistical signal generation methods we suggest more rigorous analysis that compares a range of syndromic algorithms be undertaken, and that resulting candidate approaches be verified through prospective field testing. Given outbreaks are typically detected locally, analysis should be conducted at the sentinel site level and, where possible, evaluated against data from other systems.

While the geographic and population coverage of the SI-SSS has improved, most rural populations remain outside of the system’s reach, and hence detection of outbreaks in these settings is likely irregular and delayed. Given resource constraints, the ability to expand the number of sites that participate in the SI-SSS is limited and, therefore, concerted effort to reinvigorate a relatively easily scalable and cost-effective EBS strategy is encouraged. This would require revision and promotion of the currently under-utilised national notifiable disease list and associated reporting protocols; establishment of reliable reporting mechanisms; and linkage of EBS with current IBS analysis, signal verification and response procedures.

Syndromic surveillance, as a stand-alone strategy, has significant limitations including a heavy reliance on professional judgement [4‐6, 45, 46]. We found domestic laboratory capacity to be insufficient to meet all public health needs, often requiring shipment of specimens to overseas laboratories for definitive testing, causing extended delays and rending resulting information of limited outbreak response decision-making value. A systems development approach to laboratory strengthening including strategies to build national and subnational capacity to test for priority pathogens through expanded use of RDTs and mobile molecular testing equipment (such as mobile polymerase chain reaction (PCR) or geneXpert systems); laboratory twinning; and upskilling surveillance, clinical and laboratory staff on infectious diseases will help address these limitations in SI. Development partners play a central role in supporting these actions.

The frequency at which data are collected and reported (and hence timeliness in which data can be analysed and used) is perhaps the aspect of the system’s design most amenable to improvement. The current protocol of weekly reporting of sites’ data was established in 2011 when the system was new, and principles of simplicity were paramount. Building on recent research that found SI nurses to understand the rationale for more frequent reporting and have a moderate-high willingness to comply with a more frequent reporting regime [47] modification to the frequency of reporting should be considered and trailed. Adoption of mobile technology that enables access to data collected as part of routine patient care (rather than collected solely for surveillance purposes) may be worth further investigation.

The small number of staff available to support SI-SSS is a critical factor limiting system advancement. Opportunities to capitalise on broader workforce investments (such as through in-country training, operational research and mentoring programs, or through participating in field epidemiology training programs) and achieve efficiency through integration of early warning surveillance with broader HIS initiatives should be explored.

To ensure health protection goals are met, it is imperative that measures to improve surveillance performance are balanced with efforts to strengthen response capacity, including addressing governance, human and other up-stream health system ‘roadblocks’. These will require outgoing investment preparedness planning and establishment of clearer event management arrangements that include designation of decision-making responsibilities. Further, establishing standard operating procedures for core surveillance and response activities; building the capacity of national and jurisdictional rapid response teams; and linking outbreak response structures with broader MHMS (and SIG) emergency management arrangements will enhnace the utility of the surveillance system.

The study observed that SI-SSS data collection currently sits outside the national HIS, and some data are being collected in duplicate. The recent introduction of the open-source cloud-based District Health Information System version 2 (https://​www.​dhis2.​org/​about/​) electronic HIS in SI offers unique opportunities for early warning surveillance to be integrated with, contribute to and draw on existing health informatics.

While we found the existing guidelines for early warning surveillance system evaluation useful as a starting point, we also found limitations. We found a lack of practical guidance for how to adapt the generic advice provided in the guidelines to the operational context of a developing country where measures of success are less quantifiable and where ‘adequate performance’ needs to be framed within what is realistic in the setting. We suggest supplementary guidelines are needed that explicitly address the methodological challenges faced when conducting surveillance system evaluations in complex, small data and limited resource settings.

This study has several limitations. First, the lack of complete historical data on which to base quantitative analysis required we rely on qualitative reports and modelling methods, both of which are prone to bias. Second, as sites’ SSS records are de-identified, the verification of data quality through comparison with medical records was not possible. Third, while we interviewed a broad cross-section of stakeholders, our sample was weighted towards those working in Honiara and hence we may not have comprehensively captured the views of staff from rural areas. Fourth, about one-third of interviews were conducted in group settings where interpersonal dynamics between respondents may have influenced participants’ willingness to contribute. Fifth, due to methodological and data availability limitations we were not able to assess all the attributes of a surveillance system, as outlined in the CDC guidelines [24].

While focused on to SI-SSS, we anticipate that the insights presented in this paper will be of value to practitioners working to strengthen outbreak early warning surveillance systems in other low-resource settings, in particular, in other Small Island Developing States [48].

In its 6 years of operation the SI-SSS has made a significant contribution to improving public health security in SI. Our evaluation found that the SI-SSS is performing well in certain attributes and highlighted priorities for system enhancement. We found that the SI-SSS provides a suitable mechanism through which the SIG has been able to meet part of their surveillance-related obligations under the IHR (2005). We conclude that syndromic surveillance, while a useful and a feasible surveillance tool suitable for limited resource settings, is insufficiently sensitive to small-moderate size outbreaks and hence should not be relied upon as a stand-alone surveillance strategy. Rather, the syndromic surveillance should sit within a complementary suite of early warning surveillance activities that include event-, in-patient- and laboratory-based surveillance methods.

Looking ahead, future investments need to find a balance between actions to address the technical and systems issues identified in this paper while maintaining simplicity. Where possible, investments should adopt a health system strengthening approach integrating with and capitalising on broader advances in HIS.