A cost-effectiveness analysis of three components of a syndromic surveillance system for the early warning of epidemics in rural China

The syndromic surveillance system in the ISSC project was an investigatory approach to detect outbreaks of diseases earlier and more comprehensively than might otherwise be possible with the existing infectious disease surveillance system in China. Diseases of concern in this syndromic surveillance system were acute respiratory infectious diseases (e.g., influenza, epidemic cerebrospinal meningitis, measles, chicken pox, mumps), as well as gastrointestinal infectious diseases (e.g., bacillary dysentery, enteritis, viral hepatitis, polio). The syndromic surveillance system in Hubei and Jiangxi was identical, and followed the same steps of operations which included the training of data collectors, data collection, data quality control, data processing and analysis, raw signal detection and signal verification. Data collectors were staff members of the surveillance units and they collected and reported data daily to an internal online database of the ISSC Project. Staff from the County CDC, as well as researchers in this project, assisted by automated data acquisition and generation of statistical signals, were in charge of data quality control, raw signal detection, and signal verification. Jiangxi and Hubei provinces shared a standard operation procedure of the syndromic surveillance system and training materials. Researchers from one university joined the above mentioned activities in Jiangxi, and those from another university joined in Hubei. The concrete operations of the syndromic surveillance system in Jiangxi and Hubei, especially the organizational approaches were different due to the differences of working approaches of the two universities and local staff. Surveillance data included 10-symptom data from health facilities (fever, cough, sore throat, nausea/vomiting, diarrhea, rash, muco-cutaneous hemorrhage, headache, convulsion and disturbance of consciousness), absenteeism data from primary schools (dates, numbers, ages, gender, classes, addresses, reasons for absence, and diseases or syptoms), and sales of selected medicines from pharmacies. All data collectors in Jiangxi Province were trained collectively by researchers from one university, and those in Hubei were trained by researchers from another universities More information on the ISSC project can be found in other studies [20, 21].

This study was performed in Hubei and Jiangxi Provinces, China, in the four study sites of the ISSC project. Table 1 presents the basic characteristics of the two provinces. The ISSC project committee chose the two provinces because 1) they were both more densely populated than low-income regions in western China; 2) both provinces were middle-and low-income regions in China; and 3) the local health authorities showed the willingness to take part in this project.

The sites were termed County A and County B in Hubei Province and County C and County D in Jiangxi Province. Each province had one county with higher annual income per capita of rural residents in 2011 (County B in Hubei: 8,049 RMB and County C in Jiangxi: 7,926 RMB) and smaller incidence of infectious diseases in 2011(County B in Hubei: 270/10⁵ and County C in Jiangxi: 307/10⁵) compared with its average level, and the other county with the opposite characteristics (County A in Hubei: 7,684 RMB, 625/10⁵ and County D in Jiangxi: 7,400 RMB, 668/10⁵).

Table 2 shows the numbers of surveillance units of the syndromic surveillance system in the ISSC project. County C and County D in Jiangxi Province had more surveillance units at township level than those in Hubei Province. The reasons are: 1) in general one town in County C or County D covered less population than that in County A or County B; 2) the ISSC project covered a similar size of population in each county (around 150,000 people), therefore covered more towns in County C and D; and 3) in general each town had one township hospital, one primary school and one to two pharmacies.

The sampling strategy of the ISSC project for the surveillance units were that at county level, one county level hospital and all registered county level primary schools were selected, and the number and locations of pharmacies was decided by the amount and distribution of pharmacies in each county, and that at township level, the ISSC adopted stratified cluster sampling with towns as basic sample units. In addition, sampled towns were adjacent, which was a requirement of the models in spatial cluster analysis. Therefore, it was not a randomized sampling. Willingness and capacity in performing the surveillance were also taken into consideration when finalized surveillance units for the ISSC project. This study adopted a non-randomized cluster sampling strategy following the ISSC project.

The cost analysis part was performed from the project managers’ point of view. All cost data were reported in Chinese currency (Renminbi; RMB), and costs were not discounted or adjusted for inflation due to the short time horizon.

Total costs included set-up costs (system development and training) and operating costs (data collection, quality control and signal verification). Cost data included staff time, the annual depreciation and opportunity costs of computers and overhead allocation. The staff time costs were measured by multiplying the number of full time employment equivalents with annual salaries and benefits, as suggested in several previous studies [16, 22‐24]. The costs of the depreciation and opportunity costs of the computers for the surveillance were measured by annualizing the computer purchasing costs according to their usage life and depreciation rate. Overhead costs were allocated to the surveillance with the percent of staff time as an allocation basis. More details on the cost analysis of the syndromic surveillance system were available in a previous study [17].

Primary data were collected during a three-month field trip from May to July 2013 with self designed facility and staff questionnaires, respectively, for the health facilities, the primary schools, the pharmacies, the county centres for disease control and prevention (County CDC) and the two universities who supported the implementation of the ISSC project. Facility questionnaires collected information on the size of population served, staff, computers, office space, income and expenditures, and the way in which the syndromic surveillance system had been implemented. The staff questionnaire for those from all the facilities was identical. It collected individual information on the daily working time, and the time spent on the surveillance both before and after the staff got familiar with the tasks involved in the surveillance. In total, we collected 369 valid facility and 477 staff questionnaires. Also, we collected daily timesheets of researchers from the two Chinese universities who facilitated the implementation of the surveillance in order to quantify time from the researchers’ side.

Data from the survey of our questionnaires were first entered into Epidata 3.02 (the EpiData Association, Odense, Denmark), and then exported to Stata (StataCorp LP, Texas, USA) for descriptive analysis.

The effectiveness was expressed by indicators of the outputs of the syndromic surveillance system, which were the number of reported events, raw signals and verified signals at, respectively, the health facilities, the primary schools and the pharmacies. Table 3 presents the definitions of those outputs. The number of reported events were basic data of the surveillance which were grouped into syndrome data. The syndromic surveillance system then based on those syndrome data detected raw signals for the early warning of infectious disease outbreaks. Verification of raw signals, either by checking data, telephone investigation, field investigation or laboratory tests, were followed to identify real outbreaks from background noises.

The reported events were collected from all the surveillance units by logging in the internal database of the ISSC project. The number of raw signals and their verification information were received from the local CDCs by collecting all forms for signal verification and also epidemiological investigation reports.

Reported events were imported from the ISSC internal database to Stata (StataCorp LP, Texas, USA) for descriptive analysis. Signals were sorted manually according to the content of the forms for signal verification and the epidemiological investigation reports.

Figure 1 presents the distribution of average numbers of daily reported events per surveillance unit, respectively, at the health facilities, the primary schools and the pharmacies in Hubei and Jiangxi Provinces. Huge differences existed within the three groups and also between Hubei and Jiangxi provinces: 1) the pharmacies on average reported the highest events per day per unit (P50 = 39 in Hubei, P50 = 17 in Jiangxi), with their median value dozens of times higher than that of the health facilities (P50 = 3 in both Hubei and Jiangxi), and of the primary schools (P50 = 0 in both Hubei and Jiangxi); 2) the daily number of reported events per unit within the pharmacies ranged widely in both provinces: (P25 = 22, P75 = 63 in Hubei, P25 = 10, P75 = 45 in Jiangxi), followed by those at the health facilities (P25 = 2, P75 = 4 in Hubei, P25 = 2, P75 = 6 in Jiangxi), and then those at the primary schools (P25 = 0, P75 = 1 in both Hubei and Jiangxi); and 3) the outliers of the numbers of daily reported events at the health facility group in Jiangxi province was unusual: among the 6 outliers which were extremely large numbers of daily reported events, four were from the village clinics, two from the two county hospitals but zero from the twelve township level hospitals. In general, the scales of the health facilities, the primary schools and the pharmacies decreased from the county level to township level and final the village level. It would not be surprising if extremely large numbers of daily reported events were mainly from upper levels such as country and township hospitals. It was unexpected that four outliers were from village clinics not township hospitals.

Raw signals detected by the surveillance system were mainly related to fever or influenza. Five signals were confirmed as outbreaks, of which two were chickenpox, two were influenza and the other one was mumps. Table 4 presents the numbers of signals detected by, respectively, health facilities, primary schools and pharmacies. The syndromic surveillance system detected 40 raw signals in total during the 15 months’ operation period. The signals detected had four main characteristics. First, the number of raw signals in Jiangxi Province was 9 times of that in Hubei Province. Second, the only four signals in Hubei Province were all confirmed signals generated by the data reported by a single county primary school, and the 36 raw signals in Jiangxi Province had one confirmed and 23 followed up. Third, flowing-up signals means that more attention were paid to gather further data from the surveillance units which had reported the abnormal data and generated the signals. The personnel of related surveillance units and the County CDC took preventive measures as reactions to the 23 signals followed up in Jiangxi Province. Those activities may have preventive effect but we were not able to quantify it. Fourth, the number of raw signals from the health facilities (which had 314 surveillance units) equaled that from the primary schools (91 surveillance units), and was 9.5 times of that from the pharmacies (53 surveillance units).

The set-up costs of the syndromic surveillance system in the four counties in Hubei and Jiangxi provinces were 1,459,169 RMB and the annual operating costs of the system in 2012 in the four counties was 1,086,455 RMB (the exchange of Chinese currency was 6.31 RMB for 1 US$ according to the World Bank in 2012 [25]). Table 5 presents the operating costs of the syndromic surveillance system. The costs of personnel time for data collection dominated the costs in Hubei Province, whereas the annual overhead costs were more than half the operating costs in Jiangxi Province.

The general government expenditure on health in the two counties in Jiangxi Province was around 220 million RMB according to our investigation. We did not have data on that in the two counties in Hubei Province. Nevertheless, we did a proxy comparison. The annual operating costs of the syndromic surveillance system in the four counties in Hubei and Jiangxi Provinces was around 0.45 percent of the general government expenditures on health in 2012 in the two counties in Jiangxi Province.

Table 6 shows the costs per reported event and the costs per surveillance unit in the three surveillance groups in Hubei and Jiangxi Provinces. As a whole, the three types of surveillance units in Jiangxi had both higher costs per reported event and higher costs per surveillance unit compared with their counterparts in Hubei Province. When checked the data by the types of surveillance units, the primary schools had the highest cost per reported event, followed by the health facilities and finally the pharmacies in both Hubei and Jiangxi Provinces. The pharmacies, in contrast to their lowest cost per reported event, had the highest cost per surveillance unit per year, followed by the health facilities, and finally the primary schools, also in both Hubei and Jiangxi Provinces.