Data quality of the routine health management information system at the primary healthcare facility and district levels in Tanzania

This cross-sectional study was carried out from October–November 2017 and involved 11 districts and all levels of primary health care facilities (i.e. dispensary, health centre, and hospital) in Tanzania (Fig. 1). A multistage sampling technique was used to randomly select 1–2 regions from each geographical zone [27] and one district from each region. List of facilities per district (registered and functional) were obtained from the Health Facility Registry (http://​moh.​go.​tz/​hfrportal). District hospitals were conveniently included. However, where there was no district hospital, a regional hospital was included. At least 50% of the health centres (HC) with a minimum–maximum criterion of 2–4, and 20% of dispensaries (min–max of 5–8) were randomly selected.

The study involved reviews of documents, information systems and databases and collection of primary data at facility and district levels. The source of data included facility-level registers, tally sheets and monthly summary reports (paper and electronic). Seven service areas namely Outpatient department (OPD), Inpatient department (IPD), Antenatal care (ANC), Family Planning (FP), Post-natal care (PNC), Labour and Delivery (LnD) and Provider-Initiated Testing and Counselling (PITC) were included in the assessment.

A total of 34 indicators selected from each of the service area (OPD = 5; IPD = 4; ANC = 8; PNC = 6; LnD = 2; FP = 6; PITC = 3) were assessed. The age category for the indicators followed the grouping used for reporting in HMIS tools. The selected indicators for OPD and IPD (both with age categorised as < 5 and ≥ 5 years) were anaemia (mild, moderate, severe) and malaria (confirmed malaria blood slide positive, confirmed malaria blood slide, clinical malaria). For the ANC the indicators were pregnancy (< 12 and ≥ 12 weeks of pregnancy, pregnant mothers who received tetanus toxoid, pregnant women who received intermittent preventive treatment for malaria, pregnant women who received first test of HIV, and mothers who tested positive for HIV during first test. The indicators for PNC were the delivered mothers who attended clinic within 48 h; who completed all the visits; who were diagnosed with severe anaemia; who acquired mental illness; and those who chose exclusive breast feeding. Indicators for LnD were women who delivered at health facility and those who delivered with the assistance of skilled personnel. There were 6 indicators for FP, namely number of clients for injection methods, number of clients receiving pills at health facility, inserting inter-uterine device, those screened for breast cancer, and those screened for cervical cancer. Indicators for PITC were the number of new clients, number of new clients tested positive for HIV and number of clients returned for counselling after testing. The selected indicators included those that were easy to collect, difficult to understand, difficult to compile, and takes time to compile.

The research team for each district comprised of four trained research assistants under the supervision of two senior researchers. Training was done before actual data collection and involved pilot exercise to ensure clarity on the HMIS tools (registers, tally sheets and reporting forms) and how they are supposed to be used, the type of data to be collected, as well as ethical issues when dealing with patient information.

The assessment considered and tracked data based on the order of events in the existing HMIS (referred to in this context as “data journey”). Primarily, patient data are recorded in registers at the time a client is been attended. The records are compiled at the end of each month to make a report, done in duplicate and separately for each service area. Tally sheets, designed for each service area with the same structure as the reported indicators, are used daily to track each record. The original monthly report is submitted to the district office. Used registers, tally sheets and carbon copies of all reports are kept at the facility for their use and future reference. The reports submitted to the district office are expected to be filed and organized and the data is later entered in an electronic system known as District Health Information Systems (DHIS2) for further analysis and use (Fig. 2).

At the facility, records of each indicator were tracked across all sources, i.e. physical counts from registers, records marked in tally-sheets and compiled totals in report forms. At the district office, original copies of monthly report forms submitted by facilities were reviewed. The filled records were compared to those observed in the carbon copies of reports (found at the facility), and to what has been entered in the DHIS2. The period of data assessment was January 2014 to September 2017 (45 months) with a detailed review covering 12 months. To select months for the detailed review, we fixed the last quarter of 2014 (October–December) and the first quarter of 2017 (January–March) (providing maximum time to resolve existing data issues). Then we conveniently assigned April–June 2016 and July–September 2015 to get an equal time interval (6 months) between the quarters.

At the facility level, tool availability was defined as the presence of filled/used HMIS tools (registers/tally-sheets/report forms by service area). At the district office, we tracked report forms from the health facilities for the selected 12 months. This tracking aimed to establish the fact that patient records were filled, tally-sheets were utilized, reports were compiled and submitted to the district office each month. The availability rate at the facility level was calculated as the percentage of observed registers/tally-sheets/reports out of those expected in a specified period. The availability rate at the district level was used to measure reporting completeness (percentage of reports received from HFs in a specified period). Completeness focused on recording, by reviewing the filling practices of registers compared to the provided instructions. Accuracy measured the numerical correspondence between data reported in one tool (subsequent) and that appearing in the source.

Data entry was done in EpiData 3.1 (EpiData Association, Odense M, Denmark), then migrated to STATA 13 (STATA, College Station, TX) for analysis. Quality check was done by comparing a random pick of 10% of entered data with the one in the original forms. We assessed data quality by year, service area, indicators, tool, facility level, ownership and district. Based on the total expected register-months, tally sheet-months and monthly reports, we calculated the median availability rates and the interquartile range (IQR). For the IQR we presented the 25th and 75th percentiles, and range (p75–p25) to assess variability in performance.

For accuracy, we calculated a difference ratio (an index measure that quantifies the difference between data sources) [8, 15, 18]. Six difference ratio (DR) indices, Diff1–6, were calculated, assessed, and grouped considering the three phases of the data journey, i.e. from the point a record is entered in a register at the facility to the point it reaches the DHIS2. The “health facility phase” focused on data activities occurring at the healthcare facility (recording to report compilation). The “transmission phase”, focused on report processing and communication (facility-district, district files-DHIS2) accounting for the revisions/corrections happening during this process. The “robust phase” compared data recounted from registers with totals entered in the DHIS2. With this phase, we present an ideal situation where all quality issues at the facility or during transmission are masked and data of the two extreme ends of the data journey is compared. This categorization facilitates systematic understanding, tagging quality issues and explore mechanisms to design practical-oriented data quality interventions stage wise.

DR < 1 indicates fewer data in the subsequent source, implying under-representation. DR > 1 means more data in the subsequent source, implying over-representation, while DR ≈ 1 implying consistency between sources. Matching levels were categorized into 5 groups (depending on the increase/decrease from DR = 1) and presented using colour-coded tables by different attributes. These were: (1) matched: 0.95 ≤ DR ≤ 1.05, i.e. acceptable difference  ± 5%; (2) moderately matched: 0.75 ≤ DR < 0.95 or 1.05 < DR ≤ 1.25; (3) moderately under-represented: 0.5 < DR ≤ 0.75 or moderately over-represented: 1.25 < DR ≤ 1.5; (4) highly under-represented: DR < 0.5 or highly over-represented: 1.5 < DR ≤ 2; and, (5) extremely over-represented: DR > 2. Statistical significance was tested using t tests and proportional tests, significance considered at p value < 0.05.

A total of 115 healthcare facilities in 11 districts were assessed. Both urban and rural districts were included in the study. The urban districts (and the number of facilities) were Dodoma (11), Igunga (10), Kahama (10), Kinondoni (18) and Njombe (10). The rural districts were Hai (10), Kibaha (8), Mbinga (10), Mbulu (8), Nkasi (10) and Tandahimba (10). Dodoma had no district hospital, therefore, the respective regional hospital was included. Due to a large number of private facilities in Kinondoni district, additional private hospital and two private health centres were included. Of the 115 HFs, 58.3% (n = 67) were dispensaries, 31.3% (n = 36) health centres and 10.4% (n = 12) hospitals. Of all the HFs, 114 had OPD, IPD (43), ANC (108), PNC (105), PITC (94), LnD (93) and FP (88) service-areas.

The overall median availability rate for registers was 91.1% (IQR 66.7%, 100%) compared to 77.8% (IQR 35.6%, 100%) and 86.9% (IQR 62.2%, 100%) for the tally-sheets and report forms, respectively (Table 1). HMIS tools were mostly available at the dispensaries 91.1% (IQR 60%, 100%) than health centres 82.2% (IQR 55.6%, 100%) and hospitals 77.8% (IQR 30%, 97.8%) (p value < 0.0001). Faith-based owned facilities had a significantly higher amount of tools available than the government and private-owned facilities (p value < 0.0001). The service-areas with high tool availability rates were ANC 95.6% (IQR 73.3%, 100%), FP 93.3% (IQR 66.7%, 100%) and LnD 91.1% (IQR 73.3%, 100%). PITC had the lowest rate 53.3% (IQR 20%, 88.9%). Hai, Kibaha, Mbinga, Mbulu (rural districts) had the highest availability rates while Kinondoni and Dodoma (urban districts) had the lowest (Table 1). A high variation in the range value was observed in the tally-sheet, indicating a significant difference in its utilisation between facilities. A remarkable increasing trend in the availability of tools with lesser variation between HFs was observed from 2014 (median = 83.3%; range = 100%) to 2017 (median = 100%; range = 22%) (Table 1; Fig. 3).

In terms of availability of tools, we categorised HFs into 4 groups: (1) > 75–100% (very high); (2) 50–75% (high); (3) 25–50% (average); and (4) < 25% (low). In all service areas, with exception of PITC, over 50% of the facilities were able to locate up to > 75% of the required registers (Fig. 4), with high percentages observed in the ANC (82.4%), OPD (74.6%), LnD (73.1%) and FP (72.7%) service areas. PITC registers were rarely available with only 45% of the facilities been able to locate > 75%. Over 15% of the facilities presented less than 25% of the expected PITC registers followed by PNC service-area (11.4%). Tally-sheets for ANC, LnD, and FP were available in larger proportions than those for IPD, OPD, and PNC (Fig. 5). Report forms were highly available in all service-areas except for PITC which had 42.1% of facilities providing ≤ 25% of expected report forms (Fig. 6). Urban districts of Igunga, Kibaha, Njombe, Kinondoni, and Tandahimba fell into average or low categories of tool availability particularly on tally-sheets.

At the district office, the overall median availability of submitted HF reports was 65% (IQR 48%, 75%) indicating that a third of expected report forms were not found. The district-specific performance indicated that less than half of the expected reports were found in the urban districts of Dodoma (median = 45%, IQR 25%, 51%) and Kinondoni (median = 46%, IQR 41–50%) (Fig. 7). Rural districts had higher rates: Hai (75%, IQR 67%, 82%), Igunga (74%, IQR 56%, 81%), Mbinga (73%, IQR 67%, 76%), Nkasi (72%, IQR 66%, 81%), and Mbulu (71%, IQR 71%, 86%). We matched availability of reports at district level and HF by service area for each of the four quarters included in the 12-month detailed review period. The findings indicate higher availability of reports at the HF level than at the district level with variations between service areas and HFs. Reproductive health service areas (PNC, LnD, ANC) performed better than other service areas. The PITC reports were difficult to find at both levels, especially at district office while the IPD reports were often missing. Challenges in transmission of reports, differences in programme reporting practices and weakness in the filing system at the district office were reported. There was no standard filing system of the received HF reports which hindered assessing if the report was received or not. In most districts, there was an increase in the availability of report forms over the years. However, the availability rate in the urban districts of Dodoma, Kibaha, Kinondoni and Njombe remained low during the period under review (Fig. 8).

Wrongly filled or empty cells in HF registers were common. Diagnoses were either not recorded or recorded without indicating disease severity (as instructed) or without laboratory results when available. This was common for malaria and anaemia. In the OPD registers, it was a common practice for patient’ height and weight variables to be left blank, and occasionally sex and age were not filled.

Poor adherence to the coding procedures was frequent. For instance, instead of using “N-Ndiyo” and “H-Hapana” (Kiswahili words for “Yes” and “No”, respectively), several records were in the English version of the words “Y-Yes” and “N-No”. In other situations, instead of using a “tick” mark as instructed when the service was provided, a recorder would use “N” or “X”, or leave the entries blank or use a different code that meant a different thing altogether. Consequently, this resulted in changing the meaning of that particular record. In some cases, health workers couldn’t remember the meanings of some of the codes they used. Such practices were reported to complicate compilation of the report, especially if a different person (from the one who did the recording) is compiling the report.

Improper use of carbon papers was observed in HFs, to the extent that it was difficult to identify the value recorded in the report form. The use of worn-out carbon papers was common and resulted into a blank or very faint report copies. Such poor recording practices led to differences between recounted and reported data, hence low accuracy performance.

Antenatal care service (ANC) At the HF phase, Diff1 indicates over 50% representation of data in tally-sheets while Diff2 shows extreme over-representation (of close to 3-folds) in the reports compared to registers’ records. A similar pattern was observed for Diff6 when registers counts were compared to the DHIS2 records. The transmission phase indicated consistency. The slight difference between Diff2 and Diff6 (with stable Diff3–Diff5) suggests that the reports transmitted to the district were manipulated (corrected/revised) before entered in the DHIS2, yet the changes were not documented. The over-representation levels decreased slightly over the years (Table 2).

The indicators for the provision of tetanus vaccine (TT2) and malaria intermittent preventive treatment (IPT2) performed poorly with lower registration than what had been tallied (> 2-folds), compiled, and reported (> 3-folds). This implies that there was intensive marking of clients in tally-sheets without registration. Indicators on gestation age and HIV testing for pregnant women were moderately matched in the tally-sheets and reports. The indicator for HIV testing among pregnant women < 25 years old was highly under-represented in all phases (Diff1 = 0.31, Diff4 = 0.14; Diff5 = 0.16 and Diff6 = 0.03). Data were found in registers but not reflected in the tally-sheets or report forms or DHIS2. There was variation in the ANC performance by districts in Diff1, Diff2, and Diff6 with much higher over-representation observed in Mbulu, Kinondoni, Kahama, and Nkasi districts.

Labour and Delivery service (LnD) At the HF level there was over-representation of data in the tally-sheets and reports as compared to what was recorded in the registers. In 2016, at the transmission phase, there was an over-representation of data in the district report as compared to the copy available at the facilities indicating that the reports were not comparable. There was no significant difference between Diff2 and Diff6. The Diff6 values decreased over time indicating an improvement in the data accuracy (Table 3). The first indicator had good matching levels at the HF phase. However, it was found to be revised at the transmission phase, thus more records were observed in the district copy than in the HF (source) copies, DR = 1.31. For the second LnD indicator, few data were observed in the registers than tally-sheets or reports. Although a large number of clients were indicated to deliver at the HF as marked in the tally sheet, almost none were marked of who assisted in the delivery. The values in a report for this period matched with those of who delivered at the HF. There was a little variation on the data quality performance by district on LnD.

Post-natal (PNC) service For the PNC, the quality of data, especially in the filling of tally-sheets and compilation, improved significantly over the years. However, the results indicated that sometimes the data journey was not followed hence resulting in larger DR at report/register (Diff2) than tally sheet/register (Diff1). Diff2 and Diff6 were very similar indicating that data management at the transmission phase does not influence the quality of PNC data (Table 4). Although about half of postnatal indicators performed well in Diff1, there were variations in Diff2. The first indicator (attendance within 48 h), had moderate over-representation, indicating that tally-sheets captured more attendees than those recorded in the registers (Diff1 = 1.30). However, data were extremely over-represented in report forms compared to data entered in the registers. The Diff2 of 3.09 implied that registers had less data compared to what was included in the summary reports. This indicates that reports were not compiled using data from the tally-sheets and cases were not recorded in the registers but somehow summed up in the reports. District performance in PNC differed highly in Diff2 and Diff6. Health workers reported some of the PNC registers and indicators to be difficult to understand.

Family planning (FP) service For the FP service area more data were found in the registers than in the tally-sheets (DR < 1). Comparing earlier years (2014 and 2015) against recent ones (2016 and 2017), an improvement was observed at the transmission phase (Table 5). However, the under-representation of data in tally-sheets did not improve. Overall, half of the indicators in FP services performed quite well with data presenting good matching between tally-sheets, registers and report-forms. An indicator on cervical cancer screening presented a DR less than 1 for Diff1 indicating more data were recorded in the registers than tally-sheets. The screening for breast cancer had a DR of 1.34 for Diff2 indicating that data were compiled in report forms but not indicated in registers. Most of the variations between district performance in FP were observed in Diff1.

Outpatient (OPD) service This service area indicated the highest levels of mismatch in the HF and robust phases. Diff1 showed moderate over-representation in tally-sheets versus registers, which improved significantly over time suggesting adoption on the use of tally-sheets. Extremely large Diff2 and Diff6 values were observed in 2014–2015. It was observed that records in the reports could be up to 5–7-times higher than register records, but was better in 2017 suggesting an improvement in client registration. The transmission phase performed well suggesting moderate manipulation of HF reports before been entered in the DHIS2. The variation in this manipulation between indicator, HFs, and districts is captured by the slight differences observed between Diff2 and Diff6 (Table 6). Individual OPD indicators did not perform well. Data obtained from registers were much less than in the report forms. The indicator of mild/severe anaemia performed worse with DR value indicating a difference of over 6-times between the register records and the report form. Blood smear positive records were corrected in the report forms by inflating values in the copy at the district, with a plus that the changes were documented (Diff3/Diff4 > 1.25). The performance varied between districts, with Kinondoni and Kahama having high levels of data over-representation.

Inpatient (IPD) service IPD was among the service areas that indicated a high level of mismatch between data sources, more particularly over-representation of data in the reports versus records in tally-sheets or registers. This was presented at the HF phase with Diff1 values of > 2 in 2014 and 2016, and Diff2 reporting extreme representation from 2014 to 2017 indicating fewer records in registers versus reported values. Diff1 improved slightly in 2015 and then significantly during 2017 which indicates a better use of tally-sheets. For Diff2 there was no indication of improvement observed during the 4-year period under review (Table 7). At transmission phase, an improvement was observed as the data matched better for the 2016 and 2017. Most of the IPD indicators presented differences between data sources of at least 50%. Data on severe anaemia was extremely over-represented in the tally-sheets and report forms compared to register records (Diff1 = 2.38 and Diff2 = 3.81). This implies that data were not found in registers but were marked in tally-sheets and recorded in the report forms. High difference between registers and reports were observed. There was high variation between district performance in IPD data accuracy mainly in Diff1 and Diff2 with Kinondoni and Dodoma under-utilizing tally-sheets and Mbulu over-representing data in the reports.

Provider-Initiated Testing and Counselling (PITC) service In this servicevarea, Diff2 went from moderate (in 2014) to extreme (in 2017) report over-representation indicating weakness in the registration process. PITC reports were highly manipulated during transmission phase before data was entered in DHIS2, marked by large Diff3/Diff4. There were significant variations between HFs and districts for Diff2 and Diff6 (Table 8). Little improvements were observed over the years. Indicator on ‘number of new clients’ was twofold over-represented between the register counts and the reported records and its data was corrected before been entered in DHIS2. Kinondoni, Kibaha and Nkasi showed high levels of data over-representation. Mbulu had matched data for Diff2 but high Diff6 indicating the submitted reports were revised before data were entered in the DHIS2 though, the changes were not documented. Nkasi had the highest Diff4 and Diff5 indicating corrections made during the transmission phase.

An overall annual pattern indicated a slight improvement on Diff1 (from 1.37 in 2014 to 1.26 in 2017), but a high improvement on Diff2 (from 2.61 in 2014 to 1.70 in 2017). This indicates that although tally-sheets were not fully utilized, the reports were better prepared in 2017 than in 2014. Similarly, there was a marked improvement in values for Diff6 from 2.72 in 2014 to 1.76 in 2017, indicating less variation between register’ records and DHIS2 entries over the years. Data accuracy by HF levels categorized by service areas indicates high Diff2 and Diff6 for hospitals, and in OPD, IPD, ANC, and PNC (Table 9). Data accuracy was observed to vary between HFs even within the districts.