An assessment of the accuracy and availability of data in electronic patient tracking systems for patients receiving HIV treatment in central Mozambique

In 2007, the population totalled 1.65 million in Sofala and 1.42 million in Manica with nearly 22% of the population based in the capitals of Beira and Chimoio [11]. At the end of 2009 an estimated 11.5% of Mozambique's population was infected with HIV. Central Mozambique had one of the heaviest HIV burdens in the country: 15.5% prevalence in Sofala; 15.3% in Manica [12]. In 2003, the first ART delivery site in the two provinces opened in Sofala's central hospital while a second one opened in 2004. To improve access to HIV care and treatment, the Ministry of Health integrated ART services into primary health care clinics distributed throughout Manica and Sofala provinces.

HIV-infected patients who enrolled into HIV care were assessed for ART eligibility. The initial visit included a documentation of demographic characteristics, a medical history/physical exam where patients were clinically staged according to WHO criteria and a blood sample was drawn for a CD4 cell count. Patients were then determined to be eligible for ART if they had a CD4 cell count below 200 cells/μL regardless of WHO Stage, a CD4 count below 350 cells/μL if in WHO Stage III or pregnant, or were in WHO Stage IV regardless of CD4 count [13].

Once a patient was eligible for ART, the patient was referred to a social worker for several sessions of pre-ART counseling, and ART was subsequently initiated. Three current first line regimens were used, each using fixed-dose combination pills. Triomune (D4T+3TC+NVP) was the first regimen of choice. Patients who were concurrently being treated for Tuberculosis (TB) were given Lamivir-S (D4T+3TC) and Efavirenz (EFV). Patients who were pregnant were given Combivir (AZT+3TC) and NVP [13].

After ART initiation, patients returned on a periodic basis for clinical and laboratory monitoring and on a monthly basis for pharmacy refills, according to national standards [13]. Clinic-based social workers also provided adherence support as needed. After a patient failed to return for ART medication refill, activistas (clinic-based peer counselors) were notified and began actively tracing patients. The peer counselors visited the patient's residence and encouraged them to return for treatment at the clinic if the patient was still alive. If the patient had died, the date of death was documented. These procedures were standardized across all clinics included in the study.

Mozambique's national health information system (HIS) operates much like other national data systems in the region and incorporates both paper and electronic components. As mentioned above, paper-based registries are utilized in health facilities to track the number of people coming in for services on a daily basis. Specific indicators are then tabulated on a monthly basis and aggregated into monthly facility reports. These reports are transferred to the district-level where the district planning and statistics department typically enters the monthly numbers by facility into the Ministry of Health's HIS, referred to as the Modulo Basico. Electronic files are then sent to the provincial level where they are pulled together and forwarded to the national level. Attempts to evaluate the completeness, reliability and validity of the HIS in this region are ongoing [14]. Mozambique's HIS has not been modified to respond to the growing need to have patient-level data systems for HIV care and treatment.

In Manica and Sofala provinces, HIV clinics utilized a Microsoft Access-based EPTS containing demographic, clinical and pharmaceutical data of patients enrolled into the HIV care and treatment system (Table 1). The implementation of EPTS into different clinics was a programmatic decision which factored in how long clinics had been open (those open longer tended to be prioritized) and whether they had a dedicated data technician. The database included an administrative file table containing demographic variables and socioeconomic factors and a health services file table that included pharmaceutical and clinical data. All patient-level data were data elements in the national HIV patient medical records and were also linkable through a unique patient identification number.

During the initial visit for HIV-infected patients enrolling into HIV care, demographic and clinical characteristics were recorded in the patient charts. In addition, a CD4 count blood specimen was sent to the laboratory, and after processing, the results were recorded in the patient's chart. When a patient initiated HIV treatment and for each subsequent pharmacy visit, the ART regimen, date of dispensing and next refill date were recorded. These data were first recorded in the patient's chart by providers and then were entered by the data technician into the EPTS via standardized data entry forms after the initial visit and after each of the subsequent clinic and pharmacy visits.

The clinics had standardized treatment and patient tracking protocols as all of them were managed by Mozambique's Ministry of Health with technical and logistical support provided by Health Alliance International. Databases were maintained by data technicians, while managers provided on-the-job training and quality control. Data technicians were located at the health facilities and were responsible for transcribing patient charts into the EPTS on a real-time basis. The EPTS had numerous range checks and cross-checks to enable immediate detection and correction of data entry errors. Database managers worked at the provincial-level and monitored data quality by running queries and troubleshooting with technicians if discrepancies existed. The managers circulated throughout the province to work with each of the technicians located at health facilities. In addition, the managers were responsible for producing quarterly, semi-annual and annual reports of program statistics.

To evaluate the reliability of the EPTS, we conducted a cross-sectional study among adult (≥ 15 years of age), ART-naïve patients receiving HIV treatment in 16 HIV care and treatment centers between July 1, 2004 and June 30, 2008 in Manica and Sofala provinces of Mozambique. Data from the EPTS had been previously compared to patient chart data for two of the 18 clinics and thus these two clinics were not included in this assessment [10]. We compared patient-level information contained in the EPTS with data abstracted from a total of 520 patient charts. Stratified, random sampling was utilized to select patient charts. Strata were defined by the clinic where patients currently received HIV treatment, and the number of patient charts targeted for a particular clinic depended on the size of the patient population receiving HIV care: 20 charts for clinics with < 200 patients, 30 charts for clinics with 200 to 499 patients and 40 charts for clinics with ≥ 500 patients. A total of 520 patient charts were surveyed. Assuming α = 0.05, a standard error of 0.0136 (based on observed data incorporating stratified sampling), our margin of error was ± 2.67% around an expected kappa score of 90%. Using a random list of patient identification numbers, trained data collectors abstracted demographic, clinical and pharmaceutical data from structured patient charts including gender, pre-ART CD4 count, date of CD4 count, first pharmacy pick-up date of antiretroviral medicines at treatment initiation, retention of the patient in HIV care and treatment (patient retention) and outcome date as described below. Pilot testing indicated that abstractors tended to make errors in identifying the correct pieces of information while blinded to the EPTS values due to the complexity of patient charts. Therefore, data abstractors were not blinded to the EPTS values. Patient retention was defined as patients who either remained in the program, suspended treatment per clinician's recommendation but remained in care or transferred to receive treatment at another facility. Patients who had died or were lost to follow-up for reasons other than death were considered to be not retained in the program. The patient's outcome date was classified as the last pharmaceutical refill date for patients who remained in the program or who were lost to follow-up for reasons other than death, and the date of death, suspension or transfer was used as the outcome date for those who had died, suspended or transferred.

In addition, we evaluated the availability of baseline data for all adult, ART-naïve patients who initiated HIV treatment for data commonly used in the study of attrition from HIV treatment programs: age, gender, education, weight, height, WHO stage and CD4 count [7, 8, 10, 15‐17].

Data from the standardized patient charts were compared to data contained in the EPTS. Several measures were used to assess the extent to which the EPTS accurately reflected the data in patients' charts for the sample of 520 patients. For binary variables, percent agreement and kappa scores were generated [18]. Concordance correlation coefficients were used to estimate the degree of agreement for continuous variables [19]. For dates, we calculated the percent agreeing exactly and the percent agreeing within 7 days of the date recorded in the patient chart. We calculated percentile bootstrap confidence intervals based on 1,000 replications for percent agreement, kappa scores and concordance correlation coefficients. The bootstrapping accounted for the stratified nature of our sampling for kappa scores and concordance correlation coefficients. To be consistent, we also used the approach for percent agreement. All analyses were conducted in Stata version 11.1 (College Station, TX).

The study was approved by the ethical review committees of the Mozambique Ministry of Health (Comité Nacional de Bioética Para Saúde; Reference: 337/CNBS) and the University of Washington Human Subjects Division (Reference: 30410).

A total of 16 of the 36 HIV care and treatment clinics recognized by the Ministry of Health from Manica and Sofala provinces were included in the analysis. Two of the 36 clinics were included in a previous assessment, and the remaining 18 were not included since they did not have electronic patient tracking systems. Two of these clinics were a vertical, stand-alone HIV care and treatment clinics, 14 were clinics where HIV care and treatment was integrated with primary health care services; and one of the vertical model clinics shifted into an integrated model. In addition, 7 (44%) of the clinics were located in urban settings. Four (25%) were considered to be hospitals while 12 (75%) were considered to be health centers.

The baseline characteristics of patients enrolling into HIV treatment between 1 July 2004 and 30 June 2008 are displayed in Table 2. Overall, 16,149 adult (≥15 years of age) patients enrolled for antiretroviral therapy in these 16 clinics with a median age of 33 years and a median CD4 count of 151. For these patients, the level of completeness was high for most of the variables with WHO Stage (9.9%), weight (11.5%) and height (18.6%) having the highest amounts of missing data. Of the 520 charts reviewed, 19 (3.7%) had missing CD4 counts and dates in the database, 7 (1.3%) also had missing CD4 information in the patient charts.

A total of 520 patient charts were reviewed to evaluate the reliability of specific variables in the electronic patient tracking system. All of the variables had a high level of agreement and reliability. Patient retention in the EPTS had the highest level of agreement [99.23% (95% CI: 98.46-99.81)] with a kappa score of 0.97 (0.95-0.99). The date of CD4 count at treatment initiation had the lowest level of agreement [92.61% (95% CI: 90.42-94.61)] while 97.6% (95% CI: 96.41-98.80) agreed within 7 days of the CD4 date located in the patient's chart. Among those records with discordant CD4 dates, the average difference was 27.05 days with a median of 1 day (IQR: 1-10). Of clinical significance, the value of the CD4 count at baseline also had high levels of agreement and reliability (Table 3).

Regarding the ART initiation date of patients, 97.88% (95% CI: 96.72-99.04) of the records agreed to the exact day while 98.84% (95% CI: 97.88-99.61) agreed within 7 days of the date recorded in patient charts. The mean difference for those with discrepant ART initiation dates was 26 days with a median difference of 13 (IQR: 2-26). The outcome date agreed 95.91% (95% CI: 94.35-97.47) of the time while 98.05% (95% CI: 96.88-99.02) agreed within 7 days of the date recorded in the patient's chart. For those patients with discordant outcome dates, the mean difference was 14 days with a median of 7 days (IQR: 2-17).