Background
Methods
Search strategy
Selection of studies
Data extraction and analysis
Results
n | Percent | ||
---|---|---|---|
Geographical region | |||
East Asia and Pacific | 8 | 6.1 | |
Latin America and the Caribbean | 9 | 6.8 | |
Middle East and North Africa | 2 | 1.5 | |
South Asia | 15 | 11.4 | |
Sub-Saharan Africa | 98 | 74.2 | |
Year of publication | |||
< 2000 | 3 | 2.3 | |
2000–2004 | 7 | 5.3 | |
2005–2009 | 10 | 7.6 | |
2010–2014 | 40 | 30.3 | |
2015–2019 | 72 | 54.5 | |
RHIS data as source or to inform study | |||
Data source | 128 | 97.0 | |
Inform study | 4 | 3.0 | |
Types of study design | |||
Ecological study - cross-sectional | 13 | 9.8 | |
Ecological study - longitudinal | 51 | 38.6 | |
Ecological study - descriptive | 41 | 31.1 | |
Case study | 11 | 8.3 | |
Mixed methods study | 13 | 9.8 | |
Cross-sectional study | 1 | 0.8 | |
Pre- and post-intervention study | 1 | 0.8 | |
Nested clustered randomized controlled trial | 1 | 0.8 | |
Data use purpose | |||
Program evaluation | 67 | 50.8 | |
Epidemiology | 23 | 17.4 | |
Monitoring and assessment of service provisions | 30 | 22.7 | |
Program description | 6 | 4.5 | |
Impact evaluation | 4 | 3.0 | |
Cost estimation | 2 | 1.5 | |
Health conditions/service type | |||
General (multiple aspects) | 21 | 15.9 | |
Secondary health utilization | 2 | 1.5 | |
General causes of death | 1 | 0.8 | |
Maternal and Child health/healthcare | 12 | 9.1 | |
Maternal health/healthcare | 24 | 18.2 | |
Child health/healthcare | 11 | 8.3 | |
Vaccine prevented childhood illnesses | 10 | 7.6 | |
Malaria | 30 | 22.7 | |
Malaria & HIV/AIDS | 1 | 0.8 | |
Malaria & other parasitic diseases | 1 | 0.8 | |
HIV and related diseases | 8 | 6.1 | |
Mental health/healthcare | 3 | 2.3 | |
Other diseases | 5 | 3.8 | |
Healthcare workforce and other resources | 2 | 1.5 | |
Data issue of RHIS: missingness | |||
Described how missing data was managed | 33 | 25.0 | |
No description of how missing data was managed | 99 | 75.0 | |
Data issue of RHIS: outlier | |||
Described how outlier was detected | 19 | 14.4 | |
No description of how outlier was detected | 113 | 85.6 |
Types of disease and research purpose
Analytic methods using RHIS data
Data use purpose | Type of disease/service studied | Range of data (unit) | Level of aggregation | Analytic methods | Other information sources included | Reference |
---|---|---|---|---|---|---|
Time series analysis | ||||||
Epidemiology | Child health, malaria, tooth extraction | 15 (year) - 120 (month) | Ward, municipal, district | Time series correlograms; ordinary least-squares regressions adjusted for seasonality and lag; non-linear time series correlation and regressions | GPS coordinates, Climate Hazards Group Infrared Precipitation with Station Data, satellite data, meteorological department data, program data | |
Program evaluation | General, maternal and child health, maternal health, vaccine prevented childhood illnesses, malaria | 5 (year) - 168 (month) | Facility, district, region, nation | Ordinary least squares regression; negative binomial generalized linear model; random effects negative binomial regressions; switching regression methods weighted by propensity scores | Program data, program reports, data from Bureau of Statistics and Ministry of Health, Malaria Indicator Survey, Demographic Health Survey, Health Facility Survey, community survey, satellite data, sentinel site case-investigations/surveillance, abstraction from hospital registries | |
Impact evaluation (non-program) | General | 84 months (month) | Facility | Linear mixed-effect time-series analysis with a segmented regression parameterization | None | [82] |
Interrupted time series analysis | ||||||
Program evaluation | General, maternal and child health, maternal health, malaria | 53 (month) - 132 (month) | Facility, intervention vs. control groups, district | Generalized least square model with autoregressive structure; generalized least square model with controls, with autoregressive process and moving average process; segmented linear regression | Meteorology Department data, program data, facility survey | |
Impact evaluation (non-program) | Maternal and child health | 44 (month) | District | Segmented linear regression with district fixed effect and clustered standard error at district level | Demographic Health Survey | [68] |
Difference-in-difference analysis | ||||||
Program evaluation | General, child health, maternal health | 4 (year) - 48 (month) | Facility, district, province | Ordinary least squares regression with and without propensity score matching; Wilcoxon rank-sum test on median difference-in-differences between facilities; descriptive comparison of means | Verified data from Performance-Based Financing system | |
Pre-post comparison analysis | ||||||
Program evaluation | Child health, maternal health, maternal and child health, vaccine prevented childhood illnesses, malaria, HIV or related diseases | 2 (year) - 48 (month) | Facility, district | Chi-square test; Pearson correlation; Wilcoxon signed-rank test; paired sample t-test; linear regressions; Poisson regression; negative binomial regression; logistic regression | Bureau of Statistics data, program reports, Meteorological Department data, entomological sentinel surveys, Demographic and Health Survey, UN Interagency Group for Childhood Mortality Estimation(CME Info) database, abstraction from facility registers, community surveys, vital registry, provincial maternal death notification register | |
Impact evaluation (non-program) | Child health | 26 (month) | District | Pearson chi-square test | District hospital registers, Safe and dignified burials for all deaths database | [67] |
Other longitudinal analysis | ||||||
Epidemiology | Maternal health, malaria | 12 (year) - 16 (year) | District | Chi-square test; negative binomial regression | Review of hospital death records | |
Monitoring and assessment of service provision | HIV or related diseases | 3 (year) | District | Descriptive comparison over time | Surveys with health facility managers | [96] |
Program evaluation | Genera, child health, malaria, malaria and other parasitic diseases | 3 (year) - 24 (month) | Facility, district, nation | Poisson regression to explore association between intervention coverage and disease burden; Mann–Whitney U Test to compare prevalence in intervention and non-intervention area; linear regression model; student t-test | Sentinel surveillance data, program reports, national facility and community survey, Bureau of Statistics data, program data | |
Geostatistical analysis | ||||||
Epidemiology | Child health, malaria, malaria and HIV/AIDS, meningococcal meningitis | 1 (year) - 520 (week) | District | Cluster analysis; cross-correlations of different spatial scales between time series of cases; Bayesian hierarchical Poisson model and smoothed model estimates plotted on district maps | Malaria Indicator Survey, Demographic Health Survey, program data | |
Monitoring and assessment of service provision | Malaria, maternal health | 1 (year) - 57 (month) | Facility, district | Kriging (ordinary kriging, space-time ordinary kriging, local space-time ordinary kriging); Bayesian geostatistical negative binomial model | Service Delivery Indicator Survey | |
Program evaluation | Malaria | 36 (month) | District | Bayesian geostatistical models and Bayesian generalized linear models | Malaria Indicator Survey, malaria control program data, satellite data, Demographic Health Survey, ACTWatch household surveys | [110] |
Other cross-sectional analysis | ||||||
Epidemiology | Maternal health | Median of 24 months | Province | Linear regression model | None | [111] |
Monitoring and assessment of service provision | General, child health, maternal health, mental health | 1 (year) | Facility, district, municipality, state | Descriptive statistics, Tobit regression model, bivariate and multivariate linear regression models, | Nutrition Service Delivery Assessment, abstraction from Integrated Nutrition Register, structured questionnaire with district health officers, District-level household and facility surveys, National Register of Health Service Providers, data from Institute of Geography and Statistics | |
Program evaluation | HIV and related diseases | 1 (year) | District | Mixed-methods | Register reviews and a series of patient folder (health record) reviews | [51] |
Time series analysis
Geostatistical analysis
Pre-post comparison analysis
Interrupted time series analysis
Difference-in-difference analysis
Impact of research using RHIS data
Strategies to circumvent RHIS data quality issues
Type of strategy | Description of strategy |
---|---|
Missing data | |
Exclusion | Exclude facility data if a certain threshold was reached (e.g. more than two-thirds of months in a year; more than a sixth of baseline data; facilities with any missing data) |
Restrict analysis to a period with a low level of missing data | |
Sensitivity analysis to compare analysis of restricted period and full period | |
Imputation | Assign missing observations with mean-value for the year |
Assign missing observations with the average of precedent and subsequent data | |
Imputation using conditional autoregressive model | |
Missing value was replaced as positive (binary form) to prevent exaggeration of the fade-out effect | |
Sensitivity analysis of imputation strategies: 1) single imputation using means, trimmed means, and median, 2) Poisson generalized linear modeling, 3) iterative singular value decomposition method | |
Interpolation | Interpolation using space-time kriging |
Adjust results by dividing each indicator by the percentage of reports submitted | |
Adjust the data by calibrating to the total population using proportion reported in a household survey to have occurred in health facilities | |
Verification Account in the modeling method | Manual verification of the missing data with register at the health facility |
Missing data was assumed missing at random and accounted for in the mixed-effect models using standard maximum likelihood estimation | |
Identifying extreme values | |
Specific threshold | Establishing a lower and upper limit based on proportion of the annual average or feasible value |
Univariate regression on individual facility-level to identify deviation from the mean time trend (e.g. if exceed 8 standard deviations) | |
Visual | Visual inspection of outliers |
Analytic assessment | Jackknifing analysis to assess influence |
Student residual higher than an absolute value of 2 and influence on the estimated coefficients determined by high Cook’s distance statistics | |
Handling of extreme values | |
Exclusion | Extreme values were excluded from analyses |
Replacing extreme value with average | Extreme values were assigned the average value of the year; with exceptions of low average values |
Replacing extreme value with missing | Outliers set to missing |
Verification with data source | Any drastic change in monthly data reported electronically were manually verified with register at the health facility. Discrepancies were replaced with data in the register |
Discount observation in estimation | Outliers were allocated a dummy coding to discount the observation in the calculation of coefficients |
Assess reliability | |
Data validation process | Randomly selected 10% of the total sample to check accuracy and reliability of data with reports and registers |
Verify data with another source (e.g. payroll) | |
Established routine data validation process by health information and records officer (e.g. monthly data review meetings) |