Background
Routine cause of death data from civil registration and vital statistics (CRVS) systems are an important source of information and evidence for monitoring population health, identifying health priorities, and planning public health interventions [
1‐
4]. However, many countries have registration systems that cover only part of the population, with no cause of death data for those dying outside of health facilities, and no routine compilation of data for analysis, dissemination, and policy purposes. Furthermore, many hospitals’ cause of death data, derived from medical certification, are of such poor quality to be not useful for policy making [
5‐
10]. There are many reasons for the poor quality of death certification, including: i) absent, insufficient, or inadequate training on death certification and the concept of underlying cause of death, ii) limited understanding of the importance of death certification within the health sector, iii) an absence of doctors in some rural areas to certify deaths, iv) challenges in obtaining cause of death for external causes, v) doctors’ experience in the certification of deaths, and vi) limited availability of diagnostic equipment [
11‐
13].
In 2016, approximately 145,500 deaths were registered in Perú, with registration completeness estimated at 75% [
14]. Coverage is a major challenge for the country, with 89% of deaths registered in offices of the Civil Registry located in urban areas, and only 11% registered in municipalities, despite approximately one-quarter of the population living in rural areas [
15]. While 95% of medical certificates of cause of death (‘death certificates’) are certified by doctors [
16], international studies have shown that cause of death data from certificates are of limited use for country policy due to their poor quality [
6,
17,
18]. Such findings are also supported by Perú’s low Vital Statistics Performance Index (VSPI) score (50%). The VSPI scores the quality of data on mortality and cause of death by assessing several components within a CRVS system, including the extent to which age and/or sex are missing in the data, and the number of biologically implausible underlying causes of death recorded on death certificates [
19].
One reason for the poor quality of cause of death data in Perú is the poor quality of medical certification. Medical certification of cause of death is not a primary focus of the undergraduate medical curriculum in Peruvian universities, with the current curricula mostly focusing on prevention, and the recuperation and rehabilitation of patients. Many doctors do not understand the link between the causes they record on death certificates and national mortality statistics. Further, many doctors are not aware of standard death certification guidelines and practices. These factors all affect the quality of cause of death data reported in Perú [
20‐
22].
To address these issues the Ministry of Health (MOH) of Perú, with the help of the Bloomberg Philanthropies Data for Health Initiative, introduced two specific interventions to improve the completeness of death registration, and the quality of cause of death data. The first intervention, in August 2016, was to introduce an online death notification and certification system (SINADEF
1) to all health facilities and morgues. SINADEF is modelled on the system of online birth certification introduced by the MOH in 2012, which saw birth registration increase from 67% in 2011 to 84% in 2017 [
23‐
25]. The introduction of online death certification was also supported by evidence from other countries that had shown improvements in the completeness of death notification and registration following their implementation [
26,
27].
SINADEF was introduced to improve the processes of death notification and certification through providing a secure data entry and storage system. The system also improved the quality of cause of death data entered through: i) eliminating the effect of illegible handwriting; ii) reducing the use of ill-defined conditions by having a warning pop up dialog (for ‘cardiac arrest’ only); iii) not allowing for blank spaces to be left between the lines of events; and iv) decreasing the use of abbreviations, due to the high number of characters (300) allowed on each line. Given its online nature, SINADEF also improved the timeliness of data, as online certificates are available immediately to the MOH.
To ensure maximum benefit from SINADEF, the MOH also introduced a training program to doctors on medical certification, including how to complete the International Form of Medical Certificate of Cause of Death [
28] correctly. A selected team of trained doctors and other professionals trained a group of doctors and statisticians from the main hospitals and morgues in how to use SINADEF, as well as a group of regional staff to replicate the training in other health facilities.
The training lasted two hours: one hour on the online system and one hour for certification of cause of death. The training was limited to two hours because all health facilities considered it was the maximum time that doctors could be released from clinical duties. The following topics were covered during the training: the importance of correct medical death certification of cause of death; understanding the International Form of Medical Certificate of Cause of Death; the concept of underlying cause of death (UCOD); case scenarios to demonstrate correct certification practices; and SINADEF.
Previous studies have shown the effectiveness of knowledge gained in medical certification training based on pre-and post-training tests using clinical scenarios. However, most of these studies have been conducted as one-off trainings with little or no follow up, and literature on the effectiveness of medical certification training producing sustainable changes to death certification practices is rare [
29,
30]. The literature has shown the importance of periodically monitoring death certification accuracy to identify quality issues to provide feedback to certifiers and their administrators to improve them [
31,
32].
To assess the effectiveness of the two interventions introduced by the MOH to increase the quality of cause of death data, in this study we compare the quality of medical certification correctness between three groups:
1.
Paper death certificates completed prior to the two interventions (‘pre-intervention’)
2.
Death certificates entered in SINADEF by doctors (‘online intervention’)
3.
Death certificates entered in SINADEF by doctors who had also received medical certification training (‘online and training intervention’).
This study was carried out to evaluate the quality of cause of death data in Perú, as generated from medical certificates of cause of death completed by doctors. The results will provide feedback to doctors and their administrators about current routine medical certification correctness to identify areas that need further training, and provide recommendations to the MOH’s medical certification training program.
Methods
A random sample of 300 pre-intervention, 900 online intervention, and 900 online and training intervention death certificates, completed between January 2017 and September 2017 were selected for the study. Doctors were unaware they were being evaluated before and after the intervention. All online death certificates were obtained from SINADEF.
For the assessment, the tool
Assessing the quality of death certificates from the University of Melbourne was used [
33]. This tool is designed to assess the quality of death certification practices by checking for the presence of common errors in death certificates. The tool can also be used to assess the quality of death certification as part of routine assessment, or to assess the training needs of doctors in designing cause of death certification training and to evaluate the effect of death certification training. Errors were classified as major or minor errors, as shown in Table
1; and a summary error score was developed with two points applied to major errors and one point for minor errors.
Table 1
Classification matrix of major and minor errors assessed in death certificates
Major errors |
Multiple causes per line | The WHO ICD guidelines state that only one cause should be recorded per line in a death certificate. When more than one cause is reported on a single line, it makes it difficult for coders to establish the sequence of events leading to death, thus selecting the correct underlying cause of death would be more difficult |
Absence of disease time interval | The time interval should be entered for all conditions reported on the death certificate, especially for the conditions reported in Part 1. Time intervals are very important for correctly coding certain diseases and provide a check on the accuracy of the reported sequence of conditions. |
Incorrect sequence of events leading to death | Mortality statistics are based on the underlying cause of death, which is the condition or injury that initiated the sequence of events that led directly to death. When a clinically improbable sequence of events is recorded, it is impossible to select the correct underlying cause of death. |
Ill-defined or poorly specified condition entered as the UCOD | Ill-defined or poorly specified conditions are of no value for public health officials, and do not provide any information for decision-makers to help them design preventive health programs. These include, for example, organ failure (hepatic or cardiac failure, etc.); symptoms or signs (hematemesis, dyspnoea, fever, etc.); mode of dying (cardiac arrest, respiratory arrest); pathophysiological findings (shock); other (trivial diseases such as colds, rhinitis, etc.). |
Minor errors |
Presence of blank spaces within the sequence of events | In completing a death certificate the certifier should use consecutive lines in Part 1 of the death certificate starting at Line 1a. The UCOD should be recorded in the lowest used line of Part 1. There should not be any blank lines within the sequence/chain of events leading to death. |
Abbreviations used in certifying the death | Doctors are encouraged not to use abbreviations when certifying deaths as abbreviations can mean different things to different people. There is a chance that coders may misinterpret the abbreviation and code the death to a non-relevant code. |
Additional errors on the certificate | There may be other additional errors on death certificates including: incomplete information of the external cause of death (no site of the injury, intent or nature of it, etc.); insufficient information on neoplasms (no site, whether benign or malignant, etc.); failing to identify pregnancy and maternal deaths. |
The errors were categorized as major or minor based on the impact that the error can have on the final selection of underlying cause of death (UCOD) by the mortality coder. If the risk of mis-identification by coders is high, the error was classified as a major error. For example, recording multiple causes per line makes it difficult for coders to apply selection and modification rules for selecting the UCOD. On the other hand, presence of blank lines has a minor impact on the process of UCOD selection, and so was classified as a minor error.
Inclusion and exclusion criteria
The study was conducted in almost all departments (24 out of 25, one department was excluded because of an unreliable internet connection and doctors’ resistance to use the online system) that certify deaths using SINADEF in Perú. The study death certificates entered in SINADEF directly by certifying doctors, and certificates entered by trained statisticians. To minimize other factors that may influence the quality of certification, the selection excluded death certificates from doctors who were not the attending doctor for the deceased, and death certificates that were registered outside a health facility, as the doctors there may not have sufficient technology or information to correctly complete the death certificate.
Study procedures
Data from selected death certificates (online certificates completed by doctors and paper death certificates manually entered in SINADEF by statisticians) were downloaded. The data included general information on the deceased including age, sex, place and name of health facility where the person died; and certifier data such as doctor’s name and Medical Council registration number. The name and Medical Council registration number of doctors were matched with those from attendance lists at SINADEF training, to identify certificates for the ‘online and training intervention’ group. The name of health facility was matched with information from the National Health Superintendence (SUSALUD) to select the level of the facility (I-III). To complete the data about the doctor’s seniority and speciality, information from the Medical Council was used.
The evaluation of death certificates was carried out by one doctor with experience in death certification who received a training in using the death certification assessment tool that was conducted as part of the implementation of SINADEF.
Data analysis
Bivariate and multivariate analyses were conducted to assess the effectiveness of the three intervention groups. Bivariate relationships were analysed using chi-square and t-tests. The multivariate analysis conducted was an ordinal logit regression of the error score, to identify if the interventions had a statistically significant impact on the quality of death certification (measured by the error score), controlling for potentially confounding factors. The covariates in the model were study group status (pre-intervention, online intervention, online and training intervention), age group of the deceased, sex, doctor’s experience, doctor’s specialisation, level of health facility, and cause of death (as classified into Global Burden of Disease [GBD] group). Doctor’s experience and specialisation and the level of health facility may differ between the three stages of the study, and their inclusion in the regression removes any bias they may introduce into the results for the interventions. An ordinal logit is suitable when the dependent variable is ordered, such as the error score, which ranged from 0 to 9 [
34]. The ordinal logit regression produces odds ratios; these show the odds of a variable category having a higher error score compared with the reference category for that variable, controlling for all other variables in the regression. The data were analysed using Stata 15 [
35].
Conclusions
The study has demonstrated the significant impact that an online system can have on the quality of death certification, which in Perú will result in improved mortality statistics as the system is rolled out through the country. Online certification should be developed in other countries to improve not only completeness of registration, but also data quality. The lessons learned in Perú around the design and implementation of SINADEF hold important insights for other countries looking to implement similar systems: making key variables mandatory (for example, the time interval between disease onset and death, and not allowing blank lines when entering the sequence of events leading to death) would automatically improve the quality of death certificates. Similarly, having warning windows appear on-screen when a doctor attempts to enter an ill-defined condition as an underlying cause of death would likely lead to a decrease in such behaviour, and an increase in correctly coded mortality statistics.
The addition of medical certification training did not produce as large a change in death certification correctness as the introduction of SINADEF, possibly because the improvements due to SINADEF were relatively easier to achieve (i.e. inclusion of time interval on certificate). Again, countries wishing to improve the quality of their cause of death statistics can learn from the experiences of Perú and may look to: adapt their training programs to focus on certification errors that remain after the introduction of certification interventions (such as online systems or periodic reviews); introduce a master training program at the health facility level, where local trainers can provide training sessions on a regular basis; introduce certification training in medical curriculum, to ensure doctors receive adequate time on the subject as part of their education; and implement periodic death certificate assessments for further improvement of certification correctness. For the sustainability of periodic assessments, we recommend including training in assessing the quality of death certificates to a small team of health professionals in each department/health facility, so that they can do the follow up, bring feedback periodically, and retrain doctors. This team could analyse the main errors in the different health facilities of the department and change the content of the training for each place according to the results of the analysis.
In looking to the future for Peru, the next major step is to continue to roll-out implementation of Iris, the automated coding software. A team has been established within the Ministry of Health to develop a Peruvian version of the dictionary, and has re-coded 2016 data using Iris. Initial results have been very positive, with Iris able to code over 70% of medical certificates. The remaining certificates were analysed by a team of statistical personnel from the Ministry of Health and Directorates of Integrated Health Networks in Lima and Callao. In the coming months, 2017 data will also be analysed, as the country continues to move to electronic solutions for improving the quality of its mortality data.
Acknowledgements
William Valdez for revising the manuscript; Karim Pardo and the General Office of Information Technology (Ministry of Health) for implementing SINADEF in Perú.