Figures
Abstract
Background
The Danish National Lymphoma Register (LYFO) prospectively includes information on all lymphoma patients newly diagnosed at hematology departments in Denmark. The validity of the clinical information in the LYFO has never been systematically assessed.
Methods
The coverage was tested by merging data of the LYFO with the Danish Cancer Register and the Danish National Patient Register, respectively. The validity of the LYFO was assessed by crosschecking with information from medical records in subgroups of patients. A random sample of 3% (N = 364) was made from all patients in the LYFO. In addition, four subtypes of lymphomas were validated: CNS lymphomas, diffuse large B-cell lymphomas, peripheral T-cell lymphomas, and Hodgkin lymphomas. A total of 1,706 patients from the period 2000–2012 were included. The positive predictive values (PPVs) and completeness of selected variables were calculated for each subgroup and for the entire cohort of patients.
Citation: Arboe B, El-Galaly TC, Clausen MR, Munksgaard PS, Stoltenberg D, Nygaard MK, et al. (2016) The Danish National Lymphoma Registry: Coverage and Data Quality. PLoS ONE 11(6): e0157999. https://doi.org/10.1371/journal.pone.0157999
Editor: Pei-Yi Chu, School of Medicine, Fu Jen Catholic University, TAIWAN
Received: December 29, 2015; Accepted: June 8, 2016; Published: June 23, 2016
Copyright: © 2016 Arboe et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be made publicly available for ethical reasons, since public availability would compromise patient confidentiality. Data are from the Danish National Lymphoma Registry (LYFO), the Danish National Patient Register (DNPR), The Danish Cancer Register (DCR), and The Danish National Pathology Register (DPR). Researchers can apply for data extraction when the request is approved by the Danish Data Protection Agency and The Danish Patient Safety Authority or the National Committee on Health Research Ethics. The Danish Health Authorities receives application regarding the DNPR and DCR. Data extraction from the DPR is performed by the DPR. Application for data extraction from the LYFO is sent to the Danish Clinical Registries.
Funding: PB received funding from the Danish Cancer Society (R-94-A5765-14-S21), https://www.cancer.dk. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors of this manuscript have read the journal's policy and have the following competing interests: BA: travel grant from SOBI, conference-invitation from Gilead; JSG: conference-invitation from Genzymes, Roche, and SOBI; PdNB: conference-invitation from Gilead and Roche, lectures for Roches and Bayer, and advisory board for Gilead and Janssen. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. This does not alter the authors’ adherence to PLOS ONE policies on sharing data and materials.
Introduction
Each year approximately 1,300 patients are diagnosed with malignant lymphoma in Denmark. An increase in the incidence has been observed for decades; from 15/100.000 in 2000 to 23/100.000 in 2013, in average an annual rate of 4.1% [1]. Lymphomas are heterogeneous diseases with more than 40 histologies identified in the current WHO classification. The diversity in the disease behaviour, from highly aggressive subtypes to the very indolent forms, has led to the development of several prognostic indices to enable an optimal prognostication of the different subgroups [2–5]. While randomized trials remain the gold standard for assessing the effect of an intervention such as new treatments, these studies are often characterized by strict inclusion criteria and do not necessarily reflect the outcomes of all patients.
Since 1942, the Danish Cancer Register (DCR) has collected information on diagnosis, disease stage and mortality from all new cancer patients in Denmark. Although Danish medical registries are generally known to be complete and accurate [6], they lack information on clinical and para-clinical characteristics as well as information on the specific treatment and outcome. The Danish National Lymphoma Registry (LYFO) was established in 1982 to monitor the quality of lymphoma treatment in Denmark. It contains information on clinical and para-clinical data at diagnosis as well as treatment details and outcome. In 2000 the database was implemented nationwide and now contains data from more than 23,000 lymphoma patients as a result of its high coverage [1,7]. The LYFO has been utilized for clinical research in several studies [8–11]; however, a systematic evaluation of the validity of the data has never been undertaken.
Population-based databases like the LYFO are powerful research tools for several reasons: i) data is readily available, ii) large number of patients is included, iii) they enable studies of rare lymphomas, and iiii) low cost. In contrast to randomized trials, the data are easily obtainable and can be utilized at minimum cost. The risk of bias is limited [12], however, the data collection and quality is not necessarily controlled by the investigator, and registry data should be validated before it is used for research purposes [13].
The main objective of this study was to evaluate the coverage of the LYFO and the quality of the entered data by crosschecking with the information from medical records.
Materials and Methods
The Danish hematology services are publicly funded and provide equal access to healthcare for all citizens regardless of social status or income. Patients diagnosed with lymphoma are referred to the nearest university hospital or local community hospital with a hematology department.
LYFO
Study population.
Clinicians treating lymphoma patients are required to report clinical, pathological, treatment, relapse, and progression data to the LYFO by using standardized case report forms. Table 1 shows the variables registered at different time points. Safeguarding against missing information is ensured by requests to the local departments in case patients with newly diagnosed lymphoma according to the Danish National Patient Register (DNPR) have not been reported to the LYFO or if information on critical data, such as treatment and/or relapse, is missing. Some of the critical data are validated against other Danish registries (i.e., the DCR and Danish National Pathology Register (DPR)) to confirm date of diagnosis, histology, and stage. The Danish legislation allows for this registration without the consent of the individual patient.
Central Registries
The Danish National Patient Register (DNPR).
The DNPR was established in 1977 and is currently run by the Danish National Board of Health. It has captured data on all hospital admissions since it was founded, and since 1995 data on outpatient visits have also been included. The variables in the DNPR cover administrative and clinical information, such as the unique civil registration number assigned to all Danish residents (CPR number), identification of hospital ward and the diagnosis at discharge [14,15]. A linkage to The Civil Registration System tracks changes in vital status and residential area daily. The CPR number allows unique linkage of records between all medical registries in Denmark.[16]
The Danish Cancer Register (DCR).
The DCR is a population-based register containing data on the incidence of cancer in Denmark since 1943. Reporting to the register was made mandatory in 1987. The register contains information on the date of diagnosis, topography, histological characteristics, and disease stage [17]. Since 2004, the data are captured electronically from the DNPR and from the Danish Pathology Register.
The Danish Pathology Register (DPR).
The DPR, established in 1997, has nationwide coverage and includes detailed information on all pathological specimens. To ensure correctness, a pathologist approves all diagnostic descriptions. Topography, morphology and specimen type is coded according to a modified version of the SNOMED classification [18]. The Danish Pathology Data Bank is accessible at most hospitals and offers instant updated nationwide information on pathological investigations and diagnoses [19].
Researchers can apply for data extraction when the request is approved by the Danish Data Protection Agency and The Danish Patient Safety Authority or the National Committee on Health Research Ethics. The Danish Health Authorities receives application for regarding the DNPR and DCR.[20] Data extraction from the DPR is performed by the DPR.[21] Application for data extraction from the LYFO is send to the Danish Clinical Registries.[22]
Methods
In order to evaluate the coverage of the LYFO, all patients recorded with lymphoma in the period 2000–2011 in the LYFO, the DCR and the DNPR were extracted. The coverage of the LYFO was tested against the DCR and the DNPR, respectively using the capture-recapture method.[23] Patients diagnosed in 2012 were excluded from the coverage calculation since a delay in the data delivery for the last year of diagnosis was observed when comparing the LYFO to the DCR and the DNPR. All pathology reports for patients not registered in the LYFO were reviewed in the Danish Pathology Data Bank to ensure the accuracy of the diagnosis.
To validate the quality of the data in the LYFO, patients diagnosed with lymphoma in the period 2000–2012 and recorded in the LYFO were used for a random sample selection of 3%. Patients with small lymphocytic lymphoma were excluded due to the overlap with the diagnosis of chronic lymphocytic leukaemia, and patients with the sole diagnosis of cutaneous lymphomas were excluded since these entities are usually treated by dermatologists in Denmark. Eleven variables of prognostic importance were selected for the validation of the LYFO. In addition to the random sample, a further 1,371 patients from ongoing studies of CNS lymphomas (N = 371), diffuse large B-cell lymphomas (DLBCL) (N = 164), peripheral T-cell lymphomas (PTCL) (N = 141)[24], and Hodgkin lymphomas (N = 695) were included for validation purposes. For these four subgroups, the date of diagnosis, histological subtype, ECOG performance status and Ann-Arbor stage were validated. Additional subgroup variables were included, e.g. treatment, relapse and other test results (Table 2).
Validations of these variables were done by local review of all individual medical files of the 1,760 patients (e.g. paper records, electronic records, DPR and clinical laboratory information system). Information obtained from these documents was considered as the “gold standard” to which data entered in the LYFO were compared. Variables selected for validation were labelled ‘consistent’ (if the local medical files and the LYFO had similar values), ‘inconsistent’ (if the local medical files and the LYFO did not have similar values), or ‘missing value’ (if data were missing in the LYFO or in the local medical file).
If a patient had missing data on one of the variables to be validated (information missing from the medical record or information missing in the LYFO), the patient was excluded from the validation analysis of that specific variable. Patients could only be validated once, such that if a patient was chosen for random sample validation, the same patient was not validated in the subtype validation.
Statistical methods
For the DNPR, the reference population was defined as patients recorded with lymphoma in the DNPR with ICD-10 codes C81.0–C88.0, C88.4, excluding C84.0-C84.1, C84.7-C84.8 and C86.6. For the DCR, the reference population was defined as patients recorded with lymphoma morphology codes 9590.3–9699.3, 9702.3–9719.3, excluding 9670.3.
Positive predictive values (PPVs) were estimated for variables in each subgroup and in a joined cohort. PPVs were calculated as the number of patients with correct registration divided by the number of patients registered [12,25].
Completeness of each variable was estimated for variables in each subgroup and in a joined cohort. Completeness was calculated as the number of patients registered divided by the number of patients with information regarding the variable in the medical record.
SAS 9.3 statistical software was used for the statistical analyses. Patient information was de-identified prior to analysis according to standards of the Danish Data Protection Agency.
Results
Coverage analysis
The LYFO contained information on 11,362 patients, whereas the DCR included information on 11,473 patients. Altogether, the two registries contained information on 12,234 unique patients. Seven percent of the patients (n = 872) were only registered in the DCR, and six percent of the patients (n = 761) were only registered in the LYFO. The remaining 10,601 patients were included in both registries. Among the 872 patients registered in the DCR, but not in the LYFO, 258 persons had no histology proven lymphoma and 614 had a biopsy-proven diagnosis of lymphoma. (Fig 1) Thus, the total lymphoma population in the surveyed time period reached 11,976, and the coverage of LYFO is therefore 11,362/11,976 (94.9%). A total of 436 patients were never referred to hematology departments, including 91 patients who were diagnosed post-mortem (autopsy). Therefore, the total number of lymphoma patients seen at Danish hematology departments, but not entered in the LYFO, was 178 (1.6%). Analysis of the 761 patients only registered in the LYFO revealed that the majority of patients had indolent lymphoma (Table 3). The patients were distributed equally across the period, however 16% (n = 120) were diagnosed in the last year of the study period.
The highlighted box (N = 178) represents the patients truly missing in the LYFO. *LYFO: The Danish National Lymphoma Registry.
Data validation analyses
There was a concordance within 6 months in the date of diagnosis in 10,445 of the 10,601 patients between the LYFO and the DCR (98.5%), while the remaining patients were equally distributed with 0.7% of patients having an earlier date of diagnosis according to the LYFO and the same number for the DCR.
Taken together, the DNPR and LYFO included a total of 12,940 patients with a lymphoma diagnosis in the study period, while the DNPR alone noted 12,680 patients and the LYFO 11,362 patients. Twelve percent of the patients (n = 1,578) were only registered in the DNPR, two percent of the patients (n = 260) were only registered in the LYFO, while 11,102 patients occurred in both registries. Of the 1,578 patients in the DNPR who were not registered in the LYFO, 737 were registered with a lymphoma in the DCR and therefore included in the previous description. Furthermore, 323 patients had other malignant haematological diagnoses leaving 518 patients with an erroneous lymphoma diagnosis in the DNPR.
Of the entire cohort registered in the LYFO in the period 2000–2012, a subgroup of 1,760 patients was selected for validation. A total of 54 medical records could not be found. These patients were excluded from the study. The 1,706 patients included were patients with CNS lymphoma (N = 370), DLBCL (N = 159), PTCL (N = 141), Hodgkin Lymphoma (N = 672), and a random sample (n = 364). No systematic pattern was found according to the missing medical records, hence there was no association between diagnosis in the LYFO, the hospital and the frequency of missing medical records.
Tables 4 and 5 show the variable completeness and PPV, respectively, for the joined cohort and for the subgroup validation. The values of completeness were generally high. Except for albumin (94.4%) and lactate dehydrogenase (LDH) (95.7%) the completeness ranged from 98.1% to 100%. The PPV ranged from 93.4% to 100% with the exception of albumin (87.1%). Ann-Arbor staging was not correct in 6.6% of the patients, which was even higher for PTCL (12.1%). The discrepancies were often seen in patients with limited disease, and in particular in patients with limited stage disease involving extra nodal sites where disagreements existed between the E designation and stage IV [26,27]. The definition of the date of diagnosis was correct in 94.5% of the cases. Patients with discrepancies often had multiple histological samples available, and in some cases, the date of the histology report was used as the date of diagnosis. The lymphoma subtype was correct in 98.5% of the patients, with 94.8% correct in the DLBCL subtype. The most frequent subtype error was in patients with follicular lymphoma grade III that was registered as DLBCL. Chemotherapy and immunotherapy data were not registered correctly in few patients. Patients receiving radiotherapy was not registered in 2.9% of the cases, however it was 7% among DLBCL patients. Relapse was registered correctly in 96.6%, and the patients for whom data were missing were often not treated for the relapse (data not shown).
Discussion
The current study presents an overview of registration practices, comparability, completeness and validity of data of the LYFO. The results of this evaluation show that the LYFO data have a high degree of both completeness and validity when compared to the DCR and medical records.
The completeness of the LYFO when compared to the DCR was 94.9%. Although 100% completeness is the ultimate goal of any register, 436 patients were never referred to a hematology department; more over 91 patients were diagnosed by autopsy. In general, all lymphoma patients are referred to a hematology department, but there is a variety of reasons for not referring some patients; 175 patients died within 30 days after the diagnosis (Fig 1). Among the remaining 170 patients, 107 patients had aggressive lymphomas including HIV-related lymphomas and patients with post-transplantation lymphomas, who were treated locally. Therefore only 178 (1.5%) patients were classified as truly missing in the LYFO, since they were referred to a hematology department. This corresponds to 1–2 patients every year per hematology department.
As a result of the capture-recapture procedure, we found 761(6.35%) patients in the LYFO that were not registered in the DCR. This underreporting of incident cases was primarily observed in patients with indolent lymphomas. A substantial part of these patients were followed without treatment, which may hide the patients from the capture algorithm of the DCR. The capture-recapture procedure also identified 258 patients in the DCR (2.2%) without a lymphoma diagnosis. In the DNPR, the figure was even higher at 518 (4.1%).
In a former comparison of the DNPR and the DCR, some underestimation and misclassifications were identified regarding hematological malignancies. In that analysis, the DCR was used as a reference standard, and the DNPR had a completeness of 91.5%. However, the pathological review revealed misclassifications in both registries [28]. This emphasizes the importance of a clinical registry. In the present study, 518 patients without lymphoma were registered in the DNPR as diagnose with lymphoma. A major advantage of a database like the LYFO is the clinical quality control of the data. To ensure that the patients fulfil the inclusion criteria for the registry, a clinician evaluates and validates all patients included and identifies those patients where lymphoma is not present. Most of these patients present with enlarged glands and biopsies have subsequently disproved the lymphoma diagnosis. The disadvantage with an automatized capture is the lack of data checking by clinicians and as a result, a proportion of patients without malignant lymphomas are registered in the DCR and the DNPR. The effort needed to find and since remove a patient with an erroneous malignant diagnosis from the DNCP or the DCR, is often substantial, leaving the diagnosis code unchanged.
Data from the LYFO are highly valid and have a high grade of variable completeness. Although at 87.1%, the PPV of albumin in Hodgkin lymphoma is lower, the other PPVs, which range between 93.4% and 100%, as well as the completeness, ranging between 98.1% and 100%, are high. A reason for the lower PPV for albumin could be that more effort is allocated to the clinically most important variables. Recently, the Danish National Acute Leukemia Register has been validated, and similar results were seen. Registration completeness compared to the DNPR was very high with a value of 99.6%. With variable completeness above 90% for 23 of 30 selected variables, and PPVs above 90% for 29 of 30, this register is also highly valid [29]. Furthermore, validation is ongoing for other Danish hematological registries.
The registration of a correct Ann-Arbor stage was found to vary between several of the subgroups. The discrepancy was particularly obvious when extra nodal disease was present. Even for experienced physicians, the application of a correct Ann-Arbor stage is a matter of debate, i.e. a stage IIBE patient can be interpreted as stage IV by others. Therefore a PPV of 93.4% is not surprising.
The strengths of this study are the extensive review of medical records covering a sample of four subtypes and a 3% random sample. In total our validation represents about 15% (1,706/11,362) of the patients in the database in the study period. We validated 11 different variables, selected due to their prognostic importance. We have no indication that other required variables in the database should be reported differently.
This study demonstrates the necessity of multiple sources to yield 100% coverage of all lymphoma patients. While frequently linkage can identify missing patients, there will be a need for optimising the PPV. One of the most frequent errors in data registration are transcription mistakes in the data entry process [30,31]. In the future it is planned that data is captured from electronic patient records. In the present study, clinicians identified patients without lymphoma, and the high validity of the data content was due to correct identification of the data. Direct data transfer will eliminate errors due to incorrect data transcription, but at the possible expense of incorrect data selection and therefore captured data from electronic records still need to be reviewed by clinicians.
This study also found patients in both the DNPR and the DCR that were registered with an erroneous diagnosis of lymphoma. With automatized procedures, this cannot be avoided, but emphasises the need for procedures that remove these patients, once they are entered in the DCR /DNPR.
Before using data from clinical databases for epidemiological studies, it is important to assure the quality of the data [13,32]. The LYFO has high coverage and contains high quality data with a large amount of detailed information on clinical and para-clinical data, treatment and outcome. Therefore, the LYFO is a valuable data source for epidemiological and clinical lymphoma research in the future.
Acknowledgments
We would like to acknowledge all patients diagnosed with lymphoma in Denmark, and the local data managers at participating centres for making this valuable registry possible.
Author Contributions
Analyzed the data: BA TWK PdNB. Wrote the paper: BA TCEG MRC MKN PSM DS TWK JHC JSG PdNB.
References
- 1.
Danish Lymphoma Group. LYFO annually report. lymphoma.dk. 2013; Available: http://lymphoma.dk/index.php?id=54,0,0,1,0,0
- 2. Zhou Z, Sehn LH, Rademaker AW, Gordon LI, Lacasce AS, Crosby-Thompson A, et al. An enhanced International Prognostic Index (NCCN-IPI) for patients with diffuse large B-cell lymphoma treated in the rituximab era. Blood. 2014;123: 837–42. pmid:24264230
- 3. Solal-céligny P, Roy P, Colombat P, White J, Armitage JO, Au WY, et al. Follicular Lymphoma International Prognostic Index Follicular Lymphoma International Prognostic Index. 2011;104: 1258–1265.
- 4. Hasenclever D, Diehl V. A prognostic score for advanced Hodgkin’s disease. International Prognostic Factors Project on Advanced Hodgkin's Disease. N Engl J Med. 1998;339: 1506–1514. pmid:9819449
- 5. Hoster E, Dreyling M, Klapper W, Gisselbrecht C, Van Hoof A, Kluin-Nelemans HC, et al. A new prognostic index (MIPI) for patients with advanced-stage mantle cell lymphoma. Blood. 2008;111: 558–565. pmid:17962512
- 6. Erlangsen a., Fedyszyn I. Danish nationwide registers for public health and health-related research. Scand J Public Health. 2015;43: 333–339. pmid:25759376
- 7. Mårtensson S, Frederiksen BL, De Nully Brown P, Dalton SO, Osler M. Differential Case Reporting in a National Clinical Quality Database. Qual Manag Health Care. 2012;21: 278–285. pmid:23011075
- 8. Wästerlid T, Brown PN, Hagberg O, Hagberg H, Pedersen LM, D’Amore F, et al. Impact of chemotherapy regimen and rituximab in adult Burkitt lymphoma: A retrospective population-based study from the Nordic Lymphoma Group. Ann Oncol. 2013;24: 1879–1886. pmid:23446093
- 9. Abrahamsson A, Albertsson-Lindblad A, Brown PN, Baumgartner-Wennerholm S, Pedersen LM, D’Amore F, et al. Real world data on primary treatment for mantle cell lymphoma: a Nordic Lymphoma Group observational study. Blood. 2014;124: 1288–1296. pmid:24859361
- 10. El-Galaly T, Mylam KJ, Bøgsted M, Brown P, Rossing M, Gang AO, et al. Role of routine imaging in detecting recurrent lymphoma: A review of 258 patients with relapsed aggressive non-Hodgkin and Hodgkin lymphoma. Am J Hematol. 2014;89: 575–580. pmid:24493389
- 11. Frederiksen BL, Dalton SO, Osler M, Steding-Jessen M, Brown PD. Socioeconomic position, treatment, and survival of non-Hodgkin lymphoma in Denmark—a nationwide study. Br J Cancer. Nature Publishing Group; 2012;106: 988–995.
- 12. Sorensen HT, Sabroe S, Olsen J. A framework for evaluation of secondary data sources for epidemiological research. Int J Epidemiol. 1996;25: 435–42. Available: http://www.ncbi.nlm.nih.gov/pubmed/9119571 pmid:9119571
- 13. Thygesen LC, Ersbøll AK. When the entire population is the sample: strengths and limitations in register-based epidemiology. Eur J Epidemiol. 2014; 1–8.
- 14. Lynge E, Sandegaard JL, Rebolj M. The Danish National Patient Register. Scand J Public Health. 2011;39: 30–33. pmid:21775347
- 15. Schmidt M, Schmidt S, Sandegaard J, Eherenstein V, Pedersen L, Sørensen HT. The Danish National Patient Registry: a review of content, data quality, and research potential. Clin Epidemiol. 2015;In Press: 449–490. pmid:26604824
- 16. Schmidt M, Pedersen L, Sørensen HT. The Danish Civil Registration System as a tool in epidemiology. Eur J Epidemiol. 2014;
- 17. Gjerstorff ML. The Danish Cancer Registry. Scand J Public Health. 2011;39: 42–45. pmid:21775350
- 18. Bjerregaard B, Larsen OB. The Danish Pathology Register. Scand J Public Health. 2011;39: 72–74. pmid:21775357
- 19. Erichsen R, Lash TL, Hamilton-Dutoit SJ, Bjerregaard B, Vyberg M, Pedersen L. Existing data sources for clinical epidemiology: The Danish National Pathology Registry and Data Bank. Clin Epidemiol. 2010;2: 51–56. pmid:20865103
- 20.
The Danish Health Authority [Internet]. [cited 24 May 2016]. Available: http://sundhedsdatastyrelsen.dk/da/forskerservice
- 21.
The Danish Pathology Data Bank [Internet]. [cited 24 May 2016]. Available: http://www.patobank.dk/index.php?ID=3&lang=da
- 22.
The Danish Clinical Registries [Internet]. [cited 24 May 2016]. Available: http://www.rkkp.dk/in-english/
- 23. Robles SC, Marrett LD, Aileen Clarke E, Risch H a. An application of capture-recapture methods to the estimation of completeness of cancer registration. J Clin Epidemiol. 1988;41: 495–501. pmid:3367181
- 24. El-Galaly TC, Pedersen MB, Hutchings M, Mylam KJ, Madsen J, Gang AO, et al. Utility of interim and end-of-treatment PET/CT in peripheral T-cell lymphomas: A review of 124 patients. Am J Hematol. 2015;00: n/a–n/a.
- 25. Bray F, Parkin DM. Evaluation of data quality in the cancer registry: principles and methods. Part I: comparability, validity and timeliness. Eur J Cancer. 2009;45: 747–55. pmid:19117750
- 26. Carbone PP, Kaplan HS, Musshoff K, Chairman PPC, Smithers DW, Tubiana M. Report of the Committee on Hodgkin ‘ s Disease Staging Classification Report of the Committee on Hodgkin ‘ s Disease Staging Classification. 1971;31: 1860–1861.
- 27. Crowther D, Lister T a. The Cotswolds report on the investigation and staging of Hodgkin’s disease. Br J Cancer. 1990;62: 551–2. Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1971503&tool=pmcentrez&rendertype=abstract pmid:2223571
- 28. Nørgaard M, Skriver M V, Gregersen H, Pedersen G, Schønheyder HC, Sørensen HT. The data quality of haematological malignancy ICD-10 diagnoses in a population-based hospital discharge registry. Eur J Cancer Prev. 2005;14: 201–6. Available: http://www.ncbi.nlm.nih.gov/pubmed/15901987 pmid:15901987
- 29. Ostgård LSG, Nørgaard JM, Severinsen MT, Sengeløv H, Friis L, Jensen MK, et al. Data quality in the Danish National Acute Leukemia Registry: a hematological data resource. Clin Epidemiol. 2013;5: 335–44. pmid:24039451
- 30. Wahi MM, Parks D V., Skeate RC, Goldin SB. Reducing Errors from the Electronic Transcription of Data Collected on Paper Forms: A Research Data Case Study. J Am Med Informatics Assoc. 2008;15: 386–389.
- 31. Paulsen A, Overgaard S, Lauritsen JM. Quality of data entry using single entry, double entry and automated forms processing-an example based on a study of patient-reported outcomes. PLoS One. 2012;7: 1–6.
- 32. Rosendaal FR. National registers and their use for medical research. Eur J Epidemiol. 2014;29: 539–40. pmid:25064617