Background
The encoded data of inpatient stays in Switzerland are submitted to the Federal Office of Statistics (BFS) on an annual basis for publication of epidemiological and economic health care statistics. Furthermore, they allow the classification of cases for reimbursement of acute inpatient care into Swiss Diagnosis Related Groups (Swiss DRG). Although the Swiss medical statistics of hospitals contains ICD-10 German Modification 2014 (ICD-10 GM 2014) diagnoses including flags for laterality and secondary codes for causation it gives no information on diagnosis timing [
1,
2]. Coding guidelines of 2016 in Switzerland include rules for encoding primary and secondary diagnoses as well as complications. Administrative data are increasingly used for disease surveillance, research, quality monitoring, case-mix costing (e.g. Swiss DRG), tracking healthcare performance and policy-making. Not only the publication of quality indicators by the Federal Office of Health (BAG) but also arising discussions on performance-based payment proposals show the need of a valid database [
3‐
6]. A variable for diagnosis timing has been introduced as “diagnosis onset type” in Australia, “present-on-admission” in the United States of America (US), “diagnosis type” Canada and recently as “present-on-admission” in Austria [
6,
7]. It is internationally recommended by the Word Health Organization (WHO) [
3] and on national scale by Initiative Qualitätsmedizin (IQM) [
8].
On behalf of the Centers for Medicare and Medicaid Services (CMS) in the US health care system the POA variable has to be reported since 2008. With effect on case-mix calculation it relates to hospital reimbursement claim [
9]. The introduction in Switzerland would be consistent with aims in national health care policy concerning quality [
10]. The experience gained elsewhere could serve the Swiss health care system to prepare the introduction of the variable POA into the Swiss medical statistics of hospitals. It is mandatory that quality measures accompany the introduction of SwissDRGs [
11]. The variable seems suitable for population based models or risk adjusted algorithms of quality monitoring [
12,
13]. In Europe some health systems enhance quality monitoring partially to support the introduction of reimbursement by DRGs in order to avoid adverse effects [
5,
14,
15]. In Germany and Switzerland the Initiative Qualitätsmedizin e.V (IQM) supplies quality indicators derived from routinely collected health data and peer reviews are initiated selecting cases for review using ICD coding [
16]. In Switzerland the Federal Office of Public Health publishes quality indicators for inpatient stays annually also derived from routinely collected data [
17]. The POA indicator could be used to further develop the indicators in order to ensure quality of care. With public provision (e.g. Scandinavian countries, Spain) and central financing (e.g. Italy central financing but regional management) economical pressure is lower [
15]. Nevertheless monitoring of health care indicators is implemented (e.g. Organisation for Economic Co-operation and Development OECD health care quality indicators including patient safety indicators) [
18]. International comparisons should take into account that there are variations between member states with regard to data sources and calculating of indicators. Some OECD member states use register data, some population surveys. Thus, data are collected as samples or from certain regions only. The OECD guidelines define that the data for several indicators should be based on diagnosis on admission. In some European countries quality standards are part of negotiations between insurers and hospitals and part of global payment for all patients rather than for reimbursement on individual patients’ level (i.e. Germany, the Netherlands) [
15,
19].
The resources of administrative data are limited. There are certain requirements to report on the complications of inpatient stays according to the Swiss coding guidelines, but both coded information on complications and other diagnoses do not sufficiently distinguish between being present at time of admission or not [
20]. As there is increasing interest in using routinely collected health data, the at present limited information on the time of onset needs investigation [
3,
5,
14,
21].
A survey conveyed by the Bern University of Applied Sciences, Bern, Switzerland, has demonstrated that different institutions and healthcare providers in Switzerland judge the current situation as insufficient. According to the results of the survey outcome analyses based on administrative data should be improved. Generally, valid information on the health status of a hospital’s patient population as well as on the hospital’s quality of care seems essential [
22‐
24], (unpublished data, see Additional file
1). The survey showed a high interest in quality monitoring and benchmarking by further developed administrative datasets.
By implementing the indicator POA in different health care systems worldwide the utility of administrative data for risk-adjustment could be improved [
5,
21,
25,
26], depending on the diagnostic quality and validity [
24,
25,
27‐
31]. Nevertheless, during the years since introduction in the United States different studies and reports aiming on validity, reliability and diagnostic value have shown a diversity of results depending on the type of hospital, on the diagnosis itself and on measures of improving documentation and also revealed problems of over-reporting [
23,
24]. Partially to avoid overreporting transparency has been introduced by publishing reports on case revisions [
32] and quality indicators [
16] in Germany and Switzerland. Automated screens for the plausibility of present-on-admission were developed to assess the accuracy of POA coding. The rate of secondary diagnoses, acute exacerbations, chronic diseases related to high-risk admissions, elective surgery or mortality derived from routinely collected data serve as a cost efficient method. Furthermore, with a set of statistical tests precision, bias, and validity can be assessed (e.g. variation, R-squared, correlation) [
33,
34]. Still, the reliability concerning different diagnosis groups has not been tested so far, but is relevant in respect to measures implemented consecutively. The variable POA is dependent on the prevalence of the underlying conditions and diagnoses and the associated coding guidelines as it is always applied additionally to a certain ICD code. Prevalence can be estimated using ICD codes coded in a hospital’s or a national dataset. With different national coding guidelines the prevalence of diagnoses differ, and subsequently that of the POA indicator. By Swiss coding standard inpatient cases are coded after the patient’s discharge retrospectively. Only those diagnoses which could be confirmed and caused resource consumption during the inpatient stay are allowed to be coded [
20].
Using administrative data based on ICD coding without knowledge of regulations might lead to misinterpretation as “In the absence of such a flag of diagnosis timing, researchers often assume that the main condition was present on admission, and then treat conditions coded in the secondary diagnosis fields as comorbidities for risk-adjustment.” [
4,
6].
In order to introduce the variable POA to assess the quality of care and the complexity of admitted patients the diagnostic quality has to be evaluated. The study we performed aimed on analyzing retrospectively the diagnostic quality and reliability of the indicator present-on-admission of a limited selection of diagnoses encoded in cases of the Department of General Internal Medicine, University Hospital of Bern, Switzerland in 2016. The applied indicator POA was validated by medical expertise based on the complete medical record. The medical expert served as Gold Standard as within the Swiss coding regulations the medical judgement and documentation serves as the final verification. Coders themselves are not allowed to derive diagnoses from findings and when disparities occur the final authority will be the physician involved in the patient’s treatment. Although medical experts’ judgement on diagnoses and findings vary and the reliability can be questioned as well, within the Swiss coding system they serve as final instance [
20].
The aim of the study was first to show that by attributing the indicator POA more information on the assessment of diagnosis timing can be obtained from administrative data compared to the limited requirements of the Swiss medical statistics of hospitals of 2016, second to analyze the diagnostic value and reliability of the indicator when encoded by coding specialists compared to the judgement of a medical expert and third to demonstrate the reliability and validity of POA when applied to different diagnosis groups.
An assessment at the Inselspital Bern with over 60.000 discharges annually showed, that for applying the POA indicator in addition to each coded ICD diagnosis (450.000 per year) would consume a budget of 1600 working hours. To reduce the load only specific diagnoses serving as indicators for quality could be chosen. The implementation of POA and using POA as a quality indicator or even as a factor in reimbursement (pay-for-performance) in different European countries needs a thorough discussion which should be based on studies as conducted here and should be based on an evaluation of costs.
Discussion
Comparing Swiss medical statistics of hospitals 2016 to present-on admission reporting guidelines
Analyzing the medical statistic of the selected cases and the resources of the coding guidelines it becomes evident, that without introducing further information the timing of the selected diagnosis cannot be determined. We can state that at present the medical dataset and the ICD do not carry sufficient information on complications and quality of inpatient care. Attributing the indicator POA to ICD diagnoses definitely gives more information on the time of onset of a diagnosis than the present resources of the Swiss medical statistics of hospitals and the Swiss coding guidelines of 2016. As the ICD catalogue is very limited itself concerning diagnosis timing, even further elaborated coding guidelines would not meet the need of an indicating information on each diagnosis.
Comparison of two coding methods discussion
Coding the POA indicator identified diagnoses which were pre-existent with a sufficient to good diagnostic quality but only moderate reliability. The inter-rater reliability was weak in group 3 “delirium”, but moderate in group 1 “deep vein thrombosis lower extremity” and group 2 “decubitus ulcer”. As the rating methods differed with regard to the documents used (method 1 based on the admission and discharge information and intervention reports; method 2 review of the complete patient’s report which included the whole process of treatment), the reason for this difference could be a loss of information. However, the difference of the ratio of positively rated diagnoses of all (POA/all) does not support this explanation. The medical reports had not been previously prepared for POA reporting. The study was conducted retrospectively and there was no training of documenting diagnoses at admission targeting on POA reporting. To avoid an over-reporting concerning present-on-admission and to facilitate the application of the indicator, the value “unknown” and “clinically undetermined” were reported as not present-on-admission (simplified POA Reporting Guidelines). Further investigations following this study with specifically prepared medical reports and instructions might achieve a complete database including the values “unknown” and “clinically determined” with a higher reliability. Nevertheless, the differences of the diagnosis groups might partly be due to the clinical relevance and explicitness of symptoms. The diagnosis delirium, decubital ulcer and thrombosis show specific manifestation time frames, specific severity of symptoms and a specific detectability. Also, the different diagnosis might be specifically susceptible to become a complication of inpatient treatment, with different observed incident rates.
The data supports the clinical situation with different counts of TP and TN, and different ratio of positively rated diagnoses of all (POA/all; TP/all) between the diagnosis groups with highest proportions detected in group 1 “deep vein thrombosis lower extremity” and lowest in group 3 “delirium”.
The diagnostic value calculated for all diagnoses is sufficient and meets the specification of diagnostic tests. However, the values of sensitivity and specificity as well as positive and negative predictive value differed between the diagnosis groups. We presume that the successful interpretation of symptoms as documented in the medical record following the POA Reporting Guideline “If the final diagnosis contains a possible, probable, suspected, or rule out diagnosis, and this diagnosis was based on signs, symptoms or clinical findings that were not present on admission, assign no “N” (i.e. not present-on-admission)” and “If the final diagnosis contains a possible, probable, suspected, or rule out diagnosis, and this diagnosis was based on signs, symptoms or clinical findings suspected at the time of inpatient admission, assign yes “Y.” (i.e. present-on-admission)” depends on the clinical impact of the diagnosis itself. As mentioned before, it influences the documentation in the medical report.
With regard to the diagnostic quality and the reliability we conclude that within the process of ICD coding designating diagnoses as being present-on-admission contains sufficient information to be used as an indicator (e.g. risk-adjustment). However, the use of POA as an indicator should be limited at present to specified groups of cases and should be evaluated for every target diagnosis in order to ensure validity. Considering pay-for-performance systems which aim on calculating a reimbursement rate for individual inpatient cases by use of the POA information, the only moderate to weak reliability must be improved [
23,
26]. The official Present on Admission Reporting Guidelines encourage coders to “query the providers when the documentation is unclear” [
35]. This explicit demand reveals the problems of designating the information. With this study, we omitted queries (there was no query of the documenting provider) and assigned the value “unknown” and “clinically undetermined” to not present-on-admission, in order to give an unambiguous impression of the current resources of POA reporting with no preparation of documentation beforehand and second to elaborate the inter-rater reliability. The inter-rater reliability can be improved by queries, which should be practiced when questions of accuracy of coding occur. But at this point using the two role-bound methods we tried to give an impression on the different results without interfering communication. The values suggest that monitoring quality of treatment and complexity of patient groups retrospectively by use of POA as an indicator is possible. However, with a weak reliability it should not be used for prospective quality measures or reimbursement of individual inpatient cases so far.
As the correct application of the indicator POA depends on a thorough examination at admission, on the quality and completeness of documentation in the medical report, on clear definitions and coding regulations and on the medical expertise of the coding staff, efforts in these fields should be undertaken to prepare for POA reporting in future [
22,
27,
29‐
31]). Although coding the patient’s record in hindsight influences the judgement upon symptoms at admission, the aim of a valid retrospective monitoring of the timing of diagnoses can be achieved. At discharge with the patients’ record completed, full information concerning all symptoms and diagnoses is accessible to the coder and thus leads to a verifiable decision. Using this method of applying the POA indicator enables institutions to evaluate retrospectively.
Also, as the prevalence of the underlying conditions and diagnoses varies and validity and reliability on the individual patient’s level could be questioned with insufficient reliability, the indicator should be embedded in more sophisticated approaches as for example the Australian automated Classification of Hospital Acquired Diagnoses (CHAD) [
12,
13] or the Potentially Preventable Complications (PPC) [
38] classification system which includes risk adjustment. The different prevalence has a relevant impact on the consequences resulting from false positive and false negative cases and from different values of the diagnoses groups, e.g. the diagnosis group “delirium” shows the highest prevalence but lowest PPV, (see Additional file
5).
Strengths and limitations
The strength lies in the quality of data, which is highly standardized by requirements of the Swiss medical statistics of hospitals and undergoes quality checks after coding. Therefore, the data is comparable to data of other tertiary care centers in Switzerland. This study is the first using data from Switzerland to analyze the diagnostic value of the present-on-admission indicator and one of the few worldwide calculating reliability in medical coding. It gives an incentive to further investigations, which are needed before implementing POA reporting in different European countries. Limitations of the study are the retrospective design with the documentation of the medical report not prepared for POA reporting as well as the small number of diagnoses especially as the diagnosis groups showed different results. As documentation in medical records is not standardized between the providers, the results might not be comparable [
39].
Conclusions
By these findings, in accordance with the development of requirements of administrative data in Switzerland and other countries, it can be concluded, that an implementation of the variable present-on-admission into the medical dataset should be evaluated.
The implementation would enable risk-adjustment, refined disease surveillance, research, case-mix costing and reports on quality indicators, for in-house, national and international analysis. However, these results as well as studies especially from the US point out, how important an assessment of the diagnostic value of the indicator POA and each diagnosis group is. Only when considering the validity for each diagnosis group the results of a future POA reporting should be used for monitoring and measures. The insufficient reliability reveals a further need to improve documentation and coding standards for the POA indicator in order to achieve high values on the individual patient’s level. The results show a need of standardization of documentation. To retrieve sufficient information from the documentation meeting the standard of a medical expert and to rise the reliability of coding, the diagnosis timing should be documented explicitly.
However, with regard to likelihood ratio the quality of the test seems sufficient as an indicator on population level. The differences of the results between the diagnoses groups emphasize the need to evaluate more diagnoses groups for diagnostic quality, covering more MDCs and different clinical conditions (i.e. following operation room procedures, lack of nursing, coagulation disorders).
The study shows that with rising costs in health care and rising requirements of quality of inpatient care the indicator “present-on-admission” offers the opportunity to retrieve essential information on diagnosis timing from routinely collected health data. A sensible approach would be a selection of diagnoses which are documented in literature serving as verified quality indicators (e.g. pressure ulcus) [
29,
30]. In many European health care systems encoding all inpatient cases is mandatory for national statistics and/or reimbursement (e.g. Germany, Switzerland). Coding the indicator POA for a certain set of diagnoses would mean a moderate additional effort. However, deriving information about the onset of diagnoses from the medical records demands detailed notes and a highly standardized documentation for all records. This demand meets legal requirements concerning liability, but will cause substantial effort.
If introduced with care, POA can connect analysis and research, resource allocation and quality of care based on standardized and comparable national and international administrative data.