The utility of a shortened palliative care screening tool to predict death within 12 months – a prospective observational study in two south African hospitals with a high HIV burden

In this prospective, observational study, up to ten randomly selected patients of the daily acute medical intake on weekdays at each hospital were reviewed for study inclusion by a team of two trained clinicians at each hospital during consecutive 3 month periods (during Nov-Jan in one and during Feb–Apr in the 2nd hospital. All patients aged > = 18 and admitted for the first time in this time period as an acute, non-elective admission to the general medical wards could be included. Those refusing consent, not available in the ward on the day of sampling or having died before the day of sampling, were excluded.

Patients and patient notes were assessed within two days of admission by two trained clinicians who were not the primary care givers, using the identification sheet in Additional file 1 – a simplification of the Gold Standards Framework Prognostic Indicator Guidance (GSF-PIG) that has been used in these hospitals for two years as a clinician’s aid and referral tool. The tool includes the “Surprise Question” (SQ) – “would you be surprised if the patient were to die in the next 12 months”, and specific indicators of disease. The GSF-PIG general indicators were not included in the tool locally mainly to simplify the tool for use as a single one page referral tool. The tool allowed for an unspecified category (“Other”) in the indicators section, where the assessor would complete the reason/disease why death was expected. Traditionally, either a tick in the SQ box “Yes” or/and a tick in an indicator box would be considered to be a positive “identification” of a patient. Additional demographic data was collected from the patient, caregivers, electronic medical records and dispensing records, including previous diagnoses, previous medication history and a baseline Barthel Score. Investigations, including HIV screening and testing, were performed at the discretion of the attending physicians and with patient consent; an HIV-infected status was defined as any record of a positive HIV test (on formal ELISA) at any point prior to or during index admission dating back to January 2005 (the earliest available information on the National Health Laboratory results electronic system); an HIV-uninfected status as a formal negative laboratory ELISA during this admission or the 2 months preceding it; other patients were categorised as “HIV-unknown”. The electronic discharge letter was collected with final diagnosis based on ICD-10 coded primary diagnosis. Outcome mortality data were obtained from patient folders, the hospital electronic patient management system and the Western Cape Provincial death registry which links a unique patient identification number with national death certificate records and system wide electronic records. Patients for whom there were no reported mortality outcomes at the time of data extraction were assumed to have been alive at the end of the study.

Data were analysed in Stata 14·2 (Stata Corp., College Station, TX) and GraphPad Prism 8.02 (GraphPad Software Inc., California) for Kaplan Meier Graphs. Continuous data was summarized either as means and standard deviations or medians and interquartile ranges, and count data was summarized as frequency and percent. Socio-demographic, clinical characteristics and outcomes were assessed for differences between HIV-uninfected, and HIV-infected or HIV-unknown patients. Associations between categorical variables were analysed using the chi-square test and Fisher’s exact, as appropriate. Wilcoxon rank-sum or Kruskal-Wallis test was used to compare continuous variables between two and three groups respectively.

The study was approved by the University of Cape Town – Groote Schuur Hospital Human Research Ethics Committee (HREC 532/2017).

One thousand one hundred and six patients were randomised from the intake for potential enrolment into the study, of which 121 did not meet the inclusion criteria (73 were readmissions, 6 were under the age of 18, 8 were elective admissions and 34 were not admitted to the general medical wards). Another 163 were excluded (77 were not included in the data collection of the original main study due to its exclusion criteria of “coma” or “aphasia”, 22 had other incomplete data, 28 refused consent, 32 were not on the ward on the day of study and 4 died before testing) (Additional file 2. STARD Flow diagram). Of the 822 patients included in the analysis the median age was 52 (37–67) and 46% were male. The baseline demographics are shown in Table 1. Patients were admitted with a wide variety of diagnoses, of which the most common primary diagnoses were infection other than tuberculosis (19%), tuberculosis (12%), acute coronary syndrome (12%), stroke (8%), heart failure (8%), exacerbation of chronic obstructive airways disease (5%) and cancer (4%). Multi-morbidity was very common – 72% of patients had a previous chronic disease diagnosis requiring chronic medication before admission; 40% had 2 or more pre-existing chronic diseases. Patients were generally independent before admission; 76% had a Barthel Index of 100 and only 9% of patients had a pre-admission Barthel Index score of 50 or less.

One hundred and seventy eight (22%) of the patients were HIV-infected and 440 (54%) confirmed HIV-uninfected. In 204 (25%) patients HIV status was not known. HIV-infected patients were more likely than HIV-uninfected patients to be female and were younger (median age 35; 30–44). Amongst HIV-infected patients, the most common admission diagnoses were communicable diseases (tuberculosis and infection), with non-communicable diseases being far less common. Patients not tested for HIV were much older (median age 71; 60–78) and the disease profile was similar to HIV-uninfected patients, with a larger proportion being admitted for non-communicable diseases, particularly stroke.

Overall inpatient mortality was 5.1%, 3-month mortality 14.8% and total 12-month mortality 20.0%. 12-month mortality in HIV-infected patients was 21.3 and 17.3% in HIV-uninfected patients with a non-significant OR for 12-month mortality in HIV-infected patients of 1.30 (0.87–2.0).

The performance of the identification tool in predicting mortality at 12 months is shown in Table 2 for the population as a whole and divided by HIV-infected and -uninfected populations. In 144 patients identified by the SQ only (in whom no specific indicator box was ticked) the test had a good sensitivity of 89% (68–100%), but a very poor specificity of 8% (3–11%) and we did not analyse data using the SQ alone any further, but only the combination (of the SQ and the clinical indicators, as opposed to using the GSF-PIG Guidance of “or”) and an assessment based on only the specific indicators without including those only identified by the Surprise Question.

The combination of the SQ AND Clinical Indicators performed well; however when the indicators ALONE were used to predict outcome, without including the SQ, the best test performance was obtained with a similar sensitivity of 74% (71–83%) to the SQ and indicators combined, but an improved specificity of 85% (83–88%), a PPV of 56% (49–63%) and a negative predictive value (NPV) of 93% (91–95%). The indicator-only method of predicting outcome performed equally well in the HIV-infected cohort of patients as compared with the overall or the HIV-uninfected cohort. In HIV-infected patients sensitivity, PPV and Positive Likelihood ratio were better using the indicator alone vs indicator and SQ.

Using the indicator-only component of the identification tool, 218 of the 822 patients were “identified” (IDed) as being in the last year of their life (Table 3). There were no gender differences between “identified” and “non-identified patients” (non-IDed), though identified patients were older (median age 61 vs 49). A greater majority of patients “identified” presented with stroke, heart failure, COPD and cancer; less with an infection or tuberculosis. Less of the identified patients were HIV-infected, but more were HIV-unknown. The in-patient, 3-month and 12-month mortality for the “identified” vs the “non-identified” patients were 16% vs 1.3, 48% vs 3 and 56% vs 7% respectively.

Figure 1 shows the Kaplan Meier survival curve for the IDed vs non-ID patients. Survival was significantly worse for the “IDed” patients (p < 0.0001) with a hazard ratio for 12-month mortality for the IDed versus the non-IDed of 11.52 (7.87–16.9; p < 0.001). The tool strongly predicted mortality particularly in the next 3 months: 104 of 122 IDed patients (85%) vs 18 of 604 (3%) non-IDed patients died by 3 months after admission.

Figure 2 shows the total number of patients identified according to the specific indicator met, and the number and percentage that actually had died at 12 months. Patients were identified from all the indicators, but heart failure and respiratory disease were the most common, with neurological disease (predominantly stroke) with the second highest frequency. The main indicator that was not specified and where the assessor completed the “other” box was liver failure. 12-month mortality was best predicted for patients with renal failure (100% correct), cancer (90% correct) and worst for patients with respiratory/COPD (33%), dementia/frailty (40%) and liver failure (33%). Fifty four percent of patients that fulfilled the heart failure criteria and 58% of patients that fulfilled the AIDS criteria had died by 12 months.

In this cohort of patients admitted to acute medical wards in a country with a high burden of communicable and non-communicable disease and a high prevalence of HIV, a simple checklist of specific indicators (based on the GSF-PIG) was able to identify patients with nearly 12-fold increase in hazard of death within 12-month of admission. Diagnostic accuracy measures showed good sensitivity (74% (71–83%)) and specificity (85% (83–88%)), a low to moderate positive predictive value (56% (49–63%)) and high negative predictive value (93% (91–95%)). There was no significant advantage to adding the “Surprise Question” to the specific indicators to identify such patients in this study.

The high burden of non-communicable diseases reflect the mortality statistics of South Africa [20], and the young age of patients dying are typical of developing low and middle income economies. The prediction tool’s indicators performed equally well in a younger cohort of HIV-infected patients as in an older cohort of HIV-uninfected patients. Using this tool clinicians were better able to identify patients in the last year of life with renal failure (100% IDed) and cancer (90% IDed) than with heart failure (54% IDed) or respiratory disease (33% IDed). Other studies have previously shown that predicting mortality in organ specific disease such as heart failure or respiratory failure is more difficult for clinicians than for cancer [21]. The accurate prediction of death of patients with renal failure reflects the access to management in this hospital context – where acute dialysis is available, but chronic dialysis access is restricted [22], such that patients who died were likely largely excluded from chronic dialysis programmes..

This study is the first study evaluating the utility of an identification tool for predicting which patients may be in the last year of their life, in a low to middle income country with a high HIV burden. The use of record linkage ensured that outcome could be assessed objectively. The prospective enrolment of a large number of patients from the acute medical services across two hospitals, a tertiary and a large general hospital are typical of the types of patients admitted nationally and these findings are likely generalisable across South and Southern Africa and may be very similar in other low or middle income countries with a similar burden of disease. Further generalisability was ensured by patients being assessed by generalists, not palliative care specialists.

Limitations of the study were that it was only performed in acute medical admissions where patients are acutely unwell and may not be relevant to the primary care outpatient setting. Patients were also assessed within two days of their admission, when all information may not have been available to make an accurate assessment. Mortality as reported should be considered a minimum: vital status was not obtained for all patients; patients who have moved to another Province would not have been captured as deceased. Futhermore it should be considered that this tool does not necessarily predict overall palliative care need.

O’Callahan et al. [23] performed a similar evaluation of the (full) GSF-PIG tool in 501 patients admitted to a New Zealand teaching hospital, where the average age of patients admitted was 70 years, the major diagnosis was cancer and the 12-month mortality was 67.7%. In that setting the tool performed well and very similar to in our setting with a sensitivity of 62.6%, a specificity of 91.9 a PPV 67.6% of and a NPV of 90.0%. Another widely used prediction tool, The Supportive and Palliative Indicators Tool (SPICT) was validated on a cohort of 130 patients admitted acutely with organ failure to specialist beds. 48% of identified patients had died at 12 months, reasonably similar to our data here [16]. De Bock et al. [24] found a sensitivity of 84.1% and a specificity of 57.9% for SPICT in a retrospective cohort study in a geriatric population general and clinical indicators performed equally in that study. The predictive value of the NECPAL CCOMS_ICO© tool was evaluated in a prospective, longitudinal study in primary care centres and a hospital in Spain in 1057 patients, with a mean age of 81 years and a 12-month mortality of 27.0%. The sensitivity was 91.3%, specificity 32.9%, PPV 33.5 and NPV 91.0 [17]. The high number of false positives in comparison to our study may reflect the different nature of the patients or the tool itself (which includes the SQ, disease indicators but also general indictors of severity, disease progression, co-morbidity and resource usage).

There is a need to improve the quality of care for patients near the end of their life admitted to hospitals, and the recognition that a large number of patients admitted are in the final year of their life, and that clinicians have difficulty identifying such patients, have led to the development of simple “identification” tools to assist. Ideally such tools, if used in busy acute admission wards, need to be quick, simple to use and accurate, so that resources are allocated only as appropriate. This is even more relevant in low and middle income countries where the patient burden far exceeds the availability of clinicians. The simple one page tool is an example of how that may be possible. Importantly, inclusion of the Surprise Question offered no additional benefit above criteria specific indicators. The National Policy on Palliative Care for South Africa currently references both the SPICT and the GSF-PIG as potential tools for prognostication, It also suggests that a South African Palliative Care Needs Assessment Tool (SAPCNAT) is developed using a combination of criteria from both of the above-mentioned models [1]. This abbreviated GSF-PIG tool presented here is a practical alternative until a national validated tool is developed.