Reliability of predictive models to support early decision making in the emergency department for patients with confirmed diagnosis of COVID-19: the Pescara Covid Hospital score

Between December 2019 and April 2021, the coronavirus SARS-CoV2 affected at least 130 Million people diagnosed with COVID-19 in 223 different countries, of which almost 2.9 Million died [1]. The range of complications associated with the disease generated enormous pressure on hospitalisation and intensive care [2‐4].

Clinical symptoms range from pauci-symptomatic states presenting fever, cough and fatigue, to more severe forms including acute respiratory distress syndrome (ARDS) and/or other critically severe conditions [5]. Mortality is higher in the elderly, when the disease is at an initial stage and the virus replicates. In some cases, this activity triggers a second aggressive phase in which hyper-inflammation may require immediate urgent hospitalisation [6]. Patients admitted with these characteristics show a 20-fold increased risk of death compared to non-severe cases [7].

Timely identification of COVID-19 patients at higher risk of severe complications may enable early hospitalisation, risk-tailored clinical treatment and optimal allocation of human and technical resources [8]. In these circumstances, a predictive risk tool may conveniently identify subjects requiring immediate attention, serving as a reference in the relative lack of evidence-based guidelines. A variety of methods have been used to identify subjects with a higher risk of target outcomes e.g. COVID-19 diagnosis, hospital admission and negative prognosis. However, their direct applicability in clinical practice appears still limited [9‐11].

The triage of COVID-19 patients requires multiple parameters to be assessed by a multidisciplinary team at presentation to the Emergency Department (ED) [12]. Characteristics that need to be routinely evaluated include clinical signs and symptoms known to be associated with subsequent prognosis [9] and various laboratory measurements e.g. decrease in albumin and increase in C-reactive protein (CRP), lactate dehydrogenase (LDH), lymphopenia and other haematological parameters that have been less investigated in clinical settings [9, 13].

In the framework of everyday care provided during the first wave of the outbreak, our aim was to identify patients diagnosed with COVID-19 with a higher risk of four key outcomes: hospital admission, mechanical ventilation, admission to Intensive Care Unit (ICU) and death.

The present study is a retrospective review of all consecutive cases presented at the ED over 3 months, addressing the following key research questions:

A total of 536 consecutive admissions with a positive molecular assay to SARS-CoV2 were recorded at the ED in the reference period. The samples collected for validation were equal to 224 out of 639 (35%) for December 2020 – January 2021 and 375 out of a total of 872 (43%) for January–March 2022.

Among subjects hospitalised in the baseline study sample, a total of 174 consecutive admissions involved patients aged less than 70 years. The samples used for validation were equal to 95 out of 226 (42%) for December 2020 – January 2021 and 56 out of a total of 135 (41%) for January–March 2022.

All details of the association including risk ratios and 95% confidence intervals for the total population admitted to the ED at baseline and those hospitalised below 70 are reported in Tables 2 and 3 respectively.

The Italian National Health System (Servizio Sanitario Nazionale, SSN) delivers national guidelines for the standard provision of health services across the country. However, when Italy was first hit by the coronavirus, hospitals experienced a rapid increase of hospitalisations, followed by an exceptional shortage of medical equipment and a limited set of recommendations regarding best practices [14]. The situation required immediate measures to organise local practices and provide urgent care. The rationale for the present study originates from the practical needs emerged during this phase of the emergency. To carry out our investigation, we strived to collect a large dataset that could help respond to the research questions posed at the outset.

Our results are consistent with those obtained by other studies carried out on different outcomes, including mortality, progression to severe/critical status, recovery, length of hospital stay, admission to ICU, intubation, duration of mechanical ventilation, acute distress respiratory, cardiac injury and thrombotic complications. Evidence showed predictive factors including age, comorbidities, vital signs, image features, gender, lymphocyte count and C-reactive protein [9].

In this study, we considered specific predictors, e.g. optimal cutoffs for MDW, which have not been previously considered by algorithms for COVID outcomes using data available during the early phase of admission, to support clinical decision in the Emergency Department.

We identified eight known targets of hospital management to be significantly associated with hospitalisation independently from age and gender: diabetes, dyspnea, procalcitonin≥0.2 ng/mL, MDW ≥ 22, saturation < 96%, D-dimer≥0.72 mg/L, prothrombin time < 95% and CRP ≥ 21 mg/L. Among them, MDW deserves to be presented in more detail. MDW is a novel haematological parameter recently introduced for the diagnosis of sepsis [18, 27], which has been already targeted by recent investigations [28]. Our study confirmed a significant association between MDW and sepsis [29‐31], consistently with other viral diseases [29, 30, 32]. Changes in morphology and volumetric size of white blood cells are a well-documented consequence of cellular activation upon early infection, as a part of innate immunity response [33]. Monocytes are involved in the early response to infection, acting as first interceptors of the invading microorganism, for phagocytosis and further immune processing. Further studies showed changes in the morphology during inflammation [34], differentiating into amoeboid cells, as assessed by microscopy after Giemsa staining, and increased expression of functional markers such as CD16 [35]. Another study found that monocytes homeostasis and morphology appear considerably perturbed in patients with SARS-CoV2 infection [13], with MDW showing significantly elevated values also in patients with COVID-19, compared to other upper respiratory tract infections [36, 37]. As a consequence, the possibility of monitoring monocyte size in parallel with routine blood cell counts and other clinical indicators at presentation to the ED may represent a convenient tool to identify patients at high risk among those diagnosed with COVID-19.

As a first relevant transition after hospitalisation, we focused on oxygen support among patients with COVID-19 aged less than 70 years. In this group of subjects, dyspnea, chest distress and respiratory rate were found to be highly associated with oxygen therapy, suggesting strict monitoring for parameters that can be highly associated with clinical deterioration and adverse outcomes [38]. In addition to saturation < 95%, we also found levels of LDH ≥ 237 U/L and lymphocytes< 1.2 × 10³/μL as adequate targets for oxygen therapy.

Hospitalized patients aged<70y had a higher risk of being admitted to ICU if one of the following cases applied: ESRD, LDH ≥ 334 U/L, MDW ≥ 25, D-dimer≥2.37 mg/L and Lymphocytes< 0.7 × 10³/μL. Other studies found heart disease, COPD and heart rate to be significantly associated with ICU, in addition to ESRD [39]. The dominant role found for the latter in our model can be explained by the organ damage noticed in many patients admitted to ICU, which is associated with a high prevalence of limited renal function [40]. A high value of LDH is an indicator of tissue/cell destruction that is frequently used to monitor tissue damage associated with a wide range of disorders, including liver and interstitial lung disease. Patients with severe pulmonary interstitial disease present an increased LDH as one of the most important prognostic markers of lung injury [41, 42]. Lymphopenia is a biological disorder in patients with COVID-19 frequently considered as predictor of severe infection and myocardial injury, ARDS, and mortality [43]. Lymphopenia is a common consequence of infection caused by cytokine-induced reaction [44]. Reduced CD4+ T-cell and CD8+ T-cell levels promote viral replication and predict worsening outcome [45]. T -cells appear lower and functionally exhausted, and patients with COVID-19 with T- cells≤800/μL may still require urgent intervention due to a higher risk of further deterioration of their condition [46].

Several characteristics at entry to the ED were predictive of mortality in or out of hospital, independently from age and gender, including ESRD, procalcitonin≥0.2 ng/mL, LDH ≥ 307 U/L, saturation < 96% and D-dimer≥1.04 mg/L. We did not find known any significant predictive factor associated with increased mortality, differently from other reports addressing male gender, symptoms < 10 days prior to hospital admission, diabetes, coronary heart disease, chronic liver disease [47], white cell count, temperature, respiratory rate, lymphocytes and platelets [48, 49]. The significant association found between procalcitonin and mortality could be attributed to bacterial co-infection rather than viral replication [50].

Regarding our second research question, we were able to identify accurate risk scores, based on significant coefficients extracted from predictive models. The risk scores showed an average performance ranging from moderate to very high. The performance was very high for hospitalisation, with AUC = 91% (89–94%) and PPV = 94% (90–96%), mortality prediction, with an AUC = 91% (87–94%) and a low PPV = 60% (53–66%), as well as for oxygen support, with AUC = 87% (82–92%) and PPV = 92% (86–95%). The performance was slightly lower for admission to ICU, with AUC = 81% (73–89%) and PPV = 54% (43–65%). Notably, the NPV was high for death (95%, 92–97%) and admission to ICU (91%, 83–95%), indicating that a value of the score below the threshold, particularly for ICU, may be effective in ruling out major complications and considering oxygen therapy as a viable solution.

The comparison between predictive models estimated in our study and other scientific reports may be challenging. Predictive models use different types of cohorts to investigate a variety of outcomes including confirmed diagnosis, disease severity, ICU and mortality. Studies are conducted in different hospital environments, using different laboratory standards and applying heterogeneous inclusion/exclusion criteria e.g. tuberculosis, influenza and bronchitis [9]. Variables included in predictive models are also heterogeneous, from vital signs to image features, contact with other infected individuals, lymphocyte count, liver enzyme and red distribution width [9]. In many cases, data of non-hospitalized patients are limited and do not include laboratory and imaging analyses [51], drawing conclusions only from a limited set of characteristics, with an AUC as high as 90% [52].

To better respond to our second research question, we evaluated the accuracy of our predictive models under real life conditions during the second wave in late 2020.

We found that the overall performance was fairly robust for oxygen support, hospitalisation and death. On the other hand, the results were less satisfactory for admission to ICU (ranging between 66 and 81%, which corresponds to a sensitivity drop between 51 and 82%). A possible explanation may be related to the heterogeneous characteristics of patients admitted during the second wave, which changed the prognosis and thus the predictive ability of models specified under different conditions.

In summary, we identified a set of key parameters that can be translated into risk scores to inform clinical practice. The advantage of this approach is that it can be directly applied to the next patient entering the ED, even with a pocket calculator. A substantial barrier for the continuous update and adaptation of this method is the limited interoperability of health databases in most European contexts, which makes the process of data acquisition particularly burdensome. Improving the digitalization and standardization of health information at hospitals across Europe will be paramount to strengthen our preparedness to future outbreaks and favour the adoption of research methods in clinical practice [53‐55].

Finally, several limitations of our study are worth to be outlined, along with their consequences on the future use of the algorithms.

Firstly, this is a retrospective study carried out at a single hospital, enrolling a limited number of patients during the initial phase of the COVID-19 outbreak in Italy. Consequently, the results may not equally apply to other jurisdictions and/or institutions. However, our report provides a focused stratification of subjects entering the ED that can be informative for clinical practice on a global scale.

Secondly, among the outcomes identified, only death represents a clinically objective measure, while all others reflect decisions made by clinicians at the hospital. Therefore, we cannot infer on the validity of the same models under different settings and variable conditions. However, it is a specific feature of the approach to be able to identify factors orienting practices, so that pragmatic guidelines can be offered when they are not readily available. This feature implies a repeated application of the method to make it locally relevant.

Thirdly, we searched for predictive variables among measures readily available at presentation to the ED. As for all observational studies, we may have missed a subset of characteristics that could be potentially predictive either at entry or in the subsequent follow up in and out of hospital. Nevertheless, we kept our focus on measures that are used in normal practice, so that they can be realistically applied.

Fourthly, the statistical models adopted did not take into account transitions between states e.g. oxygen support followed by intensive care. Our choice was based on the need to facilitate interpretation, rather than enhancing the statistical properties of our methods.

Finally, the stability of the algorithms should be considered in the broader perspective of a continuously evolving virus, in which the continuous update of the vaccination status may blunt the clinical response. This consideration triggers important closing reflections.

The vaccination status of individuals in our samples could not be ascertained from the available data. However, vaccination in Italy ramped up only in February–March 2021, reaching over 90% for those 60+ in early 2022 (see ECDC data at: https://​vaccinetracker.​ecdc.​europa.​eu/​public/​extensions/​COVID-19/​vaccine-tracker.​html#uptake-tab). Therefore, while obviously vaccination was not an issue in 2020, it is also fair to assume that all patients included in the “Wild/Alpha” sample in Dec 2020-Jan 2021 were not vaccinated, while the majority of those included in the “Delta/Omicron” sample could be considered vaccinated. On this ground, we can fairly assume that the vaccination status may have biased estimates obtained for the Omicron sample. On the other hand, the fact that this characteristic is not continuously available in routine hospital databases makes it a difficult candidate for risk evaluation in everyday practice (other than asking directly to the patient, which may be prone to information bias). Therefore, in practical usage, it could be appropriate to consider it as a non- observable confounder, thus incorporating its effect in the association found for other variables.

Overall, the predictive models appeared moderately accurate, irrespective of the variant dominating the reference period. The AUC did not fall below 70%, except for intensive care, for which no variable was significant under the “Delta/Omicron” variants, and only ESRD was significant under the “Wild/Alpha” variants. This slight deviation may show that the subgroup of those below 70 years may not reflect practices following the initial emergency of the “Wild” outbreak (which is partially true also for oxygen support, where only lymphocytes retain their significance). Therefore, the method seems still generally useful to pick subjects at significantly increased risk.

The significance of specific variables in predictive models may also highlight differences in their relevance according to the lineage of the virus. The measurement of MDW, D Dimer, and Prothrombin did not appear as relevant in the evolution of COVID-19 for matters related to hospitalisation. On the other hand, saturation retained its predictive value for hospitalisation and oxygen therapy, while it was not significant to predict survival in the “Delta/Omicron” sample. These aspects seem to indicate changes in the characteristics of the population affected by different variants, reflecting the higher incidence of oxygen therapy and intensive care in the Wild/Alpha, as opposed to a sharp decrease in the incidence of all events under the Delta/Omicron variants (nearly 50% hospitalisations and deaths compared to other variants). On the other hand, the Wild/Alpha sample showed an increased 50% admission to intensive therapy, as opposed to the baseline population. Only few variables could be considered relevant for the new variants of COVID-19, primarily for hospitalisation (dyspnea, procalcitonin, saturation, reactive C protein) and death (ESRD, procalcitonin, LDH, D Dimer).

Since the model was estimated on the original virus, we may assume that our scoring algorithms will continue to be particularly relevant for any future new SARS-type respiratory diseases, where severe cases of pneumonia are more frequent in the absence of vaccination. The same model could be possibly continuously updated, estimating the weights of significant variables repeatedly over time.

We identified demographic, clinical and laboratory parameters associated to hospitalisation, oxygen support, ICU admission and death in a population admitted to a large regional hospital with a confirmed diagnosis of COVID-19. Risk scores derived from multivariate models showed moderate to high predictive accuracy in flagging subjects with more severe prognosis, based upon the early evaluation of personal characteristics at the ED. The method can be conveniently applied to support clinical decisions under different conditions, using targeted data collection at a single point of entry. Recalibration of the scoring algorithms will be needed to cope with the continuous evolution of the virus in different contexts.