Development of the AL-O-A Score for Delirium Screening in Acute Internal Medicine: a Monocentric Prospective Study

Delirium (acute confusion) is frequent among patients admitted to acute care hospitals. In a meta-analysis, including patients mostly over 65 years old, its prevalence ranged from 19%–23%.¹ However, very low and higher prevalence have been reported, depending on population age, the proportion of co-morbid diseases (mainly dementia, stroke, substance abuse) and clinical care settings.²

Delirium has a significant impact on health and prognosis, being associated with a prolonged hospital stay,³ faster cognitive decline, increased risk of fall,^3‐5 institutionalisation and death.⁶ Under-diagnosis is frequent and, in practice, only 30–60% of patients with delirium are appropriately diagnosed (outside studies or systematic screening).^{2, 7, 8} The estimated yearly costs attributable to delirium are USD 164 billion in the USA and USD 182 billion across 18 European countries.^{9, 10} Thus, delirium screening and prevention have major health and financial implications.

The Confusion Assessment Method (CAM)¹¹ has long been the most-used bedside tool to diagnose delirium. Shorter versions or versions dedicated to specific clinical settings are also available, such as the CAM-ICU and the 3D-CAM.^{12, 13} Other brief screening tests (4 As Test, two-item bedside test, MOTYB-Spatial Span Forwards test) have been developed for specific populations (elderly, ICU, surgical/neurological patients).^12‐16 Nevertheless, because some need training, are time-consuming or restricted to specific settings, the distribution of many screening instruments is limited. Furthermore, most of these instruments perform poorly on patients with dementia. There remains a need to develop a brief, easy-to-use, reproducible screening tool, applicable to any admission to acute internal medicine wards. This study’s aim was, therefore, to develop a new screening score to identify inpatients with delirium at hospital admission.

To develop and internally validate a new delirium screening score, we performed a monocentric, prospective, observational study between 1 November 2019 and 6 January 2020. Physicians administered a questionnaire (index test) to all eligible patients within 48 h of their admission to the acute medical wards of a Department of Internal Medicine that included no neurology or psychiatric units. Patients underwent a neuropsychological examination to assess the presence or absence of delirium on the same day as they took the study questionnaire. Patients or their relatives gave written informed consent. These procedures followed the precepts of Good Clinical Practice and the Declaration of Helsinki. The Cantonal Ethics Committee, Vaud (CER-VD) approved the study. Reporting was performed according to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement.¹⁷

During the study period, 217 participants admitted to the acute medical ward underwent both the study questionnaire and a dedicated neuropsychological evaluation (Fig. 1). Ages ranged from 19–104 years old, with a median age of 76 (IQR: 66–85). Men (n = 108) and women (n = 109) were equally represented.

The neuropsychologist determined that 32 admitted patients (16%) had delirium. Six patients needed reassessment by two other neuropsychologists for a definitive diagnosis (3 with and 3 without delirium). The univariate analyses associated all of the study questionnaire’s predictors with delirium (Table 1). The subjective predictors and the five questions were specific but had poor sensitivity. Tasks were sensitive, and their specificity increased with the number of mistakes but with a decreasing sensitivity. The backwards digit and temporospatial orientation tests were the two most discriminative single items, with AUC of 0.82 (0.73–0.90) and 0.87 (0.79–0.94), respectively (Table 1 in the Appendix section).

The AL-O-A score (ALertness, Orientation, off-target Answers) for delirium screening in internal medicine wards is short (three items) and easy to administer. It displays excellent discriminative performance, even in subgroups categorised by age, sex and cognitive status.

Patients with delirium have a twofold increased risk of death and a two-to-three-fold increased risk of institutionalisation.⁶ Undiagnosed delirium has been associated with longer hospital length of stay and worse cognitive performance.³ Furthermore, delirium persists in up to one quarter of patients at 6 months after discharge.^{27, 28} Diagnosis of delirium thus has a direct impact on hospitalisation and potentially influences care at discharge. Since clinical judgement is insufficient, delirium diagnosis requires a high index of suspicion.^{2, 7} To that end, the AL-O-A score was developed as a screening tool that could be administered to any admission, favouring sensitivity over specificity. The resulting score displays excellent negative predictive value, easily excluding two thirds of patients. Administered by untrained, junior physicians in the present study, the AL-O-A score could be quickly incorporated into the daily practice of many hospitals by physicians with very different experience levels. Since most of the items are part of standard admissions procedures in internal medicine settings, the systematic application of the score would not extend the time dedicated to a medical workup. Nevertheless, further studies should explore this strategy’s clinical and economic relevance.

The AL-O-A score lacks several essential criteria for a definite diagnosis of delirium as listed in the DSM-V.²² Notably, the score does not capture whether a disturbance develops over a short period (usually hours/days) or fluctuates over time, and those features are essential to distinguishing delirium from dementia.⁹ Furthermore, to diagnose delirium, the physician must acknowledge that disturbances are not better explained by another neurocognitive disorder and are not occurring in the context of a severely reduced arousal level, such as coma. The score assumes that disturbances are the direct physiological consequence of another medical condition, including substance or toxin intoxication or withdrawal.²² Although the AL-O-A score was designed as a screening strategy and lacks several DSM-V criteria, its positive predictive value (83%) for high-level probabilities (score 4.5 or higher) is good enough to alter immediate medical management. The probability is sufficient to initiate a thorough workup looking for aetiology, correct associated factors and include these patients in preventive programmes (e.g. against in-hospital falls, the Hospital Elder Life Program (HELP) and others).²⁹

The best single-item tests for delirium screening were orientation and attention tests. The backwards digit (AUC 0.82) and temporospatial orientation (AUC 0.87) tests were the most discriminative. Making one mistake (in either test) had a sensitivity of around 90% and a negative predictive value of > 90%. Making three or more mistakes had a specificity around 90% and a positive predictive value from 50–65%. This observation was in line with the study by Fick et al.³⁰ However, the score constructed with objective predictors only, including both the orientation and attention tests, had no advantages over the main score and performed slightly less well, with moderate inter-rater reliability.

The score based on CAM features was the least discriminating and far more complex. It was therefore worthless in a rapid screening strategy.

Several tools exist for inpatient delirium diagnosis. The closest to the AL-O-A score—when considering the reference standard used and the items that compose the final score—is the 4 As Test (4AT).^{14, 15} This shares alertness and some items concerning orientation present in the AL-O-A score. The 4AT also tests attention through its months of the year backwards (MOTYB) test and acknowledges delirium’s acute and fluctuating courses. However, the 4AT was developed in a geriatric population aged 70 and over. It performs as well as the AL-O-A score at the 4.5 points cut-off (sensitivity of 76% vs 78%, and specificity of 94% vs 97%, respectively).¹⁵ However, the AL-O-A score displays better sensitivity at the 2 points cut-off (97% sensitivity and 71% specificity). Thus, a patient with normal alertness, giving no off-target answers and able to identify the date and their location is unlikely to be confused. The 3D-CAM is a structured assessment tool displaying 95% sensitivity and 94% specificity.¹³ It was derived among patients aged 75 years old or more. Although this widespread test performs well and takes far less time to administer than the complete CAM, it remains a complex tool that would be difficult to generalise as an initial screening test. The same research group developed an ultra-brief, two-item, bedside test for delirium composed of the MOTYB test and the “What day of the week is it?” test.³⁰ This test has 93% sensitivity and 64% specificity among geriatric populations (one quarter suffering from dementia). O’Regan and colleagues administered attention tests (MOTYB and the Spatial Span Forwards test) to 265 adult inpatients (median age 69 years old) and found them to be sensitive.¹⁶ However, they did not formally assess delirium in all participants (excluding patients who passed both attention tests), which may have influenced the attention test’s sensitivity. Besides, they administered this as a 1-day study, including patients at very different stages of their diseases (new admissions and patients about to be discharged). Finally, the population was also composed of surgical and neurological patients. Thus, to the best of our knowledge, AL-O-A is the first score developed for any admission to an internal medicine ward, unrestricted by age or dedicated to at-risk patients.

The present study has a few limitations. First, not having restricted the study by age or to a priori at-risk patients resulted in low delirium prevalence, which affected the study’s power (few events per predictor) and could have inflated the score’s performance. However, its performance was similar in both the lowest (low prevalence) and highest age quartiles (high prevalence). Secondly, two of the score items are subjective. Nevertheless, many delirium scores include these items, with good inter-rater agreement.^{12, 13, 15} The two subjective predictors were discriminative even though untrained physicians administered the test. Furthermore, the test’s performance was constant over a variety of subgroups, indicating that the presence or absence of items was easy to assess, even in older patients or patients with cognitive impairment. In an effort to develop a more reproducible measure, we tested an alternative score, including objective predictors only, but it performed worse. Thirdly, because some of the score’s questions are close to those asked during the neuropsychological examination, there could be incorporation bias. However, the AL-O-A score is far simpler and less time-consuming; therefore, we believe that its usefulness has been demonstrated. Fourthly, although external validation is the preferred validation method whenever possible, no independent sample was available for study. We instead performed an internal validation.¹⁷ Among internal validation methods, bootstrapping is the recommended technique for correcting a model’s performance for optimism.¹⁷ Fifthly, the reference standard was based on classic, validated scales administered by an experienced neuropsychologist. However, neuropsychological evaluation for the diagnosis of delirium could have differed between raters. Finally, the test was developed and evaluated in a French-speaking Swiss population sample (the English version is provided in the Appendix section), and its performance might vary in other languages or cultures. Nevertheless, AL-O-A’s individual predictors can be found in several instruments validated in different languages with equivalent performance.^{12, 13, 15} Besides, the score was administered during the 48 h after admission, and patients with strokes, acute psychiatric diseases or post-surgery were not included. Thus, the score’s performance remains unknown in these patient groups or later after admission.

In conclusion, the AL-O-A score is fast, easy and performs well, even among older adults or patients with a cognitive impairment admitted to an internal medicine ward. Further studies are now needed to assess its clinical relevance as a systematic screening tool for any admission and its performance in other settings (external validation).