Development of an ICU discharge instrument predicting psychological morbidity: a multinational study

All patients ≥ 18 years old were consecutively screened for inclusion for up to 3.5 months, depending on the starting date for each center, during a period from January to June 2016. Patients with an ICU stay ≥ 12 h (≥ 24 h for elective postoperative care patients; those with stays < 24 h were not considered true ICU patients) were eligible for inclusion. Exclusion criteria were need for neurointensive care, previously documented cognitive impairment, lack of a formal home address, inability to communicate in the language of the study site, or ICU admission solely for an elective procedure (e.g. epidural, central line placement). Patients with more than one limitation of treatment (e.g. a do-not-resuscitate order) and moribund patients were also excluded.

The primary outcome was psychological problems 3 months after discharge from the ICU, assessed with the Hospital Anxiety and Depression Scale (HADS) and the Post-Traumatic Stress Symptoms Checklist-14 (PTSS-14). Patients scoring above the predefined cut-off in any of the questionnaires were considered to be a case. Secondary outcome was mental health-related quality of life (HRQOL), assessed with the RAND-36. All questionnaires are validated in the languages of participating countries.

HADS is a validated questionnaire for critically ill patients [16]. It consists of 14 questions; 7 screening for symptoms of depression and 7 for symptoms of anxiety [16]. Each question generates 0–3 points. A subscale score ≥ 11 was used as a cut-off for substantial symptoms of depression or anxiety [17]. PTSS-14 is a validated questionnaire assessing PTS symptoms in ICU survivors [18]. Part A has four questions regarding traumatic memories from the ICU stay, answered yes or no. Part B consists of 14 questions screening for ongoing stress symptoms, each scored from 1 (never) to 7 (always). A part B score > 45 was used as a cut-off for substantial symptoms of PTS [18]. RAND-36 is a 36-item questionnaire assessing HRQOL consisting of 8 domains that can be divided into 2 component scores, the mental health component score (MCS) and the physical health component score [19]. Scores range from 0 to 100, a higher score indicating better HRQOL. We compared scores between patients classified as cases and non-cases in the four mental domains and the MCS as secondary outcomes.

Patients alive 3 months after ICU discharge received and returned the questionnaires by postal mail. Non-responders received a reminder phone call after 2 weeks. A new set of questionnaires was sent out if the original ones were missing or if patients did not respond to the phone call.

Potential predictors were selected after searching the literature and a consensus discussion with a panel of experts. For a list of reviewed studies and more detail on definitions and categorizations of predictors, see the ESM. Data were collected at ICU discharge from the patient data management systems, medical charts, patients or their next-of-kin. The selection rendered 18 risk factors divided into 3 categories: (1) premorbid risk factors, (2) in-ICU risk factors and (3) ICU discharge risk factors.

Given that ICU LOS is the eligibility criterion of choice in many ICU follow-up clinics [11, 13, 24, 25], the association between ICU LOS and psychological problems was examined. The predictive value was assessed with an AUC.

Sixteen percent of responders had missing data for any of the predictors analyzed in the univariable model, of which 10% had only one missing variable. Ten percent of responders had incomplete primary outcome data, with missing items in the follow-up questionnaires. After item replacement as described above, five patients had ambiguous total scores and were not included in the analyses. For a complete description of missing variables and the development of the weighted model for non-response, see the ESM.

Prevalence of predictors and their univariable associations are shown in Table 2. Five predictors [previous psychological problems, age, lack of social support, traumatic memories (PTSS-14 A) and symptoms of depression (modified PHQ-2) at ICU discharge] were associated with psychological problems 3 months after ICU discharge (p < 0.1) and included in the multivariable logistic regression model. All predictors except previous psychological problems were significantly associated with the adverse outcome (p < 0.05). Thus, the final predictors were age (OR age-dependent, peak risk at age 49–65 years, Fig. S1, Table S1, ESM), lack of social support (OR 3.28, 95% CI 1.47–7.32), traumatic memories (OR 1.44, 95% CI 1.13–1.82) and symptoms of depression (OR 1.29, 95% CI 1.10–1.50) assessed at ICU discharge (Table 3). Previous psychological problems did not improve the final regression model. This predictor was strongly associated with psychological symptoms at ICU discharge (p < 0.001 for both PHQ-2 and PTSS-14 part A).

The final model AUC was 0.76 (95% CI 0.70–0.81; Fig. S2, ESM). The final instrument with questions, scores and a graph to obtain patients’ individual risk is depicted in Fig. 2. The instrument is available on-line: http://​www.​imm.​ki.​se/​biostatistics/​calculators/​psychmorb/​. The positive predictive value for a high-risk group with ≥ 60% predicted risk of later psychological problems was 0.83 (95% CI 0.37–0.98). The negative predictive value for a low-risk group with < 30% predicted risk for an adverse outcome was 0.84 (95% CI 0.79–0.88). Positive and negative predictive values for different risk groups are shown in Table S2 of the ESM. For calibration of the model, see Fig. S3 of the ESM.

With internal validation in 500 bootstrap samples, the AUC was 0.73 (95% CI 0.73–0.74). The shrinkage factor for the final model was 0.89. The regression coefficients can be multiplied by that value to provide more reliable predictions for new patients. A maximum shrinkage factor of 1 indicates that no over-optimism occurs.

The AUC for ICU LOS as a predictor was 0.49 (95% CI 0.42–0.56; Fig. S5, ESM).

Our study is multinational, prospective and includes medical and surgical patients with a broad range of ICU LOS, increasing its generalizability. To our knowledge, it is the largest psychological ICU follow-up study to date. The follow-up response rate of 78% can be considered high in an ICU survivor population. The instrument has been internally validated but needs external validation in different ICU cohorts.

A potential limitation with this kind of study is missing data. Data missing due to non-response was managed with inverse probability weighting, a recognized strategy for handling loss-to-follow-up that may not be missing completely at random [23]. A substantial proportion of patients had an ICU stay shorter than 12 h, and more patients than anticipated were transferred to other ICUs before discharge, rendering a smaller study cohort than originally projected. Based on prior agreements with study sites and resource constraints, the study was stopped at the agreed time; hence fewer participants than projected were included. Potential over-fitting of predictors due to the smaller study size was assessed, rendering a high shrinkage factor indicating little over-optimism. Fewer participants than expected had psychological problems, which could partly be explained by the stricter cut-offs for the HADS subscales, using ≥ 11 (probable case) rather than the often-used cut-off of ≥ 8 (possible case). Another limitation is the time anchoring of the PHQ-2 questionnaire, assessing symptoms during the past days, an aspect that needs consideration for patients with short ICU stays. Using self-administered questionnaires instead of psychological interviews was due to limited resources. However, the PTSS-14 and the HADS are validated questionnaires screening for psychological symptoms in ICU survivors [17, 18].

The instrument facilitates identification of high-risk populations for interventional follow-up trials. This step may reduce dilution of potential beneficial effects of follow-up, a problem that may have contributed to the lack of effects in previous ICU follow-up interventions [35‐40]. The instrument has fair predictive accuracy in a mixed ICU survivor cohort, does not require special equipment or specially trained staff and enables concentration of ICU follow-up to patients most likely to benefit from post-ICU interventions. Such interventions can be initiated early and potentially reduce long-term psychological morbidity and improve HRQOL.