Development of the Tracheostomy Well-Being Score in critically ill patients

This prospective, monocentric, observational study was performed at three medical and surgical ICUs and one intermediate care unit (IMC) of the BG University Hospital Bergmannsheil Bochum, Germany. The study was approved by the authors institutional review board (20-6887) and registered at the German Clinical Trials Register (DRKS00022073). It adheres to the “Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement” (Online Resource 1) [25].

From June 8, 2020, to April 8, 2021, all eligible patients were prospectively tested. All patients included met the following criteria: (1) inserted tracheostomy tube for more than 48 h and expected to remain cannulated at least for the next 7 days; (2) adequate cognitive and motoric skills to answer the survey manually, verbally, or with support of eye-tracking (ET) devices; (3) over 18 years of age; (4) a score of − 1, 0 or 1 points on the Richmond agitation-sedation scale (RASS) [26] or a score of < 3 points on the nursing delirium screening scale (Nu-DESC) [27, 28] in patients without sedation; and (5) successful completion of the survey on two consecutive days.

Out of 71 eligible tracheostomized patients, 8 patients were excluded because of the following conditions: language barrier (n = 3), incomplete data (n = 3), and refusal to participate (n = 2). The remaining 63 patients were enrolled in the study (Fig. 1).

Standard parameters, such as sex, age, reason for hospital admission, and length of stay in hospital, were applied to describe the demographic data of the included patients. The reasons for ICU or IMC admission were divided into major trauma, non-abdominal sepsis, acute abdomen, exacerbation of chronic obstructive pulmonary disease, modification of long-term mechanical ventilation, and cancer.

We aimed to develop a questionnaire that would, on the one hand, provide a score and on the other hand allow charting of tracheostomy well-being across a profile of different categories.

The categories of TWBS were based in part on the review by Nakarada-Kordic et al. [9] The authors highlighted the typical problems of a tracheostomized patient, which primarily have a negative impact on patient well-being. Based on their results, six categories (respiration, coughing, pain, speaking, swallowing, comfort) could be defined. Corresponding questions were collected for each category.

The survey was developed by the interdisciplinary ICU team consisting of critical care nurses, intensivists, psychologists, speech and occupational therapists, and physiotherapists. The list of items was also based on comments and suggestions of tracheostomized ICU patients who were in the weaning process or were on long-term ventilation or already had been successfully weaned. From a list of 47 potential questions, 17 were removed because they were very similar and 5 items were removed because they appeared irrelevant in the intensive care setting from the authors’ point of view, so that 25 items remained from the categories. The answers to the 25 items should be graded on a 4-point Likert scale (0 = never, 1 = sometimes, 2 = often, 3 = always). The selection of the 25 questions was assessed positively by all involved, so that the final version of the questionnaire was created (Online Resource 2).

Patients potentially fulfilling our inclusion criteria were identified during the daily ward round of the interdisciplinary ICU and IMC teams. Prior to inclusion and to each session, the patient’s level of consciousness was assessed using the RASS and the Nu-DESC. In each of the six categories of the survey, an external assessment of suspected problems was also performed by the first author of the study, a medical specialist for orthopedics and trauma surgery with many years of expertise in critical care medicine, using a 4-point Likert scale (0 = unlikely, 1 = moderate, 2 = likely, and 3 = quite likely). The procedure was explained verbally to the patients, and informed consent was obtained in written form, or if not possible, verbally, via head nodding, or by blinking. This was confirmed by persons who were unrelated to the investigation. Depending on the current fine motor capabilities of their arms and communication skills, the patients were either able to answer the 25-item survey manually with paper and pen or were able to answer verbally after the questions were read aloud by the experimental supervisor, or the patients were able to answer the questionnaire via ET devices. The Tobii Dynavox I-15 + ET device (Tobii Dynavox, Danderyd, Sweden) was used for this in nonverbally restricted patients with limited fine motor capabilities [29‐31]. On the next day, the survey was repeated with each patient. Throughout the entire study, the investigation was performed by the first author of this study using this 25-item questionnaire (Online Resource 2).

In order to determine which items of the pool should be selected for the scale, we checked the relevant criteria as detailed below. (1) For an initial overview, the percentage response frequencies were determined for all 25 items (Table 2). The items of the final scale should differentiate well among respondents. Therefore, items to which most respondents provide the same response should not be considered. Based on this reasoning, items with a skewed distribution of answers (e.g., > 50% answers at one of the extremes—never or always) were excluded.

(2) In a second step (Table 3), we assessed retest reliability by correlating for each of the 25 items responses from day one with responses from day two across patients. High retest-reliability implies that differences among participants reproduce from 1 day to the next, suggesting that the item is understood well and taps into judgments that are not strongly fluctuating. As items suitable for the scale should elicit reproducible responses, we wanted to make sure that retest reliability is high.

(3) In a third step, we considered content categories. As the items in the pool had been generated by taking six categories into account it was relevant to check to what extent each item would be prototypical for the category it belongs to. To this end, we computed for each item how strongly it correlated with the average of the other items of the category it belonged to (using day one data). Prototypical items need to show a positive item-total correlation. Beforehand, inversely worded items (S1, S2, SW3, CO3, and CO5) were recoded so that all items were coded in terms of criterion and thus positively correlated with each other. We aimed to select items that would show a high item-total correlation as these can be considered as prototypical for their category. Given that retest reliability was high for the most items, we focused on how well the item would represent its category. The wording of each chosen item was to fit to the category and represent aspects of only the one category. Furthermore, item-total correlation was to indicate that the item represents the category well.

(4) Aiming at items that would distinctively represent one category, we excluded items that mentioned aspects of two categories.

The four steps reported above were taken with the aim to form a short scale that would, on the one hand, provide a score and, on the other hand, allow to compare participants on a profile representing the six categories of tracheostomy well-being. Accordingly, we selected sequentially according to the four steps charted above the two best items per category.

After completing the item selection (i.e., selecting the 12 best items out of the pool of 25), we ran to further analyses to evaluate the result. First, we tested the internal consistency of the 12 items scale using Cronbach’s alpha. A positive and high value in Cronbach’s alpha (i.e., high internal consistency) shows that there is similarity within persons (i.e., those who answer “always” or “often” on some items also do so on others) while at the same time there are substantial differences between persons (i.e., some patients in most items answer “always” or “often” other patients respond to most items with never or sometimes).

Second, we checked separately for each of the six content categories whether the report of the patient (i.e., average across the answers to the two items per category) would overlap with the clinician rating.

A high correlation between the self-report of the patient and the clinician rating would suggest that these perspectives overlap strongly. This would bring up the question if it is actually warranted to take the time and effort to assess the patient’s perspective.

Statistical analysis was performed using Microsoft^® Office Excel^® for Mac 2019 (Microsoft Corporation, Redmond, WA, USA) and IBM^® SPSS^® Statistics Version 28 2021 (IBM Corporation, Armonk, NY, USA). Demographic data are presented as mean and standard deviation or as absolute numbers and percentage. Retest reliability was assessed with the Kendall’s τ correlation coefficient. Item–total correlation was determined using the part-whole correction implemented in SPSS based on the Pearson correlation coefficient. The inter-item correlation between the two items selected for each of the six categories was assessed using Pearson correlation coefficient. As explained above, internal consistency of the scale was computed using Cronbach’s alpha.

The demographic data of the 63 enrolled patients are listed in detail in Table 1. Four out of 63 patients (6.3%) were able to answer the 25-item survey manually. The majority of patients (84.1%) used their voices for scoring. In six patients (9.5%), ET devices were necessary for answering the questionnaire.

On discharge or transfer to further rehabilitation, 28 patients (44.4%) were successfully decannulated, 26 patients (41.2%) were on long-term mechanical ventilation, and 4 patients (6.3%) were cannulated but not mechanically ventilated (e.g., cough assistance techniques, reduction of labored breathing and suctioning). During the inpatient stay, five patients (7.9%) died due to their critical illness.

An overview of all 25 items used with the respective response options for day one and day two is shown in Table 2. In Step 1 (see above), we detected that 12 items show a skewed distribution of responses and should therefore be excluded. In Step 2, we learned that the retest reliability is high for all items. All proved reliable with values ranging from 0.67 to 0.93 using Kendall’s τ (Table 3). Hence, no item had to be excluded due to a low correlation of day 1 responses with day 2 responses.

In Step 3, we found out that there are at least two items per category that can be regarded as prototypical for their category due to a high item-total correlation (see Table 3).

Scrutinizing whether items mention aspects of two categories rather than distinctly represent one category led to excluding five items in Step 4. For each of the two best items per category (printed in bold in Table 3), there was a positive inter-correlation. A positive correlation among the two items per category signals internal consistency and suggests that their average can be used to represent the category when analyzing profiles across categories.

Furthermore, Cronbach’s alpha calculated across all 12 items of the TWBS assured that there was decent internal consistency on scale level (0.74). Given that patients responded on a 4-point scale (0 = never, 1 = sometimes, 2 = often, 3 = always), the highest possible sum score across the 12 items is 36 (all items responded to with “always” while a score of zero would indicate that a patient had responded “never” to all items. In our sample, the mean was 16.6 and the standard deviation was 5.5. The minimum score obtained by a patient was 3, and the maximum was 30.

Finally, checking the correlations between the self-reports and the clinician rating (Table 4) showed that there was little overlap among the patients’ and clinician perspective (no significant correlations). An overview of the final version of the 12-item TWBS survey as it will be applied to the patients in future work listed in Fig. 2.