Development and initial validation of the bronchiectasis exacerbation and symptom tool (BEST)

The study objective was to develop a symptom diary that measures day to day changes in patient symptoms but also accurately detects exacerbations. The study was conducted in parallel with the development of the consensus definition of exacerbations by EMBARC (European Multicentre Bronchiectasis Audit and Research Collaboration). and the Bronchiectasis Research Registry (EMBARC/Bronchiectasis Research Registry- BRR) [3]. As previously reported, the authors conducted a systematic review of definitions of exacerbation used in previous studies in bronchiectasis. The results of this systematic review are reported in the EMBARC/BRR publication but were also used for the development of this diary. Further information about the results of the systematic review are provided in the Additional file 1. As described in the EMBARC/BRR publication, the symptoms most frequently used in prior definitions, and rated by Delphi process as most important were sputum volume, cough, sputum colour, dyspnoea, fatigue and systemic disturbance [3]. Patient self-report of an exacerbation was also considered important in the Delphi process. Based on these results and interviews with patients who also identified the same critical symptoms [18], we designed a daily diary that appropriately rated these symptoms. The diary was modified with feedback from patients in terms of wording and clarity of meaning until a final version was completed and deployed in the study. For rating dyspnoea we incorporated the existing MRC dyspnoea score as it is validated in bronchiectasis. The sputum colour domain followed the previously published scale by Murray et al which rates sputum from 1 to 4 ranging from clear to dark green, with additional domains of no sputum (0) and haemoptysis [5, 19]. The topic “systemic symptoms” had little meaning to patients and was modified to cold and flu symptoms. Quantification of sputum volume using millilitres was also regarded as difficult for most patients and so equivalent volumes using teaspoon/tablespoon, egg-cup and cup volumes were used. A teaspoon is equivalent to 5 ml, a tablespoon is equivalent to 15 ml, egg-cup 45 ml and cup volume is approximately 250 ml. The final diary is shown in Table 1. The maximum score was 26.

Patients were asked to complete the diary every day for a period of up to 6 months while clinically stable and then when they experienced an exacerbation. The patients were asked to contact the investigators when they experienced an exacerbation so that treatment could be commenced by the study team using a standard 14-day course of antibiotics. The tool was not used to trigger contact with the study team.

At the baseline visit patients were issued with the diary and provided with education on its completion. Paper diaries were used to avoid any bias in this population towards those familiar with mobile technologies. Patients completed the SGRQ, the COPD assessment test (CAT) and the Leicester cough questionnaire (LCQ) at the baseline visit. Patient enrolment took place in 2014 prior to the development of the disease specific tools such as the Quality of life bronchiectasis questionnaire (QoL-B) or the Bronchiectasis Health Questionnaire (BHQ) [20, 21]. Patients underwent spirometry and performed a 6-min walk test (6MWT) to evaluate convergent validity with these measures of lung function and functional status respectively. The assessments were repeated at the start and end of each exacerbation and finally at the end of study.

We addressed firstly whether the symptom diary accurately measured symptom burden in patients with bronchiectasis by demonstrating its convergent validity with existing symptoms and quality of life (QOL) measures. We hypothesised that the baseline symptom diary score would correlate with the SGRQ total and symptom scores, the LCQ, the COPD CAT and 6MWT. We next analysed the dynamic changes in bronchiectasis symptoms over time. We hypothesised that symptom scores would increase during periods of reported exacerbations. To detect unreported exacerbations we required an increase of at least 3 points in symptoms for a minimum of 48 h based on the EMBARC/BRR definition of exacerbation [3]. An unreported exacerbation by definition was associated with sustained increase in symptoms but did not lead to a medical review or antibiotic treatment.

We estimate a provisional minimum clinically important difference for the BEST tool. As there is no single accepted method of determining the minimal clinical important difference (MCID) we used 3 different approaches. First, the distribution based method using ½ the standard deviation of the scores for all participants. Second, if a correlation coefficient of > 0.3 was achieved for the relationship with scores which had an established MCID (such as the SGRQ, CAT and LCQ) we would use these as an “anchor” to estimate the change in BEST score that equated to a clinically meaningful change in the other tool. Finally as a reported exacerbation is a clinically meaningful change in symptoms, we would consider the change in BEST at exacerbation onset as a potential MCID [22].

Statistical analysis was performed using Graphpad Prism v6 and SPSS version 22 (IBM, USA). Categorical variables are presented by frequencies and percentages while continuous variables are presented as mean and standard deviation (SD) or median and interquartile range (IQR) when data are not distributed normally. The relationship between the diary scores and quality of life/symptom measures such as SGRQ, CAT, LCQ and 6MWT were performed by linear regression. The regression equation was used to calculate the change in BEST score that would equate to the MCID scores of 4 points in the SGRQ, 2 points in the CAT and 1.3 points in the LCQ [22, 23]. Students T-test was used to compare means between different time points such as stability and exacerbation. Stability of symptoms within individual patients was evaluated with the within subject standard deviation and the coefficient of variation. To establish patients’ baseline level of symptoms we took the median value in the week prior to an event (or in the first week of the study). As this was a pilot study, no formal sample size calculation was performed and the objective was to enrol 20 patients to get initial experience with the tool. For all analyses a p-value < 0.05 was considered statistically significant.

Twenty-one patients were included. All patients had idiopathic or post-infective bronchiectasis. The characteristics of the patients are shown in Table 2. Patients had predominantly moderate to severe bronchiectasis and in common with most bronchiectasis populations, were elderly and female. As the study enrolled patients with a history of exacerbations the mean exacerbation and use of prophylactic therapies such as macrolides was high in this cohort.

Nineteen patients completed the diary daily for at least 3 months. One patient had an exacerbation within the first week of starting the diary and did not persist with completion following exacerbation resolution, while another patient completed 6 weeks without an exacerbation and then ceased to adhere consistent data entry. Baseline data for the BEST are therefore shown for n = 21 patients while dynamic changes over time are analysed for N = 19 who had available long term data.

At baseline, all patients completed the SGRQ, LCQ, CAT questionnaires and performed a 6MWT as measures of health status. The relationship between the baseline BEST score and the established measures of health status are shown in Fig. 1. The correlation with the CAT score (Fig. 1a) was r = 0.61, p = 0.0037. A significant correlation was also detected between the BEST score and the SGRQ total score (r = 0.61, p < 0.0001), Fig. 1b, 6MWT (r = − 0.46, p = 0.037,Fig. 1c), LCQ score (r = − 0.52,p = 0.015), Fig. 1d, and the symptom domain of the SGRQ (r = 0.52, p = 0.015),

A 4-point change in the SGRQ is regarded as clinically significant. The above correlation suggested that a change of 3.5 points in the BEST symptom diary was equivalent to a 4-point change in the SGRQ. For the LCQ, a 1.3-point change is regarded as clinically significant, and this correlated to a 4.95-point change in the BEST score. A clinically significant 2-point change in the CAT required a 4-point change in the BEST symptom diary. Taking a cut-off of 4 points, the change in BEST score had a sensitivity of 93% and specificity of 66% for detection of treated exacerbations. Raising the threshold to 5 points had a sensitivity of 86% and specificity of 81%. At 6 points the sensitivity declined to 79% with specificity of 82%.

Figure 2 below shows the dynamics of symptoms over time for 19 patients who were able to complete the diaries consistently for at least 3 months. In the figure below, the patient reported exacerbations are identified with a red box. 29 exacerbations in total were reported during the study.

Based on the time taken for the symptom score to return to the baseline level post-exacerbation, symptoms remained elevated for 15.3 days after exacerbation diagnosis (SD 5.7). The range of recovery time was 5 days to 28 days.

Based on a sustained increase in 3 points for at least 48 h, we identified 23 unreported exacerbations in 13 patients. No patients received antibiotic treatment during periods of unreported exacerbations. 8 patients had no unreported exacerbations. The mean change from baseline in an unreported exacerbation was 4.7 (SD 1.5, range 3–8 points). The mean duration of unreported exacerbations was 10 days (SD 3.8). Unreported exacerbations were therefore significantly milder and shorter than reported exacerbations. The end of the unreported exacerbation was considered when the score returned to pre-exacerbation baseline level. The sample size was too small to establish any relationships between unreported exacerbations and clinical outcomes.

We observed that some patients had relatively stable symptoms over time while others showed remarkable day to day variability in symptom scores. The coefficient of variation in symptom scores, excluding exacerbation periods, ranged from 16.1 to 97.8%. No relationship was observed between the co-efficient of variation or the within subject standard deviation and patient characteristics (Table 3).

The correlation with the SGRQ and CAT scores suggested an MCID of 4 points would be consistent with a clinically important change in other tools. Using the ½ standard deviation method to calculate the MCID suggested a smaller MCID of 2.2 points. Exacerbation onset was associated with a change of 4 points as described above. Based on the majority of data available we propose an initial MCID of 4 points.