Background
Chronic Obstructive Pulmonary Disease (COPD) is a progressive chronic disease, characterized by an irreversible decline in lung function, exercise capacity and health status. The natural history of COPD is interrupted by exacerbations: episodes of worsening symptoms and signs, accelerating lung function decline [
1,
2] and responsible for decreased health related quality of life (HRQoL)[
3,
4], increased mortality [
5,
6] and health-care costs[
7,
8].
Irrespective of the definition of exacerbation used, the clinical diagnosis points to an acute clinical worsening that may necessitate a change in regular treatment[
9]. Early identification and prompt treatment of exacerbations has shown to reduce exacerbation recovery time, while improving health related quality of life and reducing the risk of hospital admission[
10]. Despite the importance of early treatment, several cohort studies have shown that a majority (East London 49%-54%[
1,
10,
11], Canada 68%[
12]) of exacerbations are not reported and most likely not treated. Although unreported exacerbations tend to be milder, these unreported exacerbations still have clinically relevant impact on HRQoL[
12,
13]. Therefore, event-based (based on use of healthcare services or treatment) definitions of exacerbations fail to capture all clinically important exacerbations. Symptom-based definitions, based on the increase of at least one key symptom for two consecutive days (dyspnea, sputum color, sputum volume[
14]), are more likely to catch all exacerbations. However, this does not imply that these exacerbations will all be automatically reported in daily clinical routine. Symptom-based exacerbation definitions have shown to be a valid method of retrospective exacerbation identification in several research cohorts. In the meantime, there is insufficient evidence whether prospective daily symptom registration based on symptom-based exacerbations algorithms is effective in decreasing the amount of unreported exacerbations. In addition, it is uncertain whether patients would be compliant to long-term daily symptom registration and whether reporting would still depend on patients' responsibility to seek treatment and health care. Early studies on action plans aiming at early identification of exacerbations by patients and early treatment failed to show clinically relevant effects on healthcare utilization as well as on patient-reported outcomes, although these studies were all suffering from methodological limitations and were underpowered [
15]. More recent studies have shown that the implementation of an action plan with self-administered prescription of antibiotic and prednisone had the potential to reduce physician visits [
16,
17] and hospital admissions[
17].
Given the importance of early treatment, there is a need for new low-burden strategies to capture symptom based exacerbations. The Clinical COPD Questionnaire (CCQ) has shown to be a brief and useful tool to evaluate disease severity and response to treatment [
18,
19]. The CCQ was developed and validated in 2003 in order to measure health status in daily clinical practice and showed to have strong discriminative properties and responsiveness[
19]. In this study, our aim was to determine the diagnostic value of weekly CCQ assessment to detect exacerbations. More specifically, the objective of the study was to assess the performance of weekly CCQ change in discriminating between stable and symptom-based exacerbation onset weeks, with respect to exacerbation severity. Accuracy in detecting exacerbations for weekly CCQ change was assessed using different cut off scores.
Discussion
This explorative study shows that in exacerbation onset weeks, CCQ scores were significantly increased compared to stable weeks, especially for unrecovered exacerbations. This is in line with a Canadian cohort which also indicated immediate CCQ deteriorations following identification of an exacerbation [
22]. In our study, the performance of CCQ change to discriminate between stable weeks and exacerbation onset weeks was acceptable. The highest discriminative power (AUC 0.75) was seen for the CCQ total score in the unrecovered exacerbation group. These results indicate that weekly CCQ assessment is able to detect formerly unidentified but important exacerbations. However, also a substantial number of type I (false positives) and type II error (false negatives) were present. Exploration of this ROC curve showed that overall sensitivity and PPV are relatively disappointing and substantially vary between different cut offs. Highest cut off scores result in higher overall classification rates (accuracy). Lowering the cut off score (0.1 points) is associated with the highest but still moderate sensitivity and NPV of 72.9% and 92% respectively. However, there is a marked increase in false positives. In contrast, the highest cut off of 0.4 points resulted in decreasing the false positive rate (specificity 93.1%), but is less accurate in correctly detecting exacerbation onset (sensitivity 39.6%) Interpretation and extrapolation of the results in answering whether weekly CCQ assessment is or is not a useful screening tool and identifying the most optimal trade-off point between sensitivity and specificity for detecting unreported exacerbations need to be done with caution.
The present study has several limitations. First, the examination of a relatively small prospective group of 103 consecutive patients followed up for only 6 weeks resulted in 90 exacerbations which is equal to an annual rate of 7.5 per patient-year. This is a relatively high rate and could have affected generalizability of our results since PPV and NPV are strongly related to prevalence[
23]. The high event-rate can be explained because all patients were simultaneously followed-up in the same 6-week winter period in which exacerbations have shown to be ~ 50% more likely than in other seasons[
4,
24]. Also the relative high proportion of patients included immediately after hospitalisation might have contributed to a higher exacerbation rate. Another consequence of the very short study follow-up of 6 weeks without a run-in period (allowing to include only stable patients), 17 exacerbation onset weeks (19%) and 66 (23%) stable weeks were excluded because it was not possible to assess CCQ change (exacerbation onset before inclusion, or onset in the sixth week). Therefore, diagnostic accuracy estimates showed considerable statistical uncertainty (wide 95% confidence intervals). Results of this explorative study need to be validated in larger studies with a preferable follow-up of at least one year. A run-in period including giving adequate feedback could potentially enhance patient compliance of completing questionnaires, subsequently decreasing the amount of (partly) missing or invalid data. Furthermore, we observed that reinforcement by telephone (at 7 days) has the potential to increase understanding and compliance. Future studies with daily symptom registrations should preferably incorporate these calls more frequently, especially in the early stage of follow-up (i.e. at 1 and 2 months).
Secondly, operative characteristics of this test rely heavily on the reference standard used. Despite inconsistency in the literature regarding methods to define exacerbation[
25], we decided to use the symptom-based exacerbation algorithm of Anthonisen[
14]. This algorithm has been widely used to assess symptom-based exacerbations and has shown to capture more exacerbations than strictly depending on event-based definitions[
11,
12]. Nevertheless, diagnostic accuracy for both exacerbation presence as well as severity categories has never been properly validated. Anthonisen's classification never intended to establish the severity of exacerbation using type of exacerbations (type 1, 2 and 3) but rather which patient could benefit from being treated with antibiotics.
Thirdly, discriminative performance was assessed comparing weekly CCQ change between exacerbation onset weeks and all stable study weeks. This means that stable weeks' assessment included both outcomes of patients with and without an exacerbation in the study period. It would have been favourable to perform paired analysis by comparing CCQ change in exacerbation onset weeks with stable periods within the same patient. This method could also determine individual bandwidths of non-exacerbation related CCQ variations in stable periods. Identification of CCQ change outcome beyond normal week to week variations might increase the likelihood of correct exacerbation detection. This obviously needs longer follow-up including an adequate run-in period. In addition, it can be expected that not all patients will fully recover from an exacerbation in terms of CCQ scores[
22]. Longer follow-up enables evaluation of the CCQ across several (recurrent) exacerbations including the possibility to account for non-recovery and individually adjusted baseline levels. It needs to be emphasized that including multiple exacerbations in the analysis demands accounting for dependency on the patient level and the inherent correlation of paired data. In our analysis the impact of multiple-events is marginal since only 5 of the 73 exacerbation weeks were recurrent. Future studies assessing binary classifications (ROC-curves) to detect exacerbations should preferably account for multiple events. ROC curves and according discriminative properties for (partially) paired data should be statistically combined and plotted resulting in a combined AUC with corresponding confidence interval (ref vergara 2008, Metz 1998).
Although this study has several limitations, it adds to current knowledge, being the first study exploring the diagnostic value and practical implications of a low-burden (weekly) and easy-to-complete screening instrument for capturing exacerbations, both reported and unreported. To draw conclusions on the usefulness of weekly CCQ assessments and to define a cut off point as the most optimal trade-off, it is essential to reconsider what are the most important discriminative properties, necessary for this screening tool's optimal clinical utility. For example, the implications of weekly CCQ assessments with a cut off of 0.2 points as a screening tool in the current population in a winter period were as follows: 30 exacerbations would have been detected while only 8 exacerbations were reported (and presumably treated) without weekly CCQ monitoring. The additional 22 correctly detected (but unreported) exacerbations (of which 5 were severe) can be considered as a clinical benefit of the CCQ assessment. However, to be able to detect 22 additional exacerbations, 39 exacerbations would have been incorrectly (false positive) identified and possibly contacted. Still, cost-benefit ratios of verification of (true) exacerbations by contacting patients strongly depend on the technical implementation of screening and methods of contacting patients. If these costs are acceptable, it may be worth contacting 39 patients to be able to capture 22 unreported exacerbations. Detecting exacerbations using weekly assessment results in delayed detection varying between 1-6 days. Although little is known on how early early detection should be, we believe that for obvious reasons it remains preferable to detect (some) exacerbations with a 6-day delay instead of not detecting/treating.
Weekly monitoring of the CCQ appears to be a strictly passive method to enhance detection of exacerbations. Nevertheless, this does not necessarily compete with other methods aiming at early detection and treatment of exacerbations by enhancing self-management, for example by using action plans[
15,
26]. If weekly CCQ monitoring (regardless of the practical procedure) was implemented, this could actually also reinforce self-management behavior. Confronting patients with monitoring outcome and linking this with positive and negative self-management experiences and decisions could enhance self efficacy which is an important predictor of behaviour change [
27,
28]. In the current study, both symptom-based exacerbation identification as well as the CCQ were assessed using a daily paper diary. With regard to the relatively high amount of missing CCQ data in our study, future attempts should also incorporate methods to enhance compliance, such as reminders and possibilities to provide feedback or support. New developments in the field of telemonitoring and e-health have created opportunities to monitor COPD patients on a daily basis. Although this creates a fundamental technical starting point, also for low burden weekly CCQ monitoring, evidence on effects of telemonitoring is still relatively scarce [
29].
Conclusions
CCQ monitoring is a promising low burden method to detect a substantial proportion of unreported exacerbations. Despite the relatively high number of false positives, we believe this exploration is important and promising in view of the disturbingly high incidence of unreported and subsequently untreated exacerbations. Therefore, to confirm the performance of CCQ monitoring in discriminating between normal day to day variations and exacerbations, large diagnostic studies are needed with follow-up of at least one year covering both the seasonality and clustered aspect of exacerbations. This would enable intra-individual identification of periods in which CCQ changes are beyond the bandwidth of non-exacerbation related CCQ variations in stable periods. Furthermore different screening technologies should be evaluated for their daily utility and efficiency for different CCQ cut off scenarios. Finally, if both the test and the technology are well established, follow-up studies to quantify the effect of routine monitoring on patient outcome should be evaluated using randomized trials.
Acknowledgements
We wish to thank all the respiratory nurses, respiratory and family physicians, and research assistants involved in this study. Furthermore we gratefully acknowledge Imkje Horjus, David Schaap, Mariska Manten, Rolf Groenwold for their support in this study.
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
All co-authors have read and approved the final manuscript. JCAT - leading of development of the study conceptualisation, design, refining of protocol and write up for publication. IT - major contribution to refining and substantial contribution to writing up of the protocol for publication. GHWO - major input into study conceptualisation, design and protocol publication. EMM - contribution to development of study design, statistical issues, contribution to protocol publication. JB - expert respiratory input contribution to study conceptualisation and refining on outcome measures, statistical issues and substantial contribution to writing up of the protocol for publication. TV/JWJL/AJP - contribution to study and intervention development and contribution to protocol publication.