Development of quality indicators for diagnosis and treatment of patients with non-small cell lung cancer: A first step toward implementing a multidisciplinary, evidence-based guideline

doi:10.1016/j.lungcan.2006.07.001

Lung Cancer

Volume 54, Issue 1, October 2006, Pages 117-124

https://doi.org/10.1016/j.lungcan.2006.07.001 Get rights and content

Summary

Background

While developing and distributing clinical practice guidelines are important in optimising clinical healthcare, insight into actual care is necessary to achieve successful implementation. Developing quality indicators may be the first step to becoming aware of actual care. The Dutch national practice guideline Non-small cell lung cancer: staging and treatment is one of the first clinical, multidisciplinary guidelines for oncology in the Netherlands for which quality indicators were developed systematically. We describe indicator development based on this guideline as a practical experience.

Methods

To develop a set of indicators for diagnosis and treatment of patients with non-small cell lung cancer, we systematically achieved consensus on the basis of a national, multidisciplinary, evidence-based guideline and the opinions of professionals and patients. After the researchers extracted the recommendations from the guideline, we carried out a so-called Rand-modified-Delphi procedure. This consisted of three rounds: a national panel of professionals and representatives of the national patient organization scored all recommendations, the professionals had a consensus meeting, and the final set of indicators was e-mailed for a last check. Subsequently, some clinimetric characteristics of this final set were assessed in a practice test.

Results

Thirty-two of 83 recommendations were selected in the first round. After the consensus meeting, 8 recommendations met the final criteria and were incorporated into 15 indicators, which were tested in practice. The most successful indicators for quality improvement are indicators that are measurable, have potential for improvement, have a broad range between practices and are applicable to a large part of the population.

Conclusions

For successful implementation of evidence-based guidelines, each new guideline should be developed and tested with a set of indicators based on the guideline. The procedure we describe can serve as an example for other new guidelines.

Introduction

The shift from consensus-based guidelines to evidence-based ones in the field of quality of care in the last 15 years is evident. The next important step to be taken is the development of evidence-based indicators. Indicators are necessary for measuring the application of evidence-based guidelines in daily practice. Although much about the need and the usefulness of indicators has been published in recent years [1], [2], [3], good examples of procedures to be followed are rare [4].

This article describes how such a procedure should be worked out, and gives a unique practice example of the Dutch evidence-based multidisciplinary guideline for the diagnosis and treatment of patients with non-small cell lung cancer (available in English on www.oncoline.nl) [5], [6].

Research shows that some of the diagnostic procedures and treatments in health care are ineffective or even detrimental, whereas treatments with proven activity are sometimes not given [7], [8]. The consequences can be considerable, both personally and socially. To optimize the care for patients, several associations of medical specialists and supportive organizations, such as the Dutch Institute for Healthcare Improvement (CBO) and the Comprehensive Cancer Centres in the Netherlands, started to develop guidelines years ago. These guidelines were initially drawn from consensus texts. However, in recent years, the process of guideline development has been formalized more and more, scientifically grounded, and made transparent to meet the requirements of evidence-based medicine [9], [10], [11]. This activates the precise and judicious use of the most recent scientific insights in medical decision-making in the care for individual patients.

The availability of evidence-based guidelines does not mean that patients automatically get the care that is defined as the “best care” in evidence-based guidelines. Implementation of the best care needs more than the publication and distribution of these guidelines [12], [13]. Insight into the current care and into possibly obstructive and/or beneficial factors are required for successful implementation of the guideline [14], [15], [16]. On this basis, an action plan can be tailor-made to increase guideline adherence. An important step to get insight into current care is the development of quality indicators, based on an evidence-based guideline. Quality indicators can be defined as: “measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality of care” [17], [18]. Development of quality indicators requires a careful and systematic procedure to produce valid, reliable, and useful indicators that are accepted by the target group and are sensitive to changes in the quality of care [4], [19]. Such quality indicators can give a reliable reflection of the quality of the care provided and can lead to quality improvement activities [20].

Section snippets

Background

In the 20th century, the incidence and mortality of lung cancer increased so dramatically in most European countries that it became one of the major epidemics of the century [21]. In the Netherlands, approximately 9000 new patients are diagnosed with lung carcinoma every year [21]. Approximately 80% of them have non-small cell lung cancer (NSCLC), and the incidence for women is increasing. One of 14 men and 1 of 30 women develops lung cancer. Only 25% of the patients with NSCLC qualify for

Development of indicators from the professional's perspective

The method used to develop quality indicators on the basis of the multidisciplinary guideline for NSCLC combines “evidence” from the guideline with expert opinion—the so-called Rand-modified-Delphi method [25], [26], [27]. It is unique to our procedure that the guideline developers themselves have developed the indicators. A representative panel of 17 professionals, who were also involved in developing the guideline, were approached to participate in developing the quality indicators: five

Development of indicators from the professional's perspective

The guideline for diagnosis and treatment of patients with NSCLC contained 83 recommendations for good quality of care. Researchers classified these recommendations into three dimensions of quality of care: professional quality: 60 recommendations; organizational quality: 16 recommendations; and patient-oriented quality: 7 recommendations.

Fifteen of the 17 experts answered the questionnaire in the first round. Thirty-two of the 83 recommendations met our requirements: 15 for professional

Discussion

Systematic development of quality indicators based on evidence-based guidelines is a first step toward implementing a guideline. Measuring indicators gives insight into the current care and is a condition for developing a successful implementation strategy. Indicators can also be used for monitoring the quality of care, and they can contribute to a healthy self-regulation of the care because they transparently indicate improvement opportunities.

The guideline Non-small cell lung cancer: staging

Conclusion

Systematic developing and testing of indicators for improving the quality of care is very important for good implementation of multidisciplinary evidence-based guidelines. This set of indicators should be based on the available evidence, cover all dimensions of care and include patients’ and professionals’ opinions. In this article, we have described a procedure as objectively as possible for developing quality indicators by means of the practice example of the guideline for patients with

Acknowledgements

We thank all the members of the expert team for their contributions to the development of the set of indicators:

H. Groen, J. Annema, R. Termeer, H. Schouwink, E. Bollen, A. Verhagen, L. van Die, P. Helle, R. Wanders, E. Thunnissen, M. de Jonge, V. Tjan-Heijnen, B. Verhey, W. Ros, J. Salome, N. Schrama.

We also thank the members of the national board of patient representatives for lung cancer for their participation in the procedure and the registration clerks of the Comprehensive Cancer Centre

References (32)

J. Mainz
Developing evidence-based clinical indicators: a state of the art methods primer
Int J Qual Health Care
(2003)
J.L. Malin et al.
Evaluating the quality of cancer care: development of cancer quality indicators for a global quality assessment tool
Cancer
(2000)
M. Wensing et al.
Indicators of the quality of general practice care of patients with chronic illness: a step towards the real involvement of patients in the assessment of the quality of care
Qual Health Care
(1996)
S.M. Campbell et al.
Research methods used in developing and applying quality indicators in primary care
BMJ
(2003)
Evidence Based Richtlijn Ontwikkeling (EBRO). Richtlijn Niet-kleincellig longcarcinoom: stadiëring en behandeling...
J.P. Meerbeeck van et al.
Richtlijn ‘Niet-kleincellig longcarcinoom; stadiering en behandeling’
Ned Tijdschr Geneeskd
(2005)
M.R. Chassin et al.
The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality
JAMA
(1998)
R. Grol et al.
From best evidence to best practice: effective implementation of change in patients’ care
Lancet
(2003)
R. Grol
Personal paper. Beliefs and evidence in changing clinical practice
BMJ
(1997)
R. Grol
Implementation of evidence and guidelines in clinical practice: a new field of research?
Int J Qual Health Care
(2000)

R. Grol

Successes and failures in the implementation of evidence-based guidelines for clinical practice

Med Care

(2001)

L.A. Bero et al.

Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings

BMJ

(1998)

C. Cameron et al.

No impact from active dissemination of the Ottawa Ankle Rules: further evidence of the need for local implementation of practice guidelines

CMAJ

(1999)

J.S. Burgers et al.

Characteristics of effective clinical guidelines for general practice

Br J Gen Pract

(2003)

R. Grol et al.

Attributes of clinical guidelines that influence use of guidelines in general practice: observational study

BMJ

(1998)

Grol R, Wensing M, Eccles M. Improving patient care: the implementation of change in clinical practice. Elsevier; 2005....

Cited by (98)

Timeliness of Lung Cancer Care From the Point of Suspicious Image at an Urban Safety Net Hospital
2023, Clinical Lung Cancer
Timeliness of care is an important metric for lung cancer patients, and care delays in the safety-net setting have been described. Timeliness from the point of the suspicious image is not well-studied. Herein, we evaluate time intervals in the workup of lung cancer at an urban, safety net hospital and assess for disparities by demographic and clinical factors.
We performed a retrospective analysis of lung cancer patients receiving some portion of their care at Boston Medical Center between 2015 and 2020. A total of 687 patients were included in the final analysis. Median times from suspicious image to first treatment (SIT), suspicious image to diagnosis (SID), and diagnosis to treatment (DT) were calculated. Nonparametric tests were applied to assess for intergroup differences in time intervals.
SIT, SID, and DT for the entire cohort was 78, 34, and 32 days, respectively. SIT intervals were 87 days for females and 72 days for males (p < .01). SIT intervals were 106, 110, 81, and 41 days for stages I, II, III, and IV, respectively (p < .01). SID intervals differed between black (40.5) and Hispanic (45) patients compared to white (28) and Asian (23) patients (p < .05).
Advanced stage at presentation and male gender were associated with more timely treatment from the point of suspicious imaging while white and Asian were associated with more timely lung cancer diagnosis. Future analyses should seek to elucidate drivers of timeliness differences and assess for the impact of timeliness disparities on patient outcomes in the safety net setting.
Impact of Delays in Lung Cancer Treatment on Survival
2021, Chest
Timely care is an important dimension of health care quality, but the impact of delays in care on lung cancer outcomes is unclear. Quantifying the impact of delays in cancer treatment on survival is necessary to inform resource allocation, quality improvement initiatives, and lung cancer guidelines. Review of the available literature demonstrated significant heterogeneity between studies in terms of the impact of delay. Frequently paradoxical results were reported, with delay being associated with improved survival in patients with advanced disease. However, significant methodologic flaws were identified in many studies, which probably is the reason for the paradoxical results. The most significant methodologic limitations identified were incorrectly controlling for final pathologic stage (a mediator in the causal chain from delay to survival), failure to control for confounding by acuity of cancer presentation, and failure to consider effect measure modification. The effect of delay on survival probably varies by stage. The impact of delays is lowest for subcentimeter nodules, probably highest in stage II disease, and low in patients who are only eligible for palliative care. Precise quantification of the impact of delay is not currently possible. Given the available evidence, quality metrics for the timeliness of lung cancer care should focus on local barriers to care. These metrics should be carefully designed to take into account clinical-radiographic stage at initial presentation.
Proposed Quality Metrics for Lung Cancer Screening Programs: A National Lung Cancer Roundtable Project
2021, Chest
Lung cancer screening with a low radiation dose chest CT scan is the standard of care for screening-eligible individuals. The net benefit of screening may be optimized by delivering high-quality care, capable of maximizing the benefit and minimizing the harms of screening. Valid, feasible, and relevant indicators of the quality of lung cancer screening may help programs to evaluate their current practice and to develop quality improvement plans. The purpose of this project was to develop quality indicators related to the processes and outcomes of screening. Potential quality indicators were explored through surveys of multidisciplinary lung cancer screening experts. Those that achieved predefined measures of consensus for each of the validity, feasibility, and relevance domains are proposed as quality indicators. Each of the proposed indicators is described in detail, with guidance on how to define, measure, and improve program performance within the indicator.
The Critical Role of Clinical Practice Guidelines and Indicators in High-Quality Survivorship After Childhood Cancer
2020, Pediatric Clinics of North America
Key recommendations for high-quality female oncofertility care based on international clinical practice guidelines
2020, Reproductive BioMedicine Online
Citation Excerpt :
The members were asked for their background characteristics and to rate each recommendation on its importance for assessing quality of female oncofertility care on a Likert scale of one (extremely unimportant) to nine (extremely important) (Fitch et al., 2001; Campbell et al., 2003). Additionally, they were asked, per domain, to rank which recommendations they considered most important for assessing quality of female oncofertility care to differentiate between recommendations with a high Likert score (Hermens et al., 2006). Finally, they were asked to add recommendations which, in their opinion, were missing in each domain.
Which guideline-based key recommendations can be selected for high-quality female oncofertility care?
The Delphi method was used to select a set of key recommendations for female oncofertility care. First, recommendations from (inter)national clinical practice guidelines were selected in four domains: risk communication, referral, counselling and decision-making. Thereafter, they were scored, per domain, on their importance for high-quality oncofertility care by a multidisciplinary, oncofertility expert panel, consisting of patients, referrers and counsellors, in two Delphi rounds. Finally, the selected key recommendations were presented for approval in a third round. Differences in perspectives between subgroups of the expert panel were analysed.
A panel of 86 experts was asked to select key recommendations for high-quality oncofertility care. Eleven key recommendations were selected. Key recommendations in the domains risk communication and referral focused on information provision and offering referral to a reproductive specialist to female cancer patients. With the counselling domain, key recommendations focused on all aspects of counselling, including different methods, safety, pros and cons. In the decision-making domain, key recommendations focused on shared decision-making and supporting the decision with written information. The final set of key recommendations was approved by 91% of the experts. Differences in perspectives were found between subgroups. Patients found recommendations on decision-making and information provision more important.
A set of 11 key recommendations for high-quality female oncofertility care was selected by a multidisciplinary expert panel. The involvement of the perspectives of patients, referrers and counsellors led to this valid, acceptable and credible set of key recommendations.
Guideline-based quality indicators for early pregnancy assessment units
2020, Reproductive BioMedicine Online
What valid guideline-based quality indicators can measure quality of care in early pregnancy assessment units (EPAU)?
The systematic RAND-modified Delphi method was used to develop an indicator set from four evidence-based guidelines. An international expert panel was assembled to extract recommendations from these guidelines to establish quality indicators.
A total of 119 recommendations were extracted. Eleven recommendations received a high median score and top five score above the 75th percentile and were selected as key recommendations. The expert panel reassessed 15 high score recommendations and top five score between the 50th and 75th percentile as well as one high score recommendation without consensus. Eight of these 16 recommendations were selected in the second round as key recommendations. The key recommendations were formulated into a set of 19 quality indicators, summarized as follows: women referred to an EPAU could be seen within 24 h and receive a clear explanation on treatment options; designated senior staff members could be responsible for the unit and staff could have had ultrasound training; protocols could be available for daily practice covering all treatment options for miscarriage and ectopic pregnancy; and an EPAU could have access to urine pregnancy testing and serum HCG assays.
Nineteen quality indicators to measure early pregnancy care provided by EPAU were identified.

View all citing articles on Scopus

View full text

Development of quality indicators for diagnosis and treatment of patients with non-small cell lung cancer: A first step toward implementing a multidisciplinary, evidence-based guideline

Summary

Background

Methods

Results

Conclusions

Introduction

Section snippets

Background

Development of indicators from the professional's perspective

Development of indicators from the professional's perspective

Discussion

Conclusion

Acknowledgements

Developing evidence-based clinical indicators: a state of the art methods primer

Int J Qual Health Care

Evaluating the quality of cancer care: development of cancer quality indicators for a global quality assessment tool

Cancer

Indicators of the quality of general practice care of patients with chronic illness: a step towards the real involvement of patients in the assessment of the quality of care

Qual Health Care

Research methods used in developing and applying quality indicators in primary care

BMJ

Richtlijn ‘Niet-kleincellig longcarcinoom; stadiering en behandeling’

Ned Tijdschr Geneeskd

The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality

JAMA

From best evidence to best practice: effective implementation of change in patients’ care

Lancet

Personal paper. Beliefs and evidence in changing clinical practice

BMJ

Implementation of evidence and guidelines in clinical practice: a new field of research?

Int J Qual Health Care

Successes and failures in the implementation of evidence-based guidelines for clinical practice

Med Care

Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings

BMJ

No impact from active dissemination of the Ottawa Ankle Rules: further evidence of the need for local implementation of practice guidelines

CMAJ

Characteristics of effective clinical guidelines for general practice

Br J Gen Pract

Attributes of clinical guidelines that influence use of guidelines in general practice: observational study

BMJ