Development of computerised adaptive testing (CAT) for the EORTC QLQ-C30 dimensions – General approach and initial results for physical functioning
Introduction
The primary source of information about patients’ health-related quality of life (HRQOL) is self-report questionnaires, also termed patient-reported outcomes (PROs). These instruments have typically been developed and implemented using classical methods like sum scoring (adding the scores to individual items into scales). However, these classical methods have some limitations. For example, all patients have to answer the same set of items for sum scores to be comparable. This means that PRO instruments often constitute a compromise between optimal measurement (requiring longer instruments) and reasonable response burden (requiring shorter instruments).
In recent years there has been an increasing interest in methods based on item response theory (IRT)1, 2 for measuring HRQOL (see e.g.3, 4, 5, 6, 7, 8, 9, 10). IRT is a statistical framework for assessing the characteristics of the items in multi-item scales. IRT models estimate among other things the ‘difficulty’ of each item. For example, taking a long walk is more demanding/difficult than taking a short walk. Hence, asking patients with good physical functioning about taking a long walk may be highly informative while for patients with poor functioning an item about taking a short walk may be more informative. IRT-based methods take this into account.
One of the major advantages of IRT methods compared to classical methods, and one of the reasons for the interest in these methods, is that when a set of items has been calibrated (estimated) to an IRT model all scores based on any subset of the items are on the same metric. That is, even if two patients answer different subsets of items from the same item pool, their scores are directly comparable. This unique feature means that a questionnaire can be adapted to the individual to optimise the measurement properties, yet at the same time comparability of scores are maintained across patients. This possibility for adapting the instrument is fully utilised in a computerised adaptive test (CAT)11 to construct individualised instruments: based on the responses to the preceding items, a computer programme evaluates which item should be asked next to obtain maximal information. The additional items are administered until a predefined level of information (i.e. precision) has been reached or until a predefined number of items has been administered. Fig. 1 shows two simple examples of how a CAT measuring physical functioning could proceed. If the patient reports few problems on an item, the next item will concern a more demanding task, while if severe problems are reported the following item will concern a less demanding task. In this way the questionnaire is individualised, using the most informative items for each patient. In contrast, traditional questionnaires ask the same items to all, and hence, generally need more items to obtain the same level of precision.
CAT measurement has several advantages compared to traditional questionnaires including: increased measurement precision and/or reduced response burden, increased flexibility as the questionnaire can be adapted to the individual study or patient, avoidance of asking uninformative questions, and immediate calculation and presentation of results.
Because of the clear advantages of CAT measurement a number of research groups are developing CAT’s for measurement of HRQOL, including the Patient-Reported Outcomes Measurement Information System (PROMIS)12, QualityMetric13, and the European Palliative Care Research Collaborative.14
The European Organisation for Research and Treatment of Cancer Quality of Life Group (EORTC QLG) has initiated a CAT-project for the HRQOL dimensions measured by the EORTC Quality of Life Questionnaire (EORTC QLQ-C30).15 The EORTC QLQ-C30 is one of the most widely used quality of life questionnaires in cancer research.16, 17 It consists of 30 items measuring 15 aspects of HRQOL: five functional measures, nine symptom measures and one measure of overall health/quality of life.18 At present the EORTC QLQ-C30 exists only in a traditional version where all patients are asked the same 30 items.
The aim of the EORTC CAT-project is to measure the same 15 HRQOL dimensions as measured with the QLQ-C30, but with higher efficiency and precision. This requires new items supplementing the existing items, filling ‘gaps’, e.g. new items enabling assessment of patients in very poor condition. Initial investigations indicated that it might be difficult to develop enough relevant but different items for ‘overall quality of life’. Therefore, the project focuses on developing CAT for the other 14 HRQOL dimensions of the QLQ-C30. Basing the CAT on the QLQ-C30 ensures backward-compatibility with the substantial literature of studies using the QLQ-C30. The CAT instrument aims to measure the same well-validated and well-known HRQOL dimensions with significantly improved precision. Most other CATs do not have such a direct link to an existing instrument. Instead they are developed ‘from scratch’ and are unconstrained by existing conceptual frameworks, resulting in a completely new instrument. This gives greater freedom in the development process, but may also result in unfamiliar measures.
The purpose of the current paper is to provide an overview of the methods used to develop the new EORTC QLQ-C30 CAT instrument, illustrated by the development of CAT for physical functioning (PF).
Section snippets
Methods
The development of the EORTC CAT is conducted by members of the EORTC QLG. This international CAT-project group includes oncologists, psychologists, statisticians, and others with considerable experience in developing HRQOL instruments.
The development of the item pools forming the basis for the CAT can be divided into four phases: (1) literature search, (2) operationalisation (selection and formulation of items), (3) pre-testing (patient interviews), and (4) field-testing (data collection and
Results
We illustrate the methodology using the results of the first three phases of the development of the item pool for PF. The field-testing is in progress and the results of this and the psychometric analyses (phase 4) will be described in detail in a separate paper.
Discussion
CAT measurement has several advantages compared to traditional questionnaire measurement. In particular, it can improve measurement precision without increasing the response burden for the patients. To utilise these new methods to improve the measurement of cancer patients’ HRQOL the EORTC QLG has initiated a project to develop CAT for the widely used EORTC QLQ-C30 questionnaire.
The EORTC CAT development can be divided into four phases: literature search, operationalisation, pre-testing, and
Sources of support
The study was funded by grants from the EORTC Quality of Life Group.
Conflict of interest statement
None declared.
Acknowledgement
This study was funded by grants from the EORTC Quality of Life Group.
References (32)
- et al.
Item response theory was used to shorten EORTC QLQ-C30 scales for use in palliative care
J Clin Epidemiol
(2006) - et al.
A first step in the development of an international self-report instrument for physical functioning in palliative cancer care: a systematic literature review and an expert opinion evaluation study
J Pain Symptom Manage
(2009) - et al.
Fundamentals of item response theory
(1991) - et al.
Handbook of modern item response theory
(1997) - et al.
Calibration of an item pool for assessing the burden of headaches: an application of item response theory to the headache impact test (HIT)
Qual Life Res
(2003) - et al.
Use of item response theory to develop a shortened version of the EORTC QLQ-C30 emotional function scale
Qual Life Res
(2004) - et al.
Item banks and their potential applications to health status assessment in diverse populations
Med Care
(2006) - et al.
Assessment of health-related quality of life in arthritis: conceptualization and development of five item banks using item response theory
Health Qual Life Outcomes
(2006) - et al.
The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH roadmap cooperative group during its first two years
Med Care
(2007) - et al.
IRT health outcomes data analysis project: an overview and summary
Qual Life Res
(2007)
Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement
Qual Life Res
Computerized adaptive testing: a primer
Symptom assessment in palliative care: a need for international collaboration
J Clin Oncol
The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology
J Natl Cancer Inst
Cited by (55)
Patient-reported outcome measures in cancer care: Integration with computerized adaptive testing
2023, Asia-Pacific Journal of Oncology NursingHealth state utility differed systematically in breast cancer patients between the EORTC QLU-C10D and the PROMIS Preference Score
2022, Journal of Clinical EpidemiologyDevelopment of an EORTC Item Bank for Computer-Adaptive Testing of Nausea and Vomiting
2022, Seminars in Oncology Nursing
- 1
Grant National Science Council, Taiwan, No. 95-2314-B-002-266-MY2, 97-2314-B-002-020-MY3.