Background
Clinical quality registries are increasingly common for tracking and investigating healthcare quality. The choice of outcomes is of paramount importance for capturing relevant and comparable information on the quality and effectiveness of care, and outcome harmonization is vital for improving patient results through activities such as organizational benchmarking and registry-based research. In this work, we set out to compare outcomes from clinical quality registries against a general, non-disease-specific outcome framework in a secondary and tertiary care academic hospital setting.
Outcomes are broadly defined as measurements or observations used to capture and assess the effect of treatment [
1] and what people care about most when seeking treatment [
2,
3]. In the context of clinical registries, outcomes are the recorded results of care.
Clinical quality registries record information about patients, their health status, and the healthcare received, typically focusing on patients with similar needs, medical conditions, or use of healthcare services [
4]. Healthcare professionals, hospital managers, and other decision makers use registry data to monitor and assess healthcare outcomes (i.e., how different patients respond to different treatments or interventions), thus enabling evaluation of the quality and effectiveness of care [
4,
5] and the improvement of patient care [
6]. Benchmarking is a common and effective method for measuring, analyzing, and ultimately improving organizations’ performance by comparing the data on activities and results between similar organizations, including best-practice facilities [
7]. Harmonization of outcome measurement is important for benchmarking and necessary to unleash the potential of clinical quality registries [
6,
8,
9]. In quality registries, the benchmarking might entail the maturity of the registries themselves and ultimately the results of care recorded in the registries. Registries also hold a great untapped potential for real-world research [
10‐
12]. Ideally, the framework for outcome choices would serve these different purposes.
Previously, there have been some efforts to harmonize the choice of outcome measures in clinical registries. The Outcome Measure Framework (OMF) of the American Agency for Healthcare Research and Quality (AHRQ) was created for registry development purposes [
13‐
15]. In Sweden—a forerunner in clinical quality registries—patient-reported outcome measures (PROMs) in registries have been analyzed at least twice [
16,
17], and the International Consortium of Health Outcome Measurement (ICHOM) has developed disease-specific outcome measurement standard sets [
18,
19] to steer the choice of outcome measures. To date, however, the ICHOM standard sets do not seem to use any general outcome framework. Some health system authorities have developed outcome frameworks for reporting and monitoring purposes (e.g., NHS [
20]) and others have developed frameworks for evaluating clinical registry maturity, capability, data quality, or design [
21‐
25]. These frameworks are not, however, intended for classifying the choice of outcomes in registries.
For clinical trials, harmonization efforts of outcome measurement have taken strides forward, especially with core outcome sets (COS) that have been created for numerous clinical fields [
26].
The COMET taxonomy, created by Dodd and others [
27], is an outcome framework intended for developing and assessing COS in clinical trials and is based on previous conceptual and empirical work [
28]. In a preliminary phase of this study, we performed a literature review in which we identified a total of 23 outcome frameworks that could be used in classifying choices of outcomes (see Additional file
1) and chose the COMET taxonomy for this study. In the deliberations, the following advantages were valued by the research team and hospital management in the COMET taxonomy: It classifies outcomes relevant to patients, including physiological outcomes and patient impact [
18]; it is aligned with outcome unification efforts in the clinical trial setting [
29,
30]; it is sufficiently granular without compromising comprehensibility (38 outcome domains classified into 5 core areas); it has instructions on classifying outcome measurement instruments to ensure consistency [
31]; and it includes categories for resource use, which was seen as relevant for managerial and cost-effectiveness assessment purposes.
Use of the COMET taxonomy in the clinical quality registry context could offer a possibility to bring outcome measurement in clinical trials and registries a step closer, thus enabling broader registry research, comparability of findings, and better translation of clinical research results to the real-life context.
The aim of this study is to compare existing clinical registry outcomes against the COMET taxonomy and to assess the framework’s suitability in the clinical registry setting.
The goals of this research are as follows:
(1)
to classify the outcomes in HUS Helsinki University Hospital somatic clinical quality registries with the COMET taxonomy, and
(2)
to assess the suitability of the COMET taxonomy in classifying the choice of outcomes in real-world clinical quality registries.
Additionally, the work describes a practical example of how to carry out such a classification effort and provides a benchmark for other clinical quality registry evaluators.
Methods
First, criteria for suitability of the COMET framework in classifying outcomes were discussed and agreed on within HUS Helsinki University Hospital quality management and the research team: The framework should 1) be feasible, meaning that the classification effort could be carried out in reasonable time and with reasonable resources; 2) have the ability to distinguish development needs in registries, meaning that the results can point to shortcomings in outcomes and differentiate between registries’ development stages; and 3) enable the classification of each outcome measure as unambiguously as possible. These criteria will be discussed in the Discussion -section of this article.
Second, we conducted an organizational case study with HUS Helsinki University Hospital (later HUS) clinical quality registries. Research data consisted of clinical outcome fields gathered from HUS somatic clinical quality registries. A total of 63 medical condition- or healthcare service (i.e., treatment)-specific [
4] somatic registries were included in the study (Table
2).
HUS is a secondary and tertiary care academic hospital with 27,000 employees that serves a population of 2.2 million in Southern Finland. Certain disease entities have been centralized to HUS nationally (total population of 5.5 million). Annually, around 680,000 individual patients (2.7 million visits) are treated at HUS [
32]. HUS has deployed clinical quality registries for clinical use, quality, effectiveness, and research purposes. Teams of expert clinicians have chosen the outcome measures, thus representing a local (or, in some registries, a national) expert consensus. The outcomes are recorded in a structured format, ensuring high measurement consistency. The broadness across disease areas combined with accessibility make the HUS quality registries an excellent case study target for the validation of outcome frameworks.
All data entry field titles were extracted from registry interfaces (59 registries) or technical definition documents (4 registries) and gathered in a separate research table. The title of each data entry field corresponds to the name of the variable. We also extracted items from the reporting functionalities, if functional. For each data entry field, we recorded data category, data subcategory, data entry field title, and input unit (e.g., kg, cm) or input choices (drop-down list or open text). No personal data were collected. Furthermore, for each registry, we recorded the following: the number of patient entries, reporting functionality (yes or no), and patient questionnaire functionality (yes or no).
Third, each input field was assessed with the following process: 1) identify whether the item is a potential outcome measure; 2) if yes, classify the item into the corresponding outcome domain(s) within the COMET taxonomy. Additionally, we chose to characterize outcome measurement instruments in our data with the following methods found in the literature:
-
Measurement method: Physiological measure (e.g., blood sugar)/professional-reported measure (contains a significant degree of subjectivity, e.g., Eastern Cooperative Oncology Group Performance Status Scale, ECOG)/PROM (e.g., EQ-5D)/patient-reported experience measure (PREM, e.g., patient evaluation of communication quality) (adapted from [
33,
34]).
For standardized patient-reported questionnaire instruments:
-
Scope: General (e.g., EQ-5D)/disease-specific (e.g., Oxford Hip Score) [
35]
-
Dimensionality: Composite (e.g., EORTC QLQ-C30)/unidimensional (e.g., unique question of global quality of life) [
35]
All methods were carried out in accordance with relevant guidelines and regulations. The study was reviewed and approved by the Research Administration of the Helsinki and Uusimaa Hospital District (Research Director resolution, 20 February 2020), and the data security plan and measures were implemented in accordance with relevant guidelines.
Conclusions
In conclusion, we found the COMET taxonomy to be mostly suitable and useful in a clinical quality registry context, and with some reservations, we would recommend its use for clinical registry developers, researchers, and hospital managers to assess outcome measurement and to guide the choice of outcomes. Our main concerns relate to the ambiguity of certain domains of the framework, which should be considered in similar classifying efforts and in future development of the COMET taxonomy and its guidance. Use of the COMET taxonomy in conjunction with characterization of measurement method should be sufficient for benchmarking registry maturity and could bring us one step closer to efficient quality of care benchmarking between organizations [
9]. We believe that there are benefits to sharing the same model between clinical trials and registries; it steers registry development and research, leads to more comparable and relevant clinical registry data, bridges the gap from trials to practice by helping understand clinical trial results in a local context, and encourages registry research that could combine data from multiple organizations.
Our research on HUS Helsinki University Hospital registries supports previous findings of variation in the choice of outcomes in clinical quality registries and the need for harmonization. There are very few published reviews that cover larger numbers of registries. More primary research on registries at the national and international levels is needed, as well as meta-analyses of existing narrower reviews. In our view, there is a clear need for a unified framework to elucidate the full picture of outcome choices.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.