Preliminary development of recommendations for the inclusion of patient-reported outcome measures in clinical quality registries

Patient reported outcome measures (PROMs) are designed to assess various dimensions of a person’s health and well-being from the perspective of the individual themselves. Beyond assessing treatment effectiveness in the context of clinical trials and other research activities, PROMs have been used in clinical practice, supporting patient-centred care and shared clinical decision making [1]. The use of PROMs has been demonstrated to: 1) enhance quality of care and decision making in routine care for cardiovascular disease [2], and 2) identify clinical best practice and improve average health outcomes through tracking health and disseminating outcomes from clinical quality registries (CQRs) [3].

CQRs are organisations that systematically monitor the quality of healthcare within specific clinical domains by routinely collecting, analysing and reporting health-related information [4]. They use predefined indicators to assess variation across structural, process and outcome measures in order to benchmark quality of care [5]. CQRs have received increasing attention as a means of improving quality and reducing the cost of health and medical care, through identifying variations in clinical practice and care, and assessing the uptake of effective treatment [6, 7].

Data collected using PROMs and integrated in a feedback mechanism within a CQR can be used to track the benefit of clinical interventions with the potential to improve shared decision-making and treatment outcomes for patients [8]. The inclusion of PROMs in CQRs offers numerous advantages [9]. First, incorporation of the patient voice regarding their lived experiences ensures that health outcome measurements of care are patient-centred. Further, symptom burden and quality of life (QoL) are dynamic variables that cannot be recreated accurately through retrospection; they are essentially lost if not captured “in the moment”. For this reason, routine, systematic, and longitudinal collection of PROMs have been recommended as a standard aspect of clinical practice [10, 11]. Likewise, longitudinal collection of PROMs, in addition to clinician derived medical data in CQRs, can improve understanding of the trajectory of an individual patient’s symptom burden and QoL over the course of the disease or treatment. This can inform clinicians of the variability between patient groups, provide information on the value patients place on their health status and to predict patient outcomes [9].

Numerous guides and recommendations were developed to promote patient-centered care and PROMs use in clinical practice. Users’ guide to integrating patient-reported outcomes in electronic health records provides recommendations for integrating PROMs into electronic health records, thus enabling use of outcome data for multiple applications [12]. The International Society for Quality of Life Research (ISOQOL) guide [13] provides options for how to select PROM measures, as well as guidance on data collection and reporting in clinical practice. The purpose of this guide is to help clinicians who are interested in using PROMs in their clinical practice as a tool in patient management. Similarly, guidelines have been developed for inclusion of PROMs in clinical trial protocols [14].

The above listed guides provide recommendations for clinicians capturing PROMs data in clinical practice and to tailor care to individual needs. Clinical registries play an increasingly important role as a stimulus for quality improvement by providing high-quality data and analyses that are respected by clinicians [15, 16]. PROMs in CQRs are used for reporting and benchmarking purposes [16, 17]. Implementation of lessons learned from CQRs that include PROMs will assure patients achieve optimal management of the disease and functional gain with minimal adverse events [9]. Although some registries have included PROMs as part of their current practice [18, 19], widespread adoption of PROMs as a key component in CQRs is yet to occur. PROMs are increasingly being introduced into CQRs in Australia. For example, the Victorian Orthopaedic Trauma Outcomes Registry [20] and the Prostate Cancer Outcomes Registry – Victoria [21] both collect PROMs at a time of clinical stability. PROMs data collection is currently being considered by the Australian and New Zealand Thyroid Cancer Registry (ANZTCR) [22] and the Australasian Pelvic Floor Procedure Registry (APFPR) [23].

There are a range of methodological considerations required for PROMs implementation in registries to ensure they provide the most benefit and deliver measurable and actionable outcome data, particularly as incorporating PROMs into CQRs is likely to be costly and time-consuming. Clear recommendations are needed to support ethical, effective, and transparent use of PROMs collected across all CQRs [9, 24].

The aim of this project was to develop, using a Delphi method, a set of recommendations for PROMs inclusion in a CQR setting. This publication is the second in a series describing the development of evidence-informed guidelines for PROMs inclusion within CQRs in Australia. The preceding study developed a conceptual framework for the inclusion of PROMs in CQRs, which classified findings, from both the literature and the survey of 66 Australian registries, into broad categories ranging from initial development to outcome dissemination providing the structure for development of recommendations, engaging national and international leaders in health-related QoL research, clinicians, researchers, patient advocates and consumers [25].

This was a novel study to investigate the role of PROMs in CQRs and to develop preliminary recommendations for PROMs inclusion in clinical registries. From 117 potential statements following round 2 and addition of new statements, 62 were proposed for inclusion to the final set of recommendations across eight domains: Rationale, Setting, Ethics, Instruments, Administration, Data management, Statistical methods, and Feedback and reporting.

Successful PROMs implementation in CQRs includes many challenges and requires clinical, operational, and analytic resources and expertise. Recommendations for PROMs implementation are critically important for registries to assure meaningful PROMs data capture, use, interpretation, and reporting [16]. The newly developed recommendations complement the PROMs framework for CQRs [16] and provide a set of guiding principles for implementing PROMs in CQRs to provide maximum value and best outcomes. These recommendations guide the user in a stepwise manner from conception through to operational considerations and reporting. These include rationale for PROMs data collection, setting (e.g. population size), ethics and consent arrangements, instrument selection, mode, method and frequency of administration, data management, statistical methods for PROMs data collection, and feedback and reporting of the data.

The addition of PROMs into CQRs can be used to maximise the benefits of current registry objectives, or alternatively the addition of PROMs may extend the registry’s scope and enhance the registry’s utility [9]. Registries should consider piloting PROMs implementation before the full rollout to assess feasibility and sustainability from the patient, clinician and/or system perspective. A sustainable approach to using the PROMs may require significant long-term commitment of budget, resources to build a coherent system, and active support from diverse organisations [30].

With regards to instruments adopted, the panel recommended registries include both generic instruments (designed for use among diverse populations with a broad range of medical conditions) and condition-specific instrument/s, translated into multiple languages as needed. PROMs selection should be based on recommendations by experts in the field, as well as completion time for patients, license and administration costs and overall patient burden.

Previous studies demonstrated a number of issues relating to the administration and response rates of PROMs in registries [21, 31]. In our study this was reflected in the expert consensus that PROMs should be administered via multiple methods to increase response rate. In regards to the timing of data collection, it is recommended that PROMs be administered at various time points (e.g. baseline, single or multiple time points).

The experts in our study agreed that PROMs data should be shared with various stakeholders, including patients and clinicians to inform outcomes at the health service level; and funders and service providers, to inform policy, practice and reimbursement of healthcare services. Future research should examine how PROMs completion and feedback develops, and is in turn influenced by, the process of building relationships with patients, in addition to the impact of PROMs collection on information exchange and decision making. It is also important to consider ethical issues and purpose of the PROMs data access, management of concerning PROMs data, so that participants’ information is not be released outside without the permission of the participant [32].

This was the first study to develop preliminary recommendations for PROMs inclusion in CQRs using a Delphi consensus process [26]. Although the study recruited international participants, the Delphi panel was dominated by participants from Australia, which may limit the generalisability of the recommendations. Therefore, these recommendations may not be as relevant to other jurisdictions outside Australia. Despite there being no strict guidelines for sample size in a Delphi study, a relatively small panel size was another limitation of our study. A minimum sample size of 10 is usually recommended to obtain enough information and make valid conclusions of the research study [33]. Panels of similarly trained experts provide effective and reliable utilization of a small sample from a limited number of experts in a field of study to develop reliable criteria and recommendations that support effective decision-making [34].

The study would also be strengthened by including patients and other consumers with an interest in CQRs. Patient involvement with greater diversity will be important for future work and implementation of the recommendations.

Recommendations that are both practicable and robust in the interpretation of an evidence base can be achieved with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach (also known as a method of assessing the certainty in evidence and the strength of recommendations in healthcare) [35, 36]. Due to limited literature in this area, it would not be appropriate to use GRADE on our recommendations.

The recommendations for PROMs implementation in CQRs represent a valuable resource that can be used for educating registry managers, researchers and clinicians on the effectiveness of collecting, analysing and acting upon PROMs data to improve health outcomes, and to support PROMs implementation and use.

Developing preliminary recommendations is an important first step, as is supporting PROMs collection and reporting in CQRs and finally, for evaluating key patient outcomes. Next steps will involve testing and evaluation of the newly developed recommendations in the registry settings, which may lead to revisions of the recommendations from this study. Qualitative studies with registry managers and stakeholders including patients are planned to determine the utility and impact of the recommendations.” Further study involving international data custodians of large CQRs, QoL experts and PROMs specialists will be conducted to develop a user guide for PROMs inclusion in CQRs and a checklist for PROMs data collection and reporting in the registry setting.