Implementation and Evaluation of Audit and Feedback for Monitoring Treat-to-Target (T2T) Strategies in Rheumatoid Arthritis Using Performance Measures

This article is published with digital features to facilitate understanding of the article. To view digital features for this article go to https://​doi.​org/​10.​6084/​m9.​figshare.​12961769.

Treating-to-target (T2T) is solidified as the preferred treatment paradigm in rheumatoid arthritis (RA) care [1]. T2T compels frequent reassessment of disease activity and adjustment of medications to achieve clinical remission, or at the very least, a low-disease activity state (LDA) [1]. Principles of T2T are endorsed by major rheumatology guidelines worldwide [1‐5] but practice variation may occur in busy clinical practices where the implementation of frequent clinical visits and use of composite measures of disease activity can be challenging [6].

Audit and feedback of clinical practice is one strategy that can be used to improve compliance with desired clinical behavior [7] such as adherence to T2T strategies, and it is endorsed as a mechanism for practice reflection and quality improvement by the Canadian Rheumatology Association (CRA) [8] and by the Royal College of Physicians and Surgeons of Canada [9]. Strategies for effective audit and feedback [10] have been leveraged by the Physician Learning Program (PLP) for physicians in the province of Alberta, Canada [11]. The PLP is available through a physician-driven Alberta Medical Association (AMA) benefit program with the vision that “by 2025, all Alberta physicians will care for patients in a supportive culture, driven by evidence-informed, reflective practice improvement.”

In the present study we collaborated with the PLP to operationalize and report on 5 performance measures from a recently developed national quality framework [12] to create individualized physician reports and presented the work in a group feedback session. The national quality framework was developed through extensive consultation with patients, healthcare providers, and healthcare leaders, and reflects the strategic priorities for the delivery of high-quality RA care [13]. From the collection of 21 performance measures (PMs), five which address T2T principles were selected for operationalization, testing and implementation using a pragmatic quality improvement and research platform called Rheum4U [12]. The measures address the following T2T concepts: frequent patient reassessment, monitoring patients using composite disease activity measures and targeting management to achieve remission or low disease activity (LDA).

The aim of this study was therefore to describe the operationalization and reporting of five PMs from the national RA Quality Improvement Framework [13] and to describe rheumatologists’ experiences with receiving individualized reports of their performance on the measures and the group feedback process.

In this study, we present the results of a quality improvement initiative using audit and group feedback to optimize T2T strategies in RA and report on rheumatologists’ experience with the process. For participating rheumatologists, this represents the first time they had viewed their data compared to their peers with respect to their RA practices as no current systems are in place in our center to track and trend rheumatologist performance or patient outcomes. The results demonstrate high rates of disease activity measurement that were facilitated by the Rheum4U platform used to collect the data; however, suboptimal follow-up times were found for patients not in remission, especially those with longer disease duration. Rheumatologists described many potential factors, including patient- and system-level factors, which could influence scheduling and may have contributed to this finding. Most rheumatologists participating valued the report and yet believed it would be unlikely to drive practice change as they thought they were in line with their peers in terms of patient outcomes. Those who thought their outcomes were not in line with their peers described possible contextual factors such as length of practice (with those in practice longer potentially having a larger number of patients with disease that was difficult to control or greater challenges adhering to follow-up standards given busy practices) or patient complexity that could explain differences seen and/or described possible changes in the way they would monitor disease activity in the future.

Rheumatologist behaviors in this audit and group feedback followed a particular predictable cyclical pattern as described by Cooke et al. [18], with the overarching themes of “interpreting the data”, “understanding evidence about best practices” leading to “change cues and change talk”. While our qualitative data were examined thematically and different named themes arose, many overlapped with the themes, subthemes, and sub-divisions from Cooke et al. [18]. For example, under Cooke’s overarching theme of “interpreting the data” are the subthemes of “reactions”, “understanding/questioning”, “justifying/contextualizing” and “reflecting”. In our qualitative work, we also elicited a theme of “reactions to individualized results”. The theme of “interpretation of results” also overlapped significantly with Cooke’s “justifying/contextualizing” theme as rheumatologists reflected on how data is affected by other “system”, “personal”, “patient factors” (Cook el al.’s subdivisions for this sub-theme) [18]. Our physicians did not find fault with the measures selected in relation to “evidence about best practices”. During our group feedback session “change cues and change talk” resulted in three main potential areas for change and quality improvement: (1) expansion of use of the Rheum4U platform (or data collection in general), (2) the need for an early RA protocol, and (3) expansion of similar initiatives in other patient populations.

Based on the feedback from rheumatologists received, the primary challenge encountered with this audit and feedback process was the data source. Our academic center has a hospital EMR that serves as a repository for patient information used in inpatient and outpatient settings but it does not collect, track or trend patient outcomes required for monitoring treat-to-target processes. This led to the development of a research and quality improvement platform (Rheum4U), although not all patients are enrolled in the optional platform, which impacted how rheumatologists viewed the results of our audit and feedback process. The present study has confirmed that the measures selected would be important to track on a broader scale amongst all clinic patients as they were important and relevant to rheumatologists and encouraged stricter adherence to T2T principles. This work also highlights the importance of tracking patient outcomes using the Rheum4U platform.

In the United Kingdom a national clinical audit of early inflammatory arthritis care was conducted which revealed high rates of DAS28 recorded in the charts (91%) at baseline with increasing rates of unrecorded DAS28 over follow-up [19]. In those with available data, > 60% achieved a reduction of > 1.2 from baseline DAS28 scores; however, 11% still had high disease activity after 3 months [19]. A major finding from the audit was a need to improve systems for capturing, coding and integrating patient data from outpatient clinics [19]. In the USA, the Rheumatology Informatics System for Effectiveness (RISE) is a national electronic health-record based registry that passively collects data from rheumatology practices and can be used to monitor quality of care [20]. RISE represents an example of a largescale effort to better track and trend quality of arthritis care. Electronic quality measures have been specified for use in RISE, which focus on processes of care [21]; however, due to the complexities of developing high-stakes RA outcome measures, which could affect rheumatologist compensation [22], work is still ongoing in the USA to develop and endorse RA outcome measures. Given the lower stakes of quality improvement in our center, no formal risk-assessment strategy has been employed for the outcome measures we evaluated. In Canada, primarily time-consuming chart reviews have been used in studies on audit and feedback [23], although methods of leveraging electronic records to evaluate quality of care are emerging [24].

While our study provides the first assessment of the RA PMs to facilitate T2T endorsed in the National RA Quality Framework [13], a number of limitations should be noted as well as potential consequences of this process. As highlighted by our rheumatologist participants, the high adherence to PMs observed may have been affected by patient selection, and the nature of the data source (e.g., Rheum4U data fields and automatic calculation of disease activity scores). Performance may be lower when measured on entire clinical practices, using EMR or paper chart reviews. The included measures did not account for time to treatment start. As treatment delays could impact time to remission/LDA, in future we aim to develop strategies such as linkage to pharmacy dispensation data to better determine timing of treatment changes/starts. We also did not capture insurance status of participating patients. In Alberta there is readily available medication insurance at low cost through a provincial program or through social assistance programs and we anticipate it is less likely that delays or lack of insurance coverage contributed to poor outcomes. Nevertheless, the predictors of poor outcomes according to insurance status and sociodemographic factors could be explored in the future. The results are also from a single academic center, which could have impacted performance results and rheumatologists’ views on quality reporting. It is likely that rheumatologists participating in Rheum4U, and particularly those agreeing to interviews had more favorable perspectives on quality monitoring than the general population of rheumatologists. A potential consequence of the voluntary nature of participation for patients and physicians in this quality improvement endeavor is the potential to further increase inequities in T2T practices between enrolled and non-enrolled patient participants who may differ on characteristics such as age, computer, and health literacy, and language spoken.

The audit and feedback of T2T measures has proven a valuable exercise for our center. In the future, we have plans to provide continuous reporting on these measures through a quality improvement initiative in our Department of Medicine and to expand measurement to other key areas. A new EMR will be available in our region shortly, and further work will aim to expand measurement to all patients seen in clinic to better track and trend areas in need of quality improvement among the entire patient population. Optimizing T2T strategies takes time and effort and health system infrastructure including IT platforms to assist with tracking/trending, viewing and imputing the data required for these measures will be central to a sustainable quality improvement strategy. Relying on busy individual providers to make shifts in practice without additional resources and/or incentive to do so is likely to be less successful. Leveraging allied health providers in rheumatology in the collection and tracking of quality is another strategy that should be explored during further implementation to improve sustainability of tracking and improving T2T strategies. A key area of future study will be to evaluate the impact of efforts to provide quality reports to rheumatologists on improving practices and ultimately patient outcomes.