Use of daily electronic patient-reported outcome (PRO) diaries in randomized controlled trials for rheumatoid arthritis: rationale and implementation

Rheumatoid arthritis (RA) is a systemic, inflammatory, autoimmune disease. RA has variable expression and outcomes, ranging from mild to severe, and is associated with progressive joint destruction, significantly compromised health-related quality of life (HRQoL), and reduced survival [1, 2]. Clinical trials assess the effect of drugs on certain aspects of RA, including symptoms, structural joint damage, and biomarkers of inflammation [3]. However, assessments undertaken by physicians in a clinical trial may not adequately capture the impact of disease from a patient’s perspective [3, 4]. Symptoms and impacts of the disease are best known by the patient and therefore best measured by patient-reported outcomes (PROs) [3]. The Food and Drug Administration (FDA) guidance for industry recognizes that PRO instruments should be used in measuring concepts best known by the patient or best measured from a patient’s perspective in clinical trials [5]. There has been a growing interest in the use of PROs in RA [6, 7]. Patients with RA have identified pain, physical function, fatigue, joint stiffness, participation, sleep, and emotional and psychosocial factors as important domains for the assessment of HRQoL [8‐10].

Traditionally, in randomized clinical trials (RCTs) and clinical care, PRO measures have been administered to patients in paper format during scheduled visits or face-to-face encounters [11, 12]. Paper and electronic diaries have been used in studies to capture patient-reported events between visits, more proximate to their occurrence [13]. However, paper-based methods have the risk of participants completing multiple days of reporting all at once, referred to as the “parking lot effect,” in which all responses may be completed just before the visit while sitting in the parking lot. Thus, there is an increasing use of electronic PRO (ePRO) systems to gather PROs for more accurate and complete data capture, improved compliance, decreased likelihood of data entry errors, reduced administrative burden, and higher acceptance by respondents [11]. Most ePRO systems provide date and time stamps for each data entry and restrict data entry to specific periods, thereby avoiding backward or forward filling of entries. ePRO systems can also reduce missing or unusable data by ensuring data capture at the right time [11]. Additionally, there is a potential for diminished accuracy of reporting with respect to recall of more distant events, which exists with all instruments [13]; daily assessments of PROs may capture patient experiences more accurately by minimizing recall bias.

With the increased use of electronic systems to improve documentation of source data in clinical trials, the FDA has endorsed electronic capture of clinical trial source data, including PRO endpoints [14], and has preferred the electronic form over paper-based data collection [11]. The FDA has developed the guidance for industry on the use of computerized systems for clinical investigations [14]. The FDA has also recognized that source data captured electronically should be attributable, legible, contemporaneous, original, and accurate (ALCOA) to ensure data quality and integrity, and must meet regulatory requirements for recordkeeping [15]. To ensure appropriate implementation of ePRO systems, several recommendations have been published by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) [11, 16‐19].

Electronic PRO data collection in clinical trials can be conducted on site (clinic) or off-site (patient’s home, workplace, or school); the off-site assessment is considered to be optimal, as it allows for accurate and real-time reporting of symptoms by patients. Handheld touchscreen-based devices have become the mainstay for unsupervised, off-site PRO data collection, especially for clinical trials requiring frequent data entry [11]. ePRO data collection has been utilized in RA RCTs to record pain, disability, and tender joint counts during patients’ study visits and is preferable to patients compared to paper-based measurement [20].

Although patients have identified morning joint stiffness (MJS) and fatigue as important disease-related symptoms [21‐24], none of these are included in the composite disease activity indices, such as Disease Activity Score 28 (DAS 28), Clinical Disease Activity Index (CDAI), and Simplified Disease Activity Index (SDAI), which are used in RA clinical trials. As part of a recent clinical development program of baricitinib for the treatment of RA, PRO measures reflecting Duration and Severity of MJS, Severity of Worst Tiredness, and Severity of Worst Joint Pain were developed and incorporated into the RA-BEAM (NCT01710358) and RA-BUILD (NCT01721057) phase 3 RCTs [25, 26], considering the importance of PROs in RA. A handheld ePRO device was used to capture these measures daily. This manuscript summarizes the rationale for the use of the daily ePRO electronic diary, its implementation in the baricitinib phase 3 studies, and real-time patient compliance to the diary.

The daily electronic PRO diary was used by patients with moderately to severely active RA, who were naïve to biologic treatment, and enrolled in the RA-BUILD study (N = 684) and the RA-BEAM study (N = 1305). The results of PRO assessments using the electronic diary in patients with moderately to severely active RA have been previously reported [25, 26], and the criteria for sufficient reliability, validity, responsiveness, and interpretation standards have been fulfilled. Prior qualitative studies concluded that the diary instructions were clear, items were relevant, and the devices were easy to use [27, 28].

The overall compliance with daily electronic diary reporting through Week 12 was high (more than 90%) in both the RCTs. Overall, the compliance rates were observed to be similar for both the trials: 94% in the RA-BEAM trial and 93% in the RA-BUILD trial.

The percentage of patients completing the daily electronic diary was also more than 90% throughout Week 1 to Week 12 for both the trials. The percentage of patients with missing data was high at Week 1 compared to other weeks, which may be attributable to improper device set-up and/or timing of initial data collection (Table 2). The average number of missing days per week was reported to be low in both trials (Table 3). As observed for the percentage of patients with missing data, the average number of days of missing data was also high in Week 1 compared with the other weeks. No specific pattern was observed for missing data across the remaining weeks.

Several reasons were noted by the study team for missing diary data at baseline (day of first dose) (Table 4). First, if the preferred reporting window closed prior to baseline device set-up, then the window would not reopen until the next day. If the preferred reporting window was 6:00 a.m. – 11:45 a.m. or 12:00 p.m. – 5:45 p.m. and the patient’s baseline visit occurred during or after the reporting window, the first reminder alarm would not sound until the reporting window for the next day. Second, the users abandoned or delayed the report past the reporting window. Third, patients could have ignored the device alarms or left the device out of hearing range; however, a record of the missed alarms was captured in the log file. Fourth, the sites were required to provide patients with a fully charged device; however, low battery levels (battery level ≤ 5%) were detected at the baseline visit date in some cases. This may have interfered with a patient’s ability to answer the daily diary report. Fifth, a few site errors were also noted, including delay or failure to provide the device to the patient at the baseline visit. In both the RA-BEAM and RA-BUILD trials, “first alarm next day” was the most common reason (19.6% in the RA-BEAM trial and 30.4% in the RA-BUILD trial) for missing baseline diary data.

During the course of this program, the study team learned that there should be a more robust process for tracking lost or defective devices. However, based on the high post-baseline compliance rate, we can surmise that lost or defective devices had minimal impact on data completeness.

Estimates of compliance with paper diaries have been reported to be high in prior studies [30]. However, these estimates have been derived from unconfirmed recordings of diary completion by patients, which might not be accurate. Stone and colleagues compared patient compliance with a paper diary instrumented to track diary use versus an electronic diary in patients with chronic pain [30]. Paper diaries were reported to demonstrate high overall compliance, but the per protocol or actual compliance was very low (11%), which was possibly attributable to the “parking lot effect” [30]. Compliance with the electronic system was reported to be high (94%), which was similar to the findings of our analysis. As reported by previous studies [31, 32], appropriate training, clarity of instructions, a simple user interface, and design of the electronic device to assure protocol compliance might be potential factors contributing to the high compliance rate.

Although the daily electronic diary was effective in collecting data in the two clinical trials, and high compliance was observed, several technical and process issues were identified that should be considered when implementing daily electronic diaries in future studies. These include technical issues with the electronic system (e.g., battery charging) that could limit the utility of electronic diaries in some circumstances, provision of devices in a timely fashion to study subjects, inadequate patient training, improper device set-up, and timing of initial data collection. A comparison of compliance rates between trial arms and between demographic groups was not assessed in this study but should be considered for potential future work. The potential for high compliance rate related to researcher follow-up when the diary was not completed for 3 days could also be explored in future studies.

Notwithstanding the above considerations, daily electronic diaries would be expected to reduce recall bias and enhance the sensitivity to detect changes in PRO data, resulting in reduced error variance [11]. Results from this study suggest that daily electronic diary capture is feasible in large multinational RCTs and further support the use of this method of PRO collection.

The implementation of daily electronic diaries in the two phase 3 RCTs in patients with RA and the high compliance rates reported by the patients enrolled in these trials support the PRO data collection using daily electronic diaries. Opportunities to employ PROs and diary technology may enhance the monitoring of quality of life for people with RA between face-to-face study visits and clinical encounters. Capturing and reporting more reliable PRO data, collected in real time, can provide further evidence of beneficial treatment effects on outcomes that matter most to patients. Even though RCT sponsors and investigative sites can anticipate certain technology challenges and training needs in deploying electronic diaries, our experiences reported here and in previously reported studies [27, 28] indicate the feasibility, acceptability, and quality of PRO results using this method.