An evaluation of risk-based monitoring in pragmatic trials in UK Clinical Trials Units

Monitoring of clinical trials forms one of the main approaches to ensuring that quality standards are delivered in line with International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Good Clinical Practice (ICH-GCP) guidelines [1]. Trial sponsors are expected to have monitoring arrangements in place to ensure that regulatory obligations are met, the safety and well-being of participants are maintained, and scientific integrity is retained [1]. The ICH-GCP guidance does not specify methods to be used but recommends “considerations such as the objective, purpose, design, complexity, blinding, size and endpoints of the trial” [1].

Historically, trial monitoring has been onerous, involving numerous on-site visits and up to 100% source data verification (SDV) [2] and having large implications for trial management and budgets. SDV is the process by which the data collected for a trial are compared with the original source of information. More recently, there has been a shift toward increased remote monitoring conducted by the trial sponsor or coordinating trials unit or both [3]. Central checks can be carried out on electronic records, consent forms and overall performance of participating sites [3], highlighted by the recent development of metrics in this area [4]. Although this may reduce the number or duration of visits, central monitoring has its own limitations, including access to the source data and reliance on sites maintaining data collection records. A 2012 survey highlighted that of 48 UK Clinical Research Collaboration (UKCRC) registered Clinical Trials Units (CTUs), 35% indicated that all trials had a documented monitoring plan, and most CTUs used some level of central monitoring, in some cases combined with on-site monitoring [5].

In 2016, the integrated addendum to ICH-GCP was published, building on the guidance jointly published by the Medicines and Healthcare products Regulatory Agency (MHRA), the Department of Health, and the Medical Research Council in 2011 [6], encouraging sponsors to develop more systematic and risk-based approaches to monitoring [7]. This uses the initial risk assessment completed early on in a trial to assess the level of monitoring required. For example, if a trial is assessed as high-risk, more frequent site visits may be required, whereas low-risk trials may not require any on-site monitoring [3]. Risk-based approaches may include focusing SDV on specific critical data points [8]. ICH-GCP recommendations state that “statistically controlled sampling may be an acceptable method for selecting the data to be verified”, suggesting that 100% SDV is not always necessary [1]. Other risk-based approaches may include reducing activities required on-site, such as obtaining participant consent to allow documents (e.g., consent forms) to be sent to the coordinating centre for central review [8], and resolving data queries remotely through interrogation of electronic systems. In 2012, 53% of CTUs surveyed used a risk assessment to determine the level of monitoring required [5].

Although guidelines now recommend a risk-based monitoring approach, it is not known how widely this process has been adopted across UK CTUs. A recent study in Ireland, for example, identified that only 21% of respondents had performed risk-based monitoring [9]. Furthermore, the very nature of a risk-adaptive approach suggests that this should be revisited throughout a trial, assessing whether the chosen monitoring approach remains adequate following trial protocol changes and whether new risks have been identified. There is currently limited evidence regarding reflections on chosen monitoring approaches at the end of a trial. This would provide insight into lessons learnt and inform decisions about future trials.

We therefore developed a survey, administered to representatives from UKCRC registered CTUs, to determine current monitoring practices and how these approaches are being reflected upon at the end of a trial. Responses were considered in relation to existing literature and best practice guidelines on risk-based monitoring to allow us to make recommendations for undertaking, and reflecting upon, risk-based monitoring in pragmatic trials.

The survey was developed by using Google Forms and structured, where possible, using skip logic so that respondents could avoid having to scroll through irrelevant questions. The content was developed taking into account existing literature on the topic, particularly focusing on reflection and learning from approaches used, and through discussion with colleagues in Sheffield CTU.

Almost all of the questions used tick boxes to minimise the burden on respondents. The researchers undertook extensive testing of the survey prior to its launch, particularly in relation to the use of the skip logic.

The survey had five sections: general information about the trial portfolios and approaches to monitoring, the initial risk assessment of trials, monitoring approaches and adapting to risks identified in trials, reflecting on the monitoring approach taken and the use of this information, and a “catch-all” for any further information that participants wished to provide. The full survey is reproduced in Additional file 1.

Questions were asked separately for Clinical Trial of an Investigational Medicinal Product (CTIMP) and non-CTIMP studies. A CTIMP is a trial looking at the safety or efficacy of a medicine or placebo, as defined by the Medicines for Human Use (Clinical Trials) Regulations (2004). Non-CTIMP studies do not involve investigational medicinal products and do not fall within the scope of the Medicines for Human Use (Clinical Trials) Regulations (2004).

A link to the survey was contained within the email sent out, which also had a participant information sheet attached. As the survey was designed to be anonymous, consent was considered to be implied by the completion of the survey. Also for this reason, the survey could not be designed to restrict multiple responses from the same unit or person. Therefore, the documentation provided specified that there should be only one response per unit.

Follow-up reminder emails were sent out by the Network at 2 and 4 weeks after the original invitation. These contained the same information, along with a copy of the survey. The design of the survey meant that questions were presented over a number of separate pages and so suggestions were received that it would be helpful for potential respondents to see the whole survey.

Descriptive statistics, through the use of count data, were used to present data on quantitative information gathered from completed responses. Narrative summary was used to present information from the answers to qualitative questions and supplementary information provided by respondents.

Nearly half of the registered UK CTUs responded to this survey, many with significant trial portfolios, thereby providing a good overview of the current approach to risk-based monitoring in the UK. Much of the information received was not surprising; more non-CTIMPs were undertaken than CTIMPs, and the approach to monitoring and risk assessments in CTIMPs indicated an understandably greater level of scrutiny and caution than in non-CTIMPs.

It is interesting to note that when compared with the results of the work of Tudur Smith in 2012 [5], a significantly larger percentage of trials units currently have a monitoring plan for all trials (61%, n = 14/23; compared with 35%). Furthermore, whilst the percentage of trials across units using risk assessment to guide the monitoring approach has remained largely unchanged (53% to 51%, n = 21/41 responses; Table 6), almost all use a risk-adaptive approach to monitoring (93%, n = 41/44 responses; Table 7).

The use of a risk-adaptive approach is clearly important in terms of ensuring that monitoring is cost-effective and appropriate to the needs of the trial. It is encouraging to see that almost all units are revisiting their risk assessments during their trials, albeit most often in a reactive manner to a specific issue or event, and that almost all are also using the information gathered to adapt initially agreed monitoring approaches.

However, it is unclear from this survey how significant such changes are in terms of the approach taken to monitoring. Whilst the survey does indicate that monitoring is almost entirely based on the assessed risk level of the trial, there is understandably going to be a significant use of staff judgement in making such an assessment. Such judgement, without clear mechanisms for sharing information amongst colleagues about how decisions are made, creates the potential for inconsistency in monitoring approaches within units.

In terms of reflections on monitoring approaches more widely, this was much more mixed across the units, and almost half did not reflect at the end of a trial at all. Even where reflection did take place, this was often not documented. This raises questions as to how consistency is maintained as well as how and whether learning takes place. In cases where units did reflect on their approach to monitoring, the responses provided indicated that this provides valuable learning, particularly in terms of making improvements for future trials (in terms of trial costing and eligibility and consent, for example).

This study provides a useful update on the use of risk-based monitoring, indicating the increased use of a risk-adaptive approach in the UK. Responses demonstrated that most units have established approaches in place around monitoring, including the use of risk assessments.

Despite this, our survey indicates that there is little reflection on the monitoring approach taken at the end of the trial, documented or otherwise. Those units that did undertake such reflection were able to demonstrate through their responses the value that this provides in developing staff knowledge and improving processes for the benefit of future trials.