Background
Methods
Setting
Risk classification procedure | Recommended systematic review of trial’s risk profile | |||
---|---|---|---|---|
1. Initial Risk classification | 2. Categories of risk modulators | 3. Risk classification | 4. Modulators of monitoring extent | |
Potential risk of therapeutic intervention in comparison to standard of medical care (as described in HRA & ADAMON): • Comparable (see also ClinO art. 19,20,61, category A) • Higher (see also ClinO art. 19,61, category B) • Markedly Higher (see also ClinO art. 19,20, category C) | Modulators from the following ADAMON categories may influence the initial risk category by a max. of +1 or −1 risk category (e.g. Intermediate to High Risk): 1. Potential trial participant-related critical indicatorsa
2. Robustness related indicators – “hard primary endpoints” and/or simple clinical trial procedure 3. Site-related indicatorsb
| An overall risk category is assigned based on the results of 1) and 2) as follows: • Low risk: Risk of intervention comparable & trial has at least one indicator of robustness and no participant related indicator • Intermediate risk: Risk of intervention comparable, or higher & trial has no indicator of robustness, or at least one participant-related indicator present • High risk: Risk of intervention higher or markedly higher, and trial has no indicator of robustness, or at least one participant related critical indicator present | The following modulators may influence monitoring extent (i.e. number of hours per visit) within risk category: 1. site experience with clinical trials 2. presence of an electronic database at site 3. whether site is coordinating lead site of the study | An additional risk assessment is required if the trial undergoes substantial amendments |
Risk of Study | Initiation visit | Interim visit | Content of interim visits | Close out visit |
---|---|---|---|---|
Low | optional | after first patients, then adaptable (e.g. 1/year) | Endpoints (extent to be defined), IC (usually 100%) | optional |
Intermediate | mandatory | after first patients, then adaptable (e.g. 1/year) | Endpoints (extent to be defined), IC (usually 100%), safety (usually 100%) | mandatory |
High | mandatory | after first patients, then in regular intervals | Endpoints (extent to be defined), IC (usually 100%), safety (usually 100%) | mandatory |
Quantitative retrospective analysis
-
study design,
-
study type,
-
study sponsor,
-
type of research,
-
study phase (I-IV), and
-
type of study population (e.g. inclusion of vulnerable populations).
-
site location,
-
ADAMON risk category,
-
presence of electronic database,
-
principal investigator, and whether he/she changed during conduct,
-
staff experience, and
-
number of planned patients at the site.
-
type of visit (i.e. initiation, interim, close-out),
-
the number of
-
administrative,
-
patient rights,
-
patient safety,
-
laboratory/biological specimen,
-
data point confirmation, and
-
endpoint related findings.
Semi-structured interviews
Results
Study sample characteristics
Total | |||
---|---|---|---|
n | % | ||
Total studies | 43 | 100 | |
Study design | Interventional | 34 | 79.1 |
Observational | 9 | 20.9 | |
Study type | Multicenter | 10 | 23.3 |
Singlecenter | 33 | 76.7 | |
Study sponsor | Investigator (academic) | 40 | 93.0 |
Industry | 3 | 7.0 | |
Type of research | Drug | 29 | 67.4 |
Medical Device | 5 | 11.6 | |
Biological Samplesa
| 4 | 9.4 | |
Otherb
| 5 | 11.6 | |
Study phase (drug studies, n = 29) | I | 9 | 31.1 |
II | 7 | 24.1 | |
III | 8 | 27.6 | |
IV | 3 | 10.3 | |
Otherc
| 2 | 6.9 |
Total | |||
---|---|---|---|
n | % | ||
Total studies | 43 | 100 | |
ADAMON risk category | Low | 11 | 25.6 |
medium | 23 | 53.5 | |
High | 9 | 20.9 | |
Total | 43 | 100 | |
Electronic database present at first patient in | Yes | 19 | 44.2 |
No | 24 | 55.8 | |
Total | 43 | 100 | |
Principal Investigator change during study | Yes | 3 | 7.0 |
No | 40 | 93.0 | |
Total | 43 | 100 | |
Vulnerable study populationa
| Yes | 7 | 16.3 |
No | 36 | 83.7 | |
Total | 43 | 100 | |
Total sites | 94 | 100 | |
Staff experiencedb, by site | Yes | 88 | 93.6 |
No | 6 | 6.4 | |
Total | 94 | 100 | |
Staff change, by site | Yes | 11 | 11.7 |
No | 48 | 51.1 | |
Unknown | 35 | 37.2 | |
Total | 94 | 100 |
Characteristics of monitoring findings
Total | |||
---|---|---|---|
n | % | ||
Total Studies | 43 | 100 | |
Total Monitoring Visits | 214 | 100 | |
Findings | |||
Administrative | 1367 | 46.2 | |
Patient rights | 1453 | 49.1 | |
Patient safety | 32 | 1.1 | |
Laboratory/biol. specimen | 70 | 2.3 | |
Endpoint related data point: confirmation requested | 36 | 1.2 | |
Endpoint related data point: Data point changed | 3 | 0.1 | |
Total | 2961 | 100 | |
Average n findings/visit | 13.8 | ||
Studies with | |||
at least 1 administrative finding | 43 | 100 | |
at least 1 patient right finding | 41 | 95.3 | |
at least 1 patient safety finding | 9 | 20.9 |
Finding category | Examples of findings |
---|---|
Administrative | • Changes at the investigational site (staff training, staff CVs, address, technical equipment, etc.) not documented • Functions and responsibilities log not up to date • Subject related logs not up to date • CRFs not available at site and/or not documented by authorized staff |
Patient rights | • Informed Consent Forms not signed and/or not dated correctly • No valid and approved version of Informed Consent Form used • Amendments/addenda to Informed Consent Form not communicated to patients and no re-consent obtained • Patient did not fulfill all inclusion criteria |
Patient safety | • No description of the process for detecting and reporting serious and unexpected adverse events and/or unanticipated problems involving risk to participants in place at site • Adverse events not correctly documented and/or reported as required (e.g. to Sponsor, EC, Competent Authority) • New safety information not approved by authorities • Staff not trained according to new safety information |
Laboratory/Biological Specimen | • Biological specimen not stored correctly according to protocol • Process conducted not in accordance with Good Manufacturing Practice (GMP) |
Data point confirmation requested | • Indicates whether finding challenges the credibility of data point, e.g. by stating “Please confirm that blood pressure measure is 100/65 mmHg” |
Data point changed | • Indicates whether data point was adjusted as direct consequence of finding, e.g. “Blood pressure of 100/65 mmHg was corrected to 120/80 mmHg” |
Influencing factors on number and type of findings
Cost of monitoring
The Monitor’s perspective
Factors influencing risk-based monitoring findings
“The positive effect clearly is that you think more about the study itself. If you take the effort to classify the study by risk factors you can actually eliminate many things upfront. Because of that evaluation, I know what to set value on when I open a site”. (Monitor III)
“But it all depends on the experience of staff on-site, if they have lots of experience with studies, they know how to do it. But don’t forget that staff changes so often at the site, you never get the same people from the start until the end of a study. The new ones, how will they be trained? We as monitors only hear about it half a year later and if you only visit them once a year, you hear that they have changed the recruiting physician and that patients have been informed wrongly for three quarters of a year.“(Monitor III)
Monitoring process and challenges faced
“It depends on the monitoring plan; usually we look at 100% of the Informed Consent Forms, unless there are too many patients such as in cohort studies. We always look at the inclusion and exclusions criteria. Endpoints are to be discussed and defined with the principal investigator. Depending on the budget available, we might also look at the Trial Master File, and then I am done in no time” (Monitor I)“I hope that I cover safety aspects with my monitoring. Actually I don’t see any issues with it. There is no monitoring plan without the safety aspect, usually it is 100% covered. (…) Depending on what you find, you adapt it (the monitoring plan). If you find critical issues, for example a Serious Adverse Event that was not documented, you tell the team (on-site) to look at the other patients’ data and check whether they were correct.” (Monitor II)
“I question whether this approach is compatible with GCP. According to GCP you put patients first, and then the scientific question. There you also include data protection and the ICF (informed consent form). You don’t really respect patient rights if, for example, a wrong ICF version was signed and I only notice after half a year, just because the study is a low risk study according to the evaluation. I don’t think it affects the scientific validity, but more the patient safety and rights aspects”. (Monitor III)
“One is for sure, if you don’t see that mistake at the beginning, it’s going to repeat with the next patients. So that’s a systematic error then, and of course data quality suffers! “(…) “If I knew that every three month there is a monitor at your doorstep who wants to critically look at your data, then I am of course required to get my stuff done in time and…a bit more accurate as if, you know, I know that there is anyways no one looking at my data. Then you get the running around after the data at the end of the year”. (Monitor III)
“Often, we do not look at lab values because they are not seen as risky values, maybe only 10% is seen as crucial for the primary endpoint. But then you ask yourself whether you wouldn’t miss transcription errors if you don’t look at these values at all. (…) There are not only systematic errors that you don’t see but also those that appear everywhere and they’re even more difficult to detect. And it also depends on the format in which you collect data, if it is on paper or not. The CRF (case report form) heavily influences the number of mistakes that are made, and you don’t look at all of these with the risk-based approach. (…) If you, for example, look at one patient 100% in a study and not the others, you of course detect systematic errors and point to that and make them look at the other patients in their documentation as well.” (Monitor II)
“I assume that in an inspection, you know, if they were to look at 100% of the data after my monitoring, they would find errors even in my monitored data. But I am sure you see exactly what documents were monitored and which weren’t. (…) but I believe that data quality would for sure be better with more monitoring, there I am 100% sure.” (Monitor I)“Also the change in personnel has an influence, if you don’t take care of the training of new employees, and you don’t involve them. The situation on-site is not reflected adequately in ADAMON, it’s more sort of a weak factor, that if you’re there anyways, you monitor longer, but it doesn’t influence the frequency of monitoring. Changes and structures on-site should have a stronger influence on the frequency of visits, in my opinion. “(Monitor III)
“It is not clear whether it is cheaper if the monitor visits less frequently and everything on-site goes downhill or if it wasn’t better if the monitor had visited once or twice more, you know. To make crappy data better again is also not cheap, right? Even if you adapt all ADAMON criteria, we should get away from only seeing cost savings in it. “(Monitor III)
“…just because the budget doesn’t allow, you know, I would like to monitor more frequently or follow the risk classification more strictly, but you can’t, because of the financial limitations.” (Monitor I)
Role of monitors and future perspectives
“I would wish for more acceptance of monitoring, they all think it is just a necessary evil. That you don’t do it because it’s important and helps data quality, but only because the regulators and authorities want it from you. I am sure, often, monitoring reports are not even read. There should be more trust that we help data quality and therefore, also help the answer to the study question”. (Monitor II)“I always tried to make them feel like I am their partner, and not some teacher or something…I don’t want to point the finger at them, but build a trustworthy relationship so that people at the site know they can trust me. So that they know they can call me if there is a problem. (…) The aim is to make them understand that we are partners and try to help to get to a good result that all of us do a good job, and that patients are safe and their rights are protected”. (Monitor III)