Background
Methods
Sample and setting
Data collection
Observational data
Topic | Field notes |
---|---|
Data collection | Date, location, observer |
Pre-meeting | Setting (including diagram of room layout) Arrivals (e.g. who arrives when, what order?) Pre-meeting talk (e.g. verbal and non-verbal communication) |
Main meeting | Start time, who starts/how? Organisation of meeting (e.g. role of Chair, agenda, hand-outs, atmosphere) Chair (e.g. manner of facilitating discussion, leadership style, management of tension/conflict, influence on decision-making) Content of discussion (e.g. issues raised, knowledge/information drawn upon, recommendations made) Group interactions and decision-making (e.g. concerns raised, individuals’ contributions and roles in discussion, verbal and non-verbal communication, decision-making) Mode of participation (e.g. via teleconference) Action points/tasks (e.g. what is to be done, by whom, by when?) Time interval to next meeting End time, length of meeting, who ends/how? |
Post-meeting | Interactions and behaviour (e.g. who leaves first, alone/together?) General impressions |
Interview data
Participant groups | Topics discussed in interviews |
---|---|
Chief Investigator, Trial manager | The trial: history of the trial, details of the trial, current stage, successes, current and anticipated challenges The Trial Steering Committee/Trial Management Group (TSC/TMG): expectations of TMG/TSC, composition of TMG/TSC, selection of members and chairs, nature of the group’s decision-making and members’ involvement, examples of actioned group recommendations, impact of TMG/TSC, communication between TSC and TMG, relationship and communication between trial oversight committees and with funder |
TSC members | The trial: history of participation in the TSC, views of TSC, relationships with other members, value of TSC meetings, TSC’s role in decision-making, relationship and communication between TSC, TMG and other trial oversight committees TSC meetings: meeting organisation, Chair and leadership, communication during meeting, decision-making, agreeing and assigning actions, communication of actions to other groups/trial personnel |
TMG members | The trial: history of participation in the TMG, views of TMG, relationships with other members, views of TMG and TSC, value of TMG/TSC, TMG/TSC role in decision-making, relationship and communication between TSC, TMG and other trial oversight committees TMG meetings: meeting organisation, Chair and leadership, communication during meeting, decision-making, agreeing and assigning actions, communication of actions to other groups/trial personnel, role of Patient and Public Involvement (PPI) |
Trial funders | Funders’ expectations and views of TMGs/TSCs, process of selecting TSC, examples of trial oversight working well, examples where trial oversight has not worked well, different models of TSCs, role of TSC Chair, role of PPI, role of the trial funder, regulatory bodies, recommendations |
Sponsors | Sponsors’ expectations and views of TMGs/TSCs, role of sponsor in trial, responsibilities of sponsor, relationship between trial oversight committees, sponsor and funder, challenges faced by trials, role and value of TMGs/TSCs |
Data analysis
Ethical approval
Results
Data collected
Participant ID | Role | Relationship to trial | Gender | Trial number | Trial subject area |
---|---|---|---|---|---|
01 | TSC Chair (clinician) | Independent | M | 1, 2 | Oncology |
02 | Senior trial project lead | TMG | M | 1, 5 | Oncology |
03 | TSC coordinator P1 | TMG | M | 1, 2 | Oncology |
04 | TSC coordinator P2 | TSC administrator | F | 1, 2 | Oncology |
05 | Sponsor representative | TMG | F | 1, 2 | Oncology |
06 | Sponsor representative | Observer at TSC meeting | M | 1, 2 | Oncology |
07 | Trial manager | TMG | F | 1 | Oncology |
08 | CI | TMG | F | 2 | Oncology |
09 | Trial manager | TMG | F | 2 | Oncology |
10 | Trial manager | TMG | M | 3 | Arthritis |
11 | Senior statistician | TMG | M | 3 | Arthritis |
12 | Senior trial manager | TMG | F | 3 | Arthritis |
13 | Statistician | TMG | M | 3 | Arthritis |
14A | Co-CIa
| TMG | M | 3 | Arthritis |
15 | TSC Chair (clinician) | Independent | M | 3 | Arthritis |
16 | Trial manager | TMG | F | 4 | Frailty |
17 | TSC Chair (methodologist) | Independent | M | 4 | Frailty |
18 | CI | TMG | M | 4 | Frailty |
19 | TMG Chair | TMG | F | 4 | Frailty |
20 | TMG member | TMG | F | 4 | Frailty |
21 | Trial manager | TMG | F | 5 | Oncology |
22 | Statistician | TMG | F | 5 | Oncology |
23 | CI | TMG | M | 5 | Oncology |
24 | TSC member | Independent | M | 5 | Oncology |
25 | TSC member | Independent | M | 5 | Oncology |
26 | Trial manager | TMG | F | 6 | Urology |
27 | Trial manager | TMG | F | 6 | Urology |
28 | Statistician | TMG | M | 6 | Urology |
29 | CI | TMG | M | 6 | Urology |
30 | TSC Chair (clinician) | Independent | M | 6 | Urology |
31 | TSC member | Independent | M | 6 | Urology |
32 | TMG member | TMG | M | 6 | Urology |
33 | Trial manager | TMG | F | 7 | Psychology |
34 | CI | TMG | F | 7 | Psychology |
35 | PPI Representative | TMG | M | 7 | Psychology |
36 | TSC Chair (clinician) | Independent | F | 7 | Psychology |
37 | TSC member (statistician) | Independent | M | 7 | Psychology |
38 | TSC member | Independent | M | 7 | Psychology |
39 | CTU Director | TMG | F | 7 | Psychology |
40 | Senior trial manager | TMG | F | 7 | Psychology |
41 | Trial manager | TMG | F | 8 | Oncology |
42 | TSC Chair (clinician) | Independent | M | 8 | Oncology |
43A | PPI representative | Independent | F | 8 | Oncology |
43B | PPI representative | Independent | M | 8 | Oncology |
44 | Senior statistician | TMG | M | 8 | Oncology |
45 | Sponsor representative | Independent | M | 8 | Oncology |
46 | CI of another trial/member of TSCs | n/a | M | n/a | n/a |
47 | Funder representative | n/a | M | n/a | n/a |
48 | Sponsor representative | n/a | M | n/a | n/a |
49 | Funder representative | n/a | F | n/a | n/a |
50 | Senior statistician | n/a | F | n/a | n/a |
51 | Funder representative | n/a | F | n/a | n/a |
Findings
Collaboration within the TMG and role of the CTU
‘My actual role, as I see it, is to represent the patients as much as I possibly can, to make sure their interests are dealt with correctly, that they are not just used as pawns in the research’. [#35, PPI representative, RCT 7]
‘Things work well in trials in general when you’ve got the clinician side who are on top of all of the kind of treatments and that kind of thing, and then you’ve got us lot here who make it all work. The clinicians don’t always understand how much time it takes to put in amendments and how much trouble it can cause at sites… But without them, obviously there’d be no trial’. [#41, Trial manager (TM), RCT 8]‘We look to [CIs] to make sure that we are understanding a lot of the background and the issues in clinics. We have to make sure that [they] understand the issues relating to the design of the operational aspects of the study.’ [#02, Senior trial project lead, RCTs 1 and 5]
‘So what we wanted the group to approve was just formal collaboration with [another researcher], which seems to be pretty straightforward. Yes, they just want to take some extra [tissue] samples from patients in the trial, they don’t want to do anything to them, apart from that. So I just formally wanted to request the TMG approve this’. [#23, CI, RCT 5]
‘Okay, so we need to think about what the implications are. What data they are looking for, whether this is data that we would be able to get out of the centres. Were the centres going to struggle to collect that information? Whether they want this data released, what the impact is for us? I guess this another protocol amendment?’ [#2, Senior project trial lead, RCTs 1 and 5]‘Yes.’ [#23, CI, RCT 5]‘So it’s not a small amount of work required, by our end I suspect.’ [#2, Senior project trial lead, RCTs 1 and 5]
‘They don’t need to understand it [all of the operational details of running a trial], so long as they are the kind of person who can admit, “Okay, I don’t understand this and so you’re going to advise me”… But I have worked with some CIs who are a little bit more difficult and who maybe don’t understand how trials work and try and argue sometimes’. [#41, TM, RCT 8]
‘I probably have the most input in the day-to-day management of the trial, so I update on trial progress. I would also usually be responsible for implementing the decisions that are made’. [#33, TM, RCT 7]‘It is very much a project management role when it gets to this stage: keeping an eye on deadlines and making sure that we achieve our targets.’ [#27, TM, RCT 6]
‘You’ve got weekly meetings with a sort of smaller group of people, that’s the agility you need to make decisions, even once a month to be honest is probably not quite agile enough for a TMG’. [#31, Independent TSC member, RCT 6]
‘When you are working with a good team, it is not that stressful, there is a lot of satisfaction from it’. [#29, CI, RCT 6]
Collaboration and conflict between oversight committees
‘It has been slow to recruit, but it is continuing to recruit. The TSC and the TMG at the [funder] have been very innovative, sensible and imaginative in trying to stretch our funding. [Interviewee laughs]… What was clear from the interim release was that we did not appear to be doing harm. That boosted recruitment and it encouraged clinicians to know that it was worth having the conversation: “I don’t seem to be doing harm with this trial, and it remains a very important question to answer”. That allowed us to maintain our funding and to stretch it’. [#08, CI RCT 2]
‘There wasn’t very much on the agenda really, it’s rather formulaic in that we always, you know, the recruitments and the questionnaires and how many have been filled in and then well what the DMC have said and how the thing is just going generally, because it’s going quite well, there isn’t a lot to say really. So it’s… we have to hold the meeting because you know that’s in the protocol really for a couple of times a year. So I don’t think we did anything or said anything earth shattering… I never know what people think about these things but they all come along, probably because they have to’. [#30, TSC Chair, RCT 6]
‘Our position then was, as a TSC… we have taken decisions here, and we are absolutely obligated to explain those decisions. The meeting started in a quite hostile fashion, because the PIs and the CIs were just basically saying, “What the hell has happened here? What have you done?” We had to get it across to them, and it was a very good meeting. Because by the end of it, everybody was on the same page. The previously hostile PIs were actually quite grateful that the situation had been explained to them. They now had a mechanism to engage their patients’. [#38, Independent TSC member, RCT 7]
‘What there was complete agreement about, was concern for participants and the welfare of participants in the trial… everybody had agreed on that, and it was just disagreement about how to deal with that’. [#39, CTU Director, RCT 7]
Priorities
‘I think he treats [the trial] as his baby, because he is the Chief Investigator… You kind of think he’s really personal about it and I think, you know, if the trial did get closed he’d be like pretty upset about it… he’s really highly motivated and driven… anything possible to keep it going’. [#13, Statistician, RCT 3]
‘Obviously [the CI] is very close to trial and we all want to succeed, but we’ve also got to be realistic about what is happening and… I mean, as I say, I try and look at things from an external point of view as well, try and take a step back’. [#11, Senior statistician, RCT 3]
‘The CI very, very rarely wants to have his trial closed, because it’s the sign of failure. The trials unit… can see a dead horse that they have sitting on their portfolio’. [#51, Funder representative]
‘It should be impossible for an investigator to run a trial that isn’t done through… an accredited trials unit… There should be no investigators as they used to be when I started doing trials on the back of an envelope basically in a little office somewhere, dolling out the drugs probably – it’s not on anymore… What we are talking about now is the whole professionalism of running trials… nobody should be doing this out-with a professional trials unit, and there are many of them now… And if anybody has a halfway decent idea for a trial they need to be developing it with a trials unit’. [#51, Funder representative]
‘I think in a philosophical way, [the role of the TSC] is to protect the patient, and then the integrity of the trial… it puts the interest of the patient first and foremost. As an independent, you are making sure that that is observed’. [#37, Independent TSC member, RCT 7]
‘The safety of the patient is the exclusive reserve of the DMC’. [#37, Independent TSC member, RCT 7]
‘The DMC can see the data and say there’s a problem, but they won’t know why there’s a problem… their job is to look at the data, not completely in isolation, but looking for signals, identifying the signals, passing that on to a group [the TSC] that then can look at things in more depth’. [#40, Senior TM, RCT 7]‘The TSC does not have to take the DMC’s advice, because it has to see a greater picture. It almost certainly will [take DMC advice]. But every so often, the person that the subcommittee[sic] reports to should be able to turn around and say, “Well, no, based on other things that we have to take account of, your recommendation does not overwhelm”.’ [#38, Independent TSC member, RCT 7]
#14B, Co-Chief Investigator (Co-CI), RCT 3: ‘So it might be also helpful [trial statistician], to look at the scenarios of seeing if there was a greater difference. I’m not suggesting that’s our primary strategy, I think the best strategy is to go for power. But I think it would be nice to have the scenarios of looking for a 70% difference and a 90% difference. So (trial statistician), would you be able to just do that?’ | |
#13, Statistician, RCT 3: ‘Yeah, we can do that, but when I was speaking to [senior trial statistician], yesterday he said it’s not really good practice to meddle with the difference coz you know…’ | |
#14B, Co-CI, RCT 3: ‘Well it’s very easy to say that but the other option is the trial stops, do you see what I mean? So good practice is to make sure that we fulfil the commitments of the families who have consented to this study. So we appreciate what good practice is, but I think it’s also looking for a number of scenarios in which we can present to everyone including the Data Monitoring Committee, the (Trial) Steering Committee and the funders to find a pragmatic solution to a problem of recruitment, okay. So it’s just to have those figures available really for discussion’. | |
#12, Senior trial manager (TM), RCT 3: ‘[CI], I think it’s just being aware that obviously if you’re changing the difference that you’re looking for, then you are leaving yourself open to criticism, so you need to be ready to…’ | |
#14B, Co-CI, RCT 3: ‘We are –’ | |
#14A, Co-CI, RCT 3: ‘We are aware. [Senior trial manager], we’re open to ever greater criticism, so this is not been talked about lightly or picked from thin air. There’s a real and present danger that if we go up in front of (funder) in (month) not having met our target of [x] patients in the last 6 months, there is a very real present clear danger that they will stop the trial. That is not in the best interest of the patients who have participated, that’s not in the best interest of any of us’. |
Communication between trial oversight groups
‘…make the message clear …to make sure we’re singing from the same hymn sheet…’. [#02, Senior trial project lead, RCT 1 and 5]‘Communication is definitely key. It helps that all the communication goes through me. I think the TSC and (DMC) …look for emails from my email account, and they know to respond to that.’ [#10, TM, RCT 3]
‘The most important thing is to communicate everything and expediently, and keep communications going between all groups… Transparency is important… and the relationships between the TMG, the CTU and the TSC were really vital, that’s why it worked so well’. [#40, Senior TM, RCT 7]
‘The communication issue there came up, which is, to be honest I don’t see it as my job to communicate with DMC. The route, we clarified – the route was a bit unclear way back. But during the recruitment stoppage, we were careful to clarify that the route of communication was between us and the TSC to the DMC… They advise the TSC, not us… The trouble originally was DMC badgering us’. [#34, CI, RCT 7]
‘The role of the DMC is to be independent, and they jeopardise their independence if they’re too close to the trial team – it’s not that they shouldn’t make a request for information from a TM, but it should go through some other route. The DMC reports directly to the TSC. That’s the charter that everybody uses… The first thing I did was to stop communication between the TM and the CI, and the DMC’. [#39, CTU Director, RCT 7]
‘There needs to be really, really accurate and complete communication between research team led by chief investigator and the TSC, between TSC and DMC, and these are both bi-directional not uni-directional. Then, as the need arises, between TSC and other stakeholders, which, importantly would include the funder’. [#37, Independent TSC member, RCT 7]
Power and accountability
‘They [the TSC] didn’t come to this with any independent scrutiny. So when a lot of cross over and, in effect, contamination between the arms was going on, the TSC were quite happy to accept that… We actually discharged the Chair of the [Trial] Steering Committee and appointed our own … experienced TSC Chair who we briefed to make it clear that what we wanted from them’. [#47, Funder representative]
‘If you’re the funder you’ve got power and responsibility, you have still got to make sure this stuff delivers otherwise the governments are going to get upset, so you know you have got responsibility there. The TSC, yeah it has responsibilities but they are very much within … perhaps a narrower focus of responsibility, …they have a responsibility to do what they are supposed to do according to their charter. It isn’t really the TSCs responsibility if the whole thing goes wrong’. [#31, Independent TSC member, RCT 6]
‘The [funder], they make it very, very clear: it says the primary TSC reporting line is via the Chair to the [funder] programme director. Forget about the sponsor, so this is the funder who is taking control of the TSC and whereas if you look at the research governance stuff I’m sure that it was actually the sponsor who the TSC should be reporting to, not the funder…They’re definitely trying to have more control of the TSC so their loyalty is more to the funder rather than to the sponsor or to the CI… I think the TSC has a very important role… it’s worrying that [funders] are trying to dilute their responsibilities or yeah just distorting the relationship by… making them much more a tool of the funder rather than a source of independent scientific advice …the TSC to my mind has become almost a puppet’. [#50, Senior statistician]
‘At the end of the day, the guy with the money is the guy with the power, and if the funder says do it, you’re going to do it … because they’ve got the ball. They’ll take it away if they don’t like it’. [#31, Independent TSC member, RCT 6]‘The TSC is, after all, it’s an advisory group isn’t it? It's not an executive group, they don't decide on what you can and can't do and they don't really have any control over the funding. So they're there to advise the (funder), who then make the decision.’ [#29, CI, RCT 6]
‘The question was should they have the power to stop it? I think they’ve got the power – no, the only power they’ve got is to withdraw the funding I think, because the actual protocol, as far as I’m aware, is the property of the trialists. I would have thought the trialists are able to continue the trial even if the [funder] withdraw the funding … from a pragmatic point of view, then most trials would probably stop’. [#32, TMG member, RCT 6]‘No. Because they’re thinking in terms of money as opposed to whether it’s a good idea being done properly. What matters is the question whether it’s being done properly and whether you can get an answer, and the funding is just a different matter altogether.’ [#01, TSC Chair, RCTs 1 and 2]
‘I suppose officially it would be me really… I can’t imagine the circumstance where I would be telling [names] and the statistician and Director of [CTU] you know, this is rubbish it’s got to stop… but of course it would be my job to do that, it’s in the sort of charter’. [#30, TSC Chair, RCT 6]‘The buck stops with us. They [the TSC] do slightly more than advise I guess. We would certainly look to them, I mean this is slightly confused, not confused, but there is a tripartite role really for oversight of trials because there’s us as funder, there’s the TSC employing independent advice but accountable to us as funder and the other obviously is the sponsor… the local university or hospital and they obviously have a role as well and to some extent the TSC needs to be accountable a little bit to the sponsor as well.’ [#47, Funder representative]‘The TSC guides the sponsor, and ultimately the decision to move forward or not with the study sits with the sponsor.’ [#45, Sponsor representative, RCT 8]‘[Funders] like to pretend they are but they’re not, are they. Quite clearly not, so the buck doesn’t stop with them, their role, the buck stops with the sponsor.’ [#48, Sponsor representative]
‘Well, you could see it as checks and balances, and they would stop it for different reasons … I see it as just checks and balances, but then, if you’ve got a situation of conflict, what you need is a process for resolving it … It was a huge relief to me when I went and looked at the charter, because I hadn’t been involved in setting it up, so I didn’t read the charter until this happened. I was so relieved to find that there was something in there that offered a way forward … It gave me a mandate to interfere. [Laughter]’. [#39, CTU Director, RCT 7]
‘[The funders] didn’t actually call me a liar, but they said, “We don’t believe you”… which at the time made me very angry … well, it didn’t seem to me that that was a reasonable way of behaving … it wasn’t a reasoned response to the scenario that we were putting forward to them… They’d just made their minds up, they’d looked at the recruitment figures and said “Right, we’re closing this trial”… Although they initially gave the impression that it was going to be a constructive helpful discussion as to how the trial could be salvaged, they had meanwhile made the decision that they were going to close the trial before we’d actually had the opportunity to put forward our proposals for how things could be saved’. [#50, Senior statistician]
‘I mean, we don’t just see the ones that are poorly performing, although we often do call them in and talk through if there are problems and what’s going on. But it’s then an opportunity for us to sometimes provide good advice to them about how they might turn things round… and actually most people come away from those feeling they’ve had a good meeting. You know, clearly not everyone, but it’s not the sort of thing where people come up and get a bit of a kicking as it were… I mean, you do have the odd difficult one where… you basically do have to give people bad news that they’re clearly going nowhere and it’s time to close the trial down’. [#47, Funder representative]
‘Because ultimately the TSC is accountable to the funder, and there is the big worry that they are the people that could stop your trial … that they would withdraw your funding. So I suppose people want to keep them sweet… Make them think you are doing a good job. Also because it is human nature, everybody does it’. [#33, TM, RCT 7]‘We want to look good; we want to look confident. I think people in academia, people doing research, are people who probably tend to be on the perfectionist end of things. They want things to be nice; they want things to go well. Yes, I think people want to feel like they’re competent and in charge and on top of their game, and can do it.’ [#20, TMG member, RCT 4]Interviewer: ‘Mm-hmm, and so we – we don’t lie, do we?’ ‘No, I don’t think we lie. I don’t think I have ever sat here and lied, and I don’t think any of us have ever sat here and lied, but I think we put spin on things… there’s what actually has happened, and then there will be the spin we put on that to make it look like something different happened. I think we do that all the time.’ [#20, TMG member, RCT 4]
‘I’ve got a couple of CIs who like to go off in tangential discussions about the study and how it might be better, and you know opportunities missed. I say to them, “Could you just not do that?” It’s not the purpose, we are here to basically reassure the independent [TSC members]. Now, on the one hand I can see that, intellectually that might be not quite right, that we are there to have an open discussion… So I think the best kind of TSC is a good, compromise between those two. That there is a genuine sense of open debate and inquiry, but it’s based on rock solid data, which has been produced to a professional standard’. [#37, Independent TSC member, RCT 7]
‘I guess the CI could try to do that, but I think if the TSC has enough independent members and is doing its job correctly, they would be willing to question and wouldn’t be led inappropriately’. [#49, Funder representative]‘I would like to think that the old goats that serve on these things are long enough in the tooth to see through that rubbish.’ [#28, Statistician, RCT 6]
‘You’ve got to take your dirty laundry to the TSC and get it cleaned. That’s what they’re there for. A lot of CIs don’t understand that the TSC is there to protect them and to help them resolve difficult issues. It’s fine when it’s all going hunky-dory, but a lot of CIs don’t understand, I think, that the trial oversight is their protection’. [#39, CTU Director, RCT 7]‘What you need though is a very open culture to do that, because it’s the responsibility of the trial team to bring to the TSC those issues as they arise so you’ve got to, you’ve got to have a system and a culture which does that rather than hides things away.’ [#48, Sponsor representative]
Discussion
1. Led by their Chairs, trial oversight committees must foster a culture of openness and mutual respect, recognising and drawing upon the skills of all trial oversight committee members. To ensure optimal decision-making and problem-solving, committee conduct should be respectful of all voices and Chairs should actively seek opinions from all members | |
2. Recognise the partnership role of CTUs and TMs in managing a trial and supporting the CI in decision-making, management and achieving deliverables | |
3. The differing priorities of trial stakeholders in overseeing and delivering the trial should be explicitly stated, considered and, where necessary, realigned to the shared priority across stakeholder groups to produce a good-quality trial that informs practice. Guidance and best practice on resolving differences should be shared and could be collated and hosted by CTUs and funders | |
4. Clear lines of communication between oversight groups should be established in advance of the trial starting, documented in the trial Charter, shared between stakeholders, and maintained. From the trial outset, the frequency of oversight meetings should be considered and agreed. The frequency should be regular enough to be responsive to challenges and implement trial oversight decisions promptly, while allowing for extraordinary meetings in the event of challenges | |
5. Ensure a primary, single point of contact for the trial and coordinate communication with trial oversight stakeholders | |
6. Consider and agree before trial initiation who will act as arbiter if/when needed; the CTU may or may not play this role | |
7. Consult with stakeholders to determine the full implications of funders appointing independent TSC members to trials, and agree an approach to this issue | |
8. Clarify the roles and responsibilities of all those involved in trial oversight, and make stakeholders aware of these. This includes the different reasons for shutting down a trial, and which oversight bodies might do so in which circumstances. Each trial should consider these from the outset. Discuss before trial initiation the information needs of different stakeholders and communicate efficiently and in a timely manner as needed | |
9. Be aware of how the threat of monitoring meetings, or of closing trials, can negatively impact trial conduct and relationships, especially the way a CI or TMG might present the trial to the TSC. It might be of benefit to identify risks to the trial from the outset, and report on these at each meeting | |
10. Acknowledge how the close relationship between TSC and funder and the threat of the latter withdrawing funding is in tension with the role of the TSC in providing expert support to the TMG. Power hierarchies between committees can restrict the effectiveness of trial oversight, so efforts should be made to decentralise power |