Acceptance checklist for clinical effectiveness pilot trials: a systematic approach

The table of proposed trial components, monitoring methods and monitoring outcomes formed the draft ACCEPT document.

Eligible participants were people with primary progressive dementia and their family carers. Early feedback from recruiters identified uncertainty in assessing eligibility when presentations were a-typical, where there were comorbidities such as stroke, head injury or movement disorders, and where potential participants without a formal diagnosis of dementia were nevertheless in receipt of dementia-related treatments and services. During the pilot we reviewed and clarified the screening tool, and encouraged discussion with clinical researchers where there was uncertainty. Eligibility of clients with a-typical presentations was discussed on a case-by case basis throughout the trial, but the eligibility of most potential participants could be determined on the basis of the original screening questions. We therefore concluded that changes to the assessment of eligibility criteria could be considered minor and were not a barrier to the carry forward of data from the pilot to full trial.

The two psychosocial interventions had different delivery formats: CSP focused on individual carers while RYCT focused on groups of people with dementia and their carers. We therefore needed a trial design that allowed simultaneous delivery of individual interventions and a group programme such that a random sample of participants received the combined intervention. We devised a 2-stage randomisation process in which each participant was randomised individually immediately post-baseline assessment to either CSP intervention or control, and subsequently entered into a group randomisation process to either RYCT or control. Experience in the first pilot site was that the time-lag between the first and last individual randomisation was such that the proposed timetable for follow-up interviews was put in jeopardy. In the second pilot the individual intervention was yoked to the group timetable and procedures tightened such no potential participants undertook initial assessments until a minimum cohort size had been identified as willing and eligible. In the main trial there continued to be variation in the time difference between baseline assessment and the start of intervention, and between the end of intervention and final follow-up interview. The monitoring data will allow comparison between arms of the trial and a decision on whether to include pilot data in the final dataset will need to be made once final follow-ups are complete.

Also relating to trial design, feedback from researchers in the first pilot site alerted the research team to participants expressing greater interest in the individual support intervention than the group intervention. This had implications for intervention uptake and the feasibility of the group intervention sessions. Prompt discussion with the trial methodologists before the group randomisation in the first pilot site enabled us to adjust randomisation ratios for the group randomisation from 1:1 to 2:1 (RYCT:control). The revised randomisation ratios facilitated adequate RYCT group size in subsequent rounds of recruitment and intervention. If this modification had not been made prior to the first group randomisation, data from the first pilot site could not have been carried forward to the main trial dataset as the trial design of the pilot would have differed significantly from that used in the main trial.

In order to ensure intervention fidelity, experienced clinicians in the research team were closely involved in the set up and provision of interventions at the pilot stage. Recording systems on aspects of intervention fidelity were devised and tested during the pilot and then implemented fully during the full trial. Variation of intervention provision was greater during the full trial than it had been in the pilot sites due to differences in the funding and organisation of services in each geographical area. For example, in the trial sites the RYCT intervention was delivered with involvement from National Health Service (NHS) employees and the Carer Supporter intervention provided by organisations within the voluntary and charitable sector. This model of intervention provision was replicated for most rounds of recruitment and intervention for the full trial but in some additional sites both interventions were provided by the NHS and in one further site a voluntary/charitable sector organisation provided both. The same RYCT group trainer was used throughout the trial, and, although minor details of the content of training were amended through the course of the trial, the core of the training manual was consistent throughout the pilot and full trial. Intervention fidelity was higher in the pilot sites, but fell within the ‘envelope’ of variation seen across the full trial. A decision was therefore made that pilot data should not be held back from being pooled with the main trial dataset on grounds of variation in intervention fidelity.

An early audit of the completeness of baseline data in the first pilot site enabled us to identify systematically missing data. The main issue was the formatting of data collection tools, which allowed researchers or respondents to misinterpret question routing and omit questions. Hence we reformatted these tools and re-trained researchers. The corresponding audit in the second pilot site showed a reduction in missing data. Thus early identification of data collection issues, and immediate establishment of monitoring, enhanced the quality of the dataset throughout the trial, and kept the proportion of missing data to a minimum. A high proportion of missing data would have led to a decision to exclude pilot data from the main trial dataset.