Methods
The MRC-START optimised patient information materials were embedded within the Healthlines Depression and Healthlines CVD trials. The embedded recruitment intervention involved a change to trial procedures, to assess whether optimised patient information materials improved the proportion of patients responding positively to an invitation to take part in each trial and, within Healthlines Depression only, the proportion actually randomised.
Development of the recruitment intervention
The process involves optimisation of readability and navigation of the documents, using expertise and considerable experience in writing for patients, as well as drawing on evidence of what works [
13] and expertise in information graphic design. The revisions are then informed by repeated user testing [
14‐
17], where the ability of patients to locate and understand key pieces of information is evaluated objectively through multiple versions. The optimised sheet covers the same topics as the original, but differs in
form (i.e. appearance, structure and wording) and
effectiveness (i.e. the ability to transmit the required information). For the user testing, we recruited healthy members of the public, who had a similar socio-demographic profile (age, education and occupation) to the likely samples in the
Healthlines trials. We excluded people who had taken part in any healthcare trial or readability testing in the previous 6 months.
In the user testing, each participant saw only one version of the information [
14,
15]. We conducted three rounds of user testing, with ten different participants in each round. We aimed to ensure some variation in participant characteristics in each round as well as consistency between rounds. In each round of 10 participants there were: at least 3 of each gender; no more than 2 higher education graduates; at least 4 people aged 18–39 years and at least 1 each aged in their 40s, 50s, 60s and 70–74 years. We also excluded anyone who had been treated for depression (or heart disease, as applicable) in the previous 6 months. Participants were recruited from the participant database held by Luto Research Limited (Leeds, UK), who were commissioned to undertake the user testing; participants were members of the public who had come forward for readability testing studies. They were paid a small fee to compensate them for their time.
The first round tested the original Healthlines materials, after which the revised versions were developed. The second and third rounds tested the revised versions, with minor changes made to wording and layout in response to the findings of each round of testing. In user testing, each participant first read a version of the patient information sheet and covering letter and was then asked to respond to 20 factual questions: 3 related to the covering letter and 17 to the information sheet. The questions were drawn from four categories of information that would apply to any trial: the nature and purpose of the trial (three questions); the process and meaning of consent (four questions); trial procedures (ten questions); safety, efficacy and nature of the tested intervention (three questions). For each question, participants were asked to locate the answer in the letter or sheet (testing navigation and organisation of the information), then give the answer in their own words (testing understanding). The data derived from user testing is indicative, not definitive, and so no particular scoring threshold was used for revisions to be required. Problems locating an answer suggest a need for clearer navigation or structure; problems in understanding suggest a need for re-wording. The decision to revise the materials is also influenced by the commentary from the person who conducted the user testing – this can indicate the severity of a problem within a document.
The original Healthlines patient information materials for both trials were 8-page A5 patient information sheets in booklet form, together with an A4 covering letter from the general practitioner (GP).
The patient information sheet was designed in accordance with National Research Ethics Service (NRES) guidance and based on materials used in a related earlier survey. The patient information materials were developed and revised through discussion amongst the Healthlines research team, and then reviewed for plain language by another researcher, external to the Healthlines team, with relevant expertise.
The new versions of the
Healthlines information sheets evaluated in the embedded trials were presented as 4-page A4 booklets and were divided into 8 sections (as compared with 15 sections in the original versions), with contrasting colour and a larger font used for section headings, to aid navigation. The front page contained a ‘bulleted’ list of trial summary information, logos, a contents list and contact details, which were jointly intended to aid reader navigation and have visual appeal. The re-wording of the information sheets included greater use of lay terms, short sentences and paragraphs. Additionally, the covering letters were revised by shortening them by around one third, particularly by removing content that was replicated in the information sheets, as well as adding ‘bullets’ for lists and using bold, lower case text for emphasis. Examples of the original and revised patient information material and covering letters used in
Healthlines Depression are available as Additional files
1,
2,
3 and
4. Data on word count and readability scores [
18] can be found in Additional file
5. The letters were reduced in length by the process, although the patient information sheets were increased. Readability was maintained or improved in all cases. It should be noted that word length and readability indices are presented for illustrative purposes only, as they are only partial indicators of the difficulty of a document and changes in these were not the primary focus of the process.
Description of the Healthlines host trials and the embedded recruitment trials
Both embedded trials of recruitment interventions were conducted over a 4-month period (March 2013 to June 2013) towards the end of the
Healthlines recruitment window at the final 3 participating primary care practices in the Bristol area (from a total of 43 practices involved in
Healthlines). Practices serving populations from a range of inner city, suburban and rural settings were recruited with the help of the Primary Care Research Network (now NIHR Clinical Research Network,
http://www.crn.nihr.ac.uk/). For efficiency we restricted inclusion in the embedded trial to practices with patient list sizes of 9000 or more, using computer systems compatible with the
Healthlines search query (EMIS-LV, EMIS Web, and TPP SystmOne), and in one of three clinical commissioning groups in the South West for which the study had ethical approval. Letters of invitation, which briefly explained the nature of the study and commitment expected from each practice, were sent to practices meeting our criteria and identified by the Primary Care Research Network. Interested practices returned an expression of interest form, which was passed onto researchers to initiate relevant meetings.
Potentially eligible participants were identified from practice lists. For
Healthlines Depression, the eligibility criteria were aged at least 18 and a confirmed diagnosis of clinical depression using standardised measures. For
Healthlines CVD, participants were aged 40–74, with 20 % or greater risk of having a cardiovascular event in the next 10 years (QRISK2 score [
19]) and had to have at least 1 of 3 modifiable risk factors (systolic blood pressure of 140 or greater; a body mass index (BMI) ≥ 30; or current smoker). Additional inclusion criteria were access to a telephone, the Internet and having an Email address for personal use. There were no additional inclusion or exclusion criteria for the embedded trials.
Standard Healthlines procedure was for lists to be screened by GPs, and for patients with known exclusion criteria to be removed. A researcher assisted with searches and then randomised patients to receive either the original or optimised patient information materials (using simple randomisation in a 1:1 ratio). Each practice list was randomised separately following the same procedure: each patient was assigned a computer-generated random number using Excel (Microsoft Inc., Redmond, WA, USA). The list included an ID number for each patient, rather than any identifiable information, thereby reducing the risk of any selection bias on the part of the researcher conducting the randomisation. The list was then sorted by random number, with one half assigned to the optimised patient information materials and the other half to the original.
Patients were then sent the appropriate allocated information materials as part of the invitation to participate in The
Healthlines Study. Due to an expectation of lower recruitment, a reminder letter was mailed out to non-responders after 2 weeks in
Healthlines Depression (an administrative error meant that all non-responders were sent the standard letter, regardless of initial allocation in the embedded trial). No reminder was used in
Healthlines CVD. Patients were blind to the embedded recruitment trial (i.e. they were not told they were part of a recruitment trial – see ‘
Research ethics approval’), while researchers knew about the trial, but were blind to patient allocation.
The patient was requested to respond by returning either a valid consent form or a decline form in a pre-paid, pre-addressed envelope. If a patient did not wish to participate, they had the option of providing reasons on the decline form. In order to carry out pre-screening, interested patients in
Healthlines Depression were also asked to complete and return a standardised depression assessment tool, the Patient Health Questionnaire-9 (PHQ-9) [
20]. If the patient returned a consent form, researchers then contacted them to ask further eligibility screening questions. For
Healthlines Depression, those patients who scored 10 or more on the PHQ-9 were telephoned and verbally completed another standardised depression assessment, the Clinical Interview Schedule - Revised (CIS-R) [
21]. Subsequently, all eligible patients were randomised on a 1:1 ratio to receive either the
Healthlines Depression intervention and usual care, or usual care alone.
The process for Healthlines CVD was slightly different. Importantly, the number of patients actually randomised within each practice was restricted. This was because practice staff were required to carry out further eligibility assessments and staff availability was a concern. In particular, a CVD risk assessment was carried out by staff at the practice in order to calculate the patient’s QRISK2 score. The assessment involved taking clinical measures (e.g. blood pressure, weight and height) and collecting information that is required to compute a QRISK2 score. As covered in more detail above, patients with a 10-year CVD risk of 20 % or greater, as well as one or more modifiable risk factors (high blood pressure, overweight, or current smoker) were deemed eligible. Following further eligibility screening by the researchers, the first 25 eligible patients to complete this process at each practice were randomised on a 1:1 basis to the Healthlines CVD trial.
Outcome measures
For the embedded trial of optimised patient information materials in Healthlines Depression, the primary outcome was the proportion of patients randomised. Secondary outcomes were the proportion of patients who accepted the offer of invitation to participate, and the proportion of eligible patients who actively opted out of the trial (i.e. returned a ‘decline’ form). Although ‘harm’ in this context could include reduced recruitment in the intervention group, we did not measure other potential ‘harms’ (such as perceptions of increased pressure to participate in the intervention group).
For the embedded trial of optimised patient information materials in Healthlines CVD, the primary outcome was the proportion of patients who responded positively to the invitation to participate. This, rather than actual randomisation, was selected as the primary outcome because of a cap on recruitment numbers whereby only the first 25 eligible participants were randomised in each practice. This upper limit was implemented because of practice staff availability to carry out these assessments, and an initial agreement with researchers that 25 patient assessments would be sufficient to reach target recruitment across participating GP practices. The secondary outcome in Healthlines CVD was the proportion of eligible patients who actively opted out (i.e. returned a ‘decline’ form).
Sample size
The sample size calculations for the
Healthlines host trials are detailed in the original protocol [
5].
In the MRC-START programme, recruitment of host trials is guided by a generic sample size calculation which suggests a minimum of 400 patients to be approached in each arm of the host trial (note that this is the number of patients approached, not the number randomised which is the basis of standard sample size calculations). This generic calculation is detailed in the published protocol paper [
4].
Since the embedded trials were planned to take place in only a subset of all the primary care practices participating in Healthlines, data from the host trials were already available on the numbers of potentially eligible patients identified, responding, and randomised with which to inform the sample sizes of the embedded studies. Prior to the embedded trials, approximately 500 and 250 patients per practice were sent invitations to Healthlines Depression and Healthlines CVD respectively. Of these, approximately 5 % were randomised in Healthlines Depression, and 25 % responded positively in Healthlines CVD. By conducting the embedded trials within 3 practices, absolute differences of around 6–7 % in the primary outcomes in both embedded trials would, therefore, be detectable with 80 % power and 5 % 2-sided alpha. While the corresponding relative effects, particularly in Healthlines Depression of a greater than 2-fold increase in randomisation rate may seem implausible for the intervention under investigation, this was a pragmatic decision based on the time it took to prepare and obtain approvals for the optimised information materials. Indeed, the original intention was to include four GP practices, but there was a delay in obtaining ethical approval to implement the optimised patient information sheets.
Data analysis
Analyses were conducted in line with a standard statistical plan developed at Barts and the London Pragmatic Clinical Trials Unit by SE and VM (details available from the authors). Preliminary graphical and tabular examination of the data explored baseline comparability of trial arms and representativeness of the sample in terms of the overall eligible population. Outcomes were first described separately by arm, and then compared using logistic regression to estimate the between-group odds ratio (OR) and corresponding 95 % confidence interval (CI) on the basis of the intention-to-treat principle. A planned secondary analysis was performed to explore whether the impact of the intervention was moderated by gender. This was done by inserting the appropriate interaction term in the logistic regression models. All analyses were conducted using Stata version 12.1 (Stata Corp., College Station, TX, USA). The statistician conducting the analyses remained blind to allocation until all analyses were completed.
Research ethics approval
The MRC-START programme of embedded trials of recruitment interventions was approved by the NRES Committee, Yorkshire and the Humber – South Yorkshire (Reference: 11/YH/0271) on the 5 August 2011.
The Healthlines host trials were approved by the NRES Committee, South West, Frenchay, Bristol (Reference: 12/SW/0009). The embedded trials of recruitment interventions were added later as a substantial amendment, with optimised versions of the patient information sheets and covering letters submitted for approval (granted 14 March 2013). As the core information for patients was unchanged, but offered in a different format, approval was given so that participants did not need to be informed that they were taking part in an embedded trial of a recruitment intervention. This also served to avoid bias and the potential of misunderstanding by patients.
Trial registration
The Healthlines trials are registered with Current Controlled Trials: Healthlines Depression ISRCTN27508731 and Healthlines CVD ISRCTN14172341. The embedded trials of recruitment interventions were added as sub-studies to the original trial registrations.
Authors’ contributions
The following are members of The Healthlines Study Group responsible for the MRC-START in Healthlines embedded trials: Mei-See Man (MSM); Clare Thomas (CT); Louisa Edwards (LE); Alan A. Montgomery (AAM); and Chris Salisbury (CS). Other members of the Healthlines research team not directly involved with the MRC-START sub-study are listed under acknowledgements. MM was Healthlines programme manager (maternity cover) during trial recruitment and led the ethics amendment and sub-study registration for conducting the embedded MRC-START trial within The Healthlines Study. CT was Healthlines programme manager during the set-up of the Healthlines trials. LE was the Research Associate and conducted the practice database searches, randomisation of participants and collected the primary outcome data. CT and LE collated the data for analysis. AAM was the PI and lead statistician for the Healthlines trials. CS is the chief investigator for the Healthlines research programme and has overall responsibility for the design and implementation of the Healthlines trials. MM, LE, CT, AAM and CS all commented on a draft of this paper.
The following are members of the MRC-START Group: Peter Bower (PI: PB); David Collier (DC); Sandra Eldridge (SE); Jonathan Graffy (JG); Adwoa Hughes-Morley (AHM); Anne Kennedy (AK); Peter Knapp (PK); Vichithranie Madurasinghe (VM); Jo Rick (JR); Chris Salisbury (CS); Nicola Small (NS); David Torgerson (DT); and Shaun Treweek (ST). JR, as project manager for the MRC-START Study, worked with the Healthlines group to develop the embedded study protocol, data sharing agreement, substantial amendment and embedded study registration materials. JR drafted this version of the paper based on reporting guidelines developed by SE and VM. PK led on the methods and development of the optimised participant information. SE and VM designed and VM conducted the data analysis, PB led on the discussion and implications. PB, DC, SE, JG, AK, PK, CS, DT and ST were applicants on the original MRC-START application. PB, DC, SE, JG, AHM, AK, PK, VM, JR, CS, NS, DT, and ST all commented on drafts of the paper. All authors read and approved the final manuscript.
The Healthlines Study Group
Mei-See Man, School of Social and Community Medicine, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK
mei-see.man@bristol.ac.uk
Clare Thomas, Centre for Academic Primary Care, NIHR School for Primary Care Research, School of Social and Community Medicine, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK
clare.thomas@bristol.ac.uk
Louisa Edwards, Centre for Academic Primary Care, NIHR School for Primary Care Research, School of Social and Community Medicine, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK
louisa.edwards@bristol.ac.uk
Alan A Montgomery, Faculty of Medicine and Health Sciences, Queen’s Medical Centre, Nottingham, NG7 2UH, UK
alan.montgomery@nottingham.ac.uk
Chris Salisbury, Centre for Academic Primary Care, School of Social and Community Medicine, University of Bristol, 39 Whitely Road, Bristol, BS8 2PS, UK
c.salisbury@bristol.ac.uk
MRC-START Group
Peter Bower (PI), Medical Research Council North West Hub for Trials Methodology Research, Manchester Academic Health Science Centre, Centre for Primary Care, University of Manchester, Oxford Road, Manchester, M13 9PL, UK
peter.bower@manchester.ac.uk
David Collier, William Harvey Research Institute, Barts and the London Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ, UK
d.j.collier@qmul.ac.uk
Sandra Eldridge, Blizard Institute, Barts and The London School of Medicine and Dentistry, 4 Newark Street, London, E1 2AT, UK
s.eldridge@qmul.ac.uk
Jonathan Graffy, Department of Public Health and Primary Care, The Primary Care Unit, University of Cambridge, Cambridge, CB2 OSR, UK
jpg43@medschl.cam.ac.uk
Adwoa Hughes-Morley, Medical Research Council North West Hub for Trials Methodology Research, Manchester Academic Health Science Centre, Centre for Primary Care, University of Manchester, Oxford Road, Manchester, M13 9PL, UK
adwoa.hughes-morley@manchester.ac.uk
Anne Kennedy, NIHR Collaboration for Leadership in Applied Health Research and Care (CLAHRC) Wessex, Health Sciences, University of Southampton, Highfield, Southampton, SO17 1BJ, UK
a.kennedy@soton.ac.uk
Peter Knapp, Department of Health Sciences, University of York, University Road, Heslington, York, YO10 5DD, UK
peter.knapp@york.ac.uk
Vichithranie Madurasinghe, Blizard Institute, Barts and The London School of Medicine and Dentistry, 4 Newark Street, London, E1 2AT, UK
v.madurasinghe@qmul.ac.uk
Jo Rick, National Institute of Health Research (NIHR) School for Primary Care Research, Manchester Academic Health Science Centre, Centre for Primary Care, the University of Manchester, Oxford Road, Manchester, M13 9PL, UK
jo.rick@manchester.ac.uk
Chris Salisbury, Centre for Academic Primary Care, School of Social and Community Medicine, University of Bristol, 39 Whitely Road, Bristol, BS8 2PS, UK
c.salisbury@bristol.ac.uk
Nicola Small, National Institute of Health Research (NIHR) School for Primary Care Research, Manchester Academic Health Science Centre, Centre for Primary Care, the University of Manchester, Oxford Road, Manchester, M13 9PL, UK
nicola.small@manchester.ac.uk
David Torgerson, York Trials Unit, Department of Health Sciences, University of York, University Road, Heslington, York, YO10 5DD, UK
david.torgerson@york.ac.uk
Shaun Treweek, Health Services Research Unit, University of Aberdeen, Fosterhill, Aberdeen, AB25 2ZD, UK
streweek@mac.com