Protocol for a process evaluation of an external pilot cluster randomised controlled trial of a theory-based intervention to improve appropriate polypharmacy in older people in primary care: the PolyPrime study

The PolyPrime intervention is a theory-based intervention aimed at improving appropriate polypharmacy in older people (aged ≥65 years) in primary care and was developed in accordance with the MRC guidance [10‐12]. This involved conducting Theoretical Domains Framework (TDF)-based semi-structured interviews with general practitioners (GPs) and community pharmacists [10]. The TDF is an integrated framework of behaviour change theories that enables mediators of behaviour change to be identified and targeted. These interviews aimed to identify potential barriers and facilitators to prescribing and dispensing appropriate polypharmacy, which were then mapped to four behaviour change techniques (BCTs) to form the “active components” of the intervention [11]. The intervention, which was then tested in a small-scale feasibility study [12], consists of two main components: an online video demonstrating how GPs can improve appropriate polypharmacy during typical consultations with older patients (BCT: ‘Modelling or demonstrating of behaviour’; [13]) and a patient recall process, whereby patients are invited to medication review consultations with GPs (on two occasions; an initial medication review and a 6-month follow-up appointment with the same GP) [12, 14]. The video includes feedback from both a practising GP and a simulated patient emphasising the positive outcomes of the consultation (BCT: ‘Salience of consequences’ [13]). The video also includes additional educational slides highlighting key issues which GPs should consider when conducting the medication reviews, along with links to additional resources for GPs [14, 15]. To facilitate the patient recall process, two complementary intervention components involve GPs making explicit plans at weekly practice staff meetings of when and how they would ensure that target patients are prescribed appropriate polypharmacy (BCT: ‘Action planning’ [13]) and GPs receiving prompts from reception staff to carry out this plan when target patients attend their appointments (BCT: ‘Prompts/cues’ [13]).

The MRC guidance also states that process evaluation is an important step in developing and testing complex interventions [8, 9]. Process evaluations are studies that run parallel to, or follow, intervention trials and which can be used to assess intervention implementation, potential mechanism of impact, and to identify relevant contextual factors (i.e. barriers or facilitators to the implementation or effects of the intervention) [9]. These evaluations align with the underlying aims of pilot studies which seek to evaluate the feasibility of recruitment, retention, assessment procedures, and implementation of a novel intervention, which often include assessments of intervention fidelity [16].

Investigating how complex interventions, which often include multiple components, are implemented is imperative in understanding how each component works in practice and if modifications are needed before proceeding to larger scale trials [17]. One of the main aspects of intervention implementation is intervention fidelity, defined as whether the intervention is delivered as intended [9]. The Treatment Fidelity Workgroup of the NIH Behavioral Change Consortium has recommended that the following five areas need to be addressed to assess intervention fidelity: study design, training, treatment delivery, treatment receipt, and treatment enactment [18, 19]. It is also assumed that when an intervention or the ‘active components’ of the intervention are delivered as per protocol and a high level of fidelity is achieved, researchers can have greater confidence in the research outcomes [20].

The successful implementation of a complex intervention often depends on the acceptability of the intervention to both those delivering and receiving the intervention [21]. While the MRC guidance highlights that assessing acceptability is an important element within a process evaluation study, they do not provide guidance on the specific methodology to achieve this [9, 22]. Sekhon et al. who define acceptability as a multi-faceted construct that reflects the extent to which people delivering or receiving a healthcare intervention consider it to be appropriate, based on anticipated or experienced cognitive and emotional responses to the intervention have developed a theoretical framework of acceptability (TFA). The TFA consists of seven constructs: affective attitude, burden, intervention coherence, ethicality, opportunity costs, perceived effectiveness and self-efficacy, which researchers can use to guide assessments of acceptability (see Table 1) [22, 23]. Previous research has found that the use of this framework provided a more substantial assessment of intervention acceptability when asking participants about this construct [24].

The MRC guidance also suggests that exploring the potential mechanisms of action (i.e. the process by which an intervention component exerts its effect) is important in understanding how the intervention works and which components are essential. The use of a mixed methods approach is encouraged, whereby quantitative data can be used to investigate the hypothesised causal pathways, and qualitative data can be used to better understand the pathways from the participants’ perspective [25]. This often utilises the theory underlying intervention development to focus the evaluation and provide additional insight into causal mechanisms affecting outcomes [26].

This paper describes the rationale, methods and analysis plan for an embedded process evaluation for the PolyPrime intervention. This external pilot cluster randomised controlled trial is ongoing and will be completed by January 2022. A protocol paper for the main pilot trial has been published [14].

The aim of the PolyPrime process evaluation is to test the implementation of the PolyPrime intervention in primary care.

The objectives of the study are to:

A mixed methods process evaluation study will be undertaken, involving interviews with GPs and practice staff, patient feedback questionnaires, data collection forms completed by practice staff and audio-recordings of patient medication review appointments. Ethical approval has been obtained from by the North of Scotland (REC reference: 19/NS/0100) and the Irish College of General Practitioners Research Ethics Committees (RECs).

The main PolyPrime study is currently being conducted in nine GP practices in Northern Ireland (NI) and the border counties of the Republic of Ireland [ROI] (Donegal, Leitrim, Sligo, Cavan, Monaghan and Louth). Due to the COVID-19 pandemic, three practices have withdrawn from the original 12 which were initially recruited. The process evaluation will be conducted only in the GP practices randomised to the intervention arm. An overview of the main pilot trial and process evaluation activities can be seen in Fig. 1.

A detailed overview of the sampling and recruitment procedures for GPs and patients has been described in detail elsewhere [14]. Briefly, GP practice recruitment (now complete) took place in two stages. Firstly, a random sample of GP practices was contacted via a letter seeking expressions of interest. The second stage involved research nurses from the Northern Ireland Clinical Research Network (NICRN – Primary Care) and Trinity College Dublin (TCD) telephoning the practice manager or lead GP in each practice to determine their interest in receiving information about the study. GPs were asked to provide written informed consent, which included agreeing to take part in an interview towards the end of the study and to audio-record the discussion during a medication review with a patient’s consent should the practice be randomised to the intervention arm.

One member of practice staff (i.e. practice manager or receptionist) from each intervention arm practice (i.e. those involved in implementing the intervention within the practice) was recruited to facilitate implementation of the intervention (i.e. patient recruitment and scheduling medication review appointments), complete data collection forms and to participate in a post-intervention feedback interview. Practice staff received an information sheet providing details about the interviews and were asked to provide written informed consent.

Following GP practice recruitment, practice staff were asked to screen patient records to identify potentially eligible patients with the goal of recruiting up to 10 patients per practice (for details on patient recruitment, see protocol paper for the main pilot trial [14]). Patients have been asked to provide written informed consent, which includes agreeing to complete a feedback questionnaire towards the end of the study and to the GP audio-recording the discussion during their medication review appointments should they be from a GP practice randomised to the intervention arm. Patients will still be able to participate in the main pilot cRCT even if they do not agree to have their medication review appointments audio-recorded. At the time of submission, patient recruitment is almost complete.

The following text refers to the data sources identified in Tables 2 and 3.

The following data will be collected on study-specific data collection forms:

Analysis in relation to study-specific data collection forms and patient questionnaires will use descriptive statistics (e.g. counts, means, standard deviations) and will be conducted using SPSS (version 26.0). Recorded practice-level data will relate to the number of times the intervention GPs access the online video, the number of weekly practice staff meetings held at which explicit plans are made to recall patients for medication reviews, and the number of prompts GPs receive from practice staff to conduct medication reviews. These counts will provide an indication as to whether the intervention has been delivered and received as intended (i.e. intervention fidelity), in conjunction with the outcome data collected in the pilot trial [14]. Patient-level data will be available from the patient feedback questionnaires (see Additional File 1). The acceptability of the study procedures and medication reviews to patients will be assessed via responses to questions (yes/no), scales (Likert-type) or a series of statements relating to acceptability.

Analysis of the post-intervention feedback interviews will use a framework analysis [29, 30], which will involve seven stages. The first stage will involve an in-depth familiarisation process. Coding of the transcripts will then take place in two phases. The second stage (first coding phase) will adopt a deductive approach using the TFA constructs [22] to frame the questions relating to the overall acceptability of the intervention in the GP and practice staff interviews (see the “Intervention acceptability” section and Table 1). The third stage (second coding phase) will adopt an inductive approach, in which emerging themes will be identified. This will be followed by the development of a working analytical framework (Stage 4), applying the analytical framework (Stage 5), charting data into the framework matrix (Stage 6) and interpreting the data (Stage 7). All transcripts will be analysed independently by at least two members of the research team. Two sets of analysis will be undertaken, one for each professional group (i.e. GPs and practice staff).

Analysis of the medication review appointment transcripts will be conducted in a similar way to that outlined above. The second stage (first coding phase) will adopt a deductive approach using the BCT taxonomy [13] which will be used to assess if GPs delivered any additional BCTs during the patient recall process. The third stage (second coding phase) will adopt an inductive approach, in which any additional emerging themes will be identified.

Once all quantitative and qualitative data have been analysed separately by the researchers, the results from all five data sets (GP feedback interviews, practice staff interviews, patient feedback questionnaires, medication review appointment recordings, study-specific data collection forms) will be compared. This will follow a triangulation protocol [31] involving six steps (Table 4). Step 1 will involve two researchers examining the interpretative summaries of all analyses to identify the key findings for each data set. The findings related to each research question (outlined above) from all five data sets will be sorted and separated into three files (one for each research question—fidelity, acceptability, mechanisms of action). Step 2 will involve researchers reviewing the contents of each file to identify the key themes discussed in each data set to create a unified list of themes. These themes will form the rows of a convergence coding matrix used to summarise similarities from the second step; each key finding will be compared across the five data sets to create the matrix. For each key finding, paired comparisons will be made to compare the data coming from each data set. The relationship between data will be marked as one of four categories: silence, dissonance (or disagreement), partial agreement and agreement (see Table 4). Comparisons will be labelled as not applicable to denote the source of data which has supplied a theme or when neither data set in a paired comparison contained data related to the finding. This will be followed by convergence assessment (Step 3) and completeness comparison (Step 4). The level of agreement between the two researchers with respect to the degree of convergence between data sets (in both meaning and prominence of key findings) will then be calculated (Step 5). A method established by Miles and Huberman will be used, whereby agreements are scored with 1 and disagreements scored with 0, and the percentage agreement calculated [32]. Miles and Huberman suggest that an inter-coder reliability of 80% agreement on level of convergence is acceptable agreement among multiple coders. Any disagreements will be resolved by consensus through discussion with another researcher. The triangulated results will then be presented to the wider research team for discussion and to make a final decision on the interpretation of the findings (Step 6). All process evaluation activities have been summarised in Table 5.

All participants (patients, GPs, practice staff and GP practices) will be given a unique study ID number and data will be anonymised/pseudonymised (e.g. study-specific data collection forms and interview transcripts). Personal data including consent forms, completed questionnaires or transcripts will be held in a locked filing cabinet, within a locked office on a secure keycode-protected floor of the School of Pharmacy, Queen’s University Belfast (QUB) or the School of Pharmacy and Pharmaceutical Sciences, TCD. Electronic data will be stored within the QUB or TCD network space and will be password-protected to ensure confidentiality. Interviews will be digitally recorded (with permission and written, informed consent), transcribed and checked for accuracy. Once transcripts have been checked for accuracy recordings will be deleted and transcriptions stored within QUB or TCD. All participant consent forms, questionnaires, study-specific data collection forms and transcripts stored at TCD will be transferred to QUB, in line with General Data Protection Regulation guidelines for transferral of data, upon study completion. This will be done via recorded delivery for hard copy data and encrypted email for electronic data. When the study has been completed, the forms and transcripts will be securely stored for 5 years and then destroyed. The investigators will conduct the study in compliance with the protocol given approval/favourable opinion by the relevant RECs. Changes to the protocol may require REC approval prior to implementation, except when modification is needed to eliminate an immediate hazard(s) to patients. The CI and trial team, in collaboration with the sponsor, will submit all protocol modifications to the RECs for review in accordance with the governing regulations. Protocol compliance will be monitored by the trial team who will undertake site visits or telephone calls (if pandemic restrictions are in place) to ensure that the trial protocol is adhered to, such as, scheduling of medication reviews, contacting patients for appointments). Study paperwork will be reviewed, such as questionnaires and consent forms, to ensure that they are being completed appropriately and fully. An independent Trial Steering Committee has been established (consisting of two academic GPs, a health services researcher and statistician) which is overseeing the conduct of the trial and advising on safety in respect of the study.

The findings of this process evaluation will be communicated to all participants, published in relevant journals and presented at conferences. In addition to publishing scientific papers, we also plan to host a seminar for participating practices and patients in convenient locations to present the findings to them. This will be contingent on any pandemic restrictions that may be in place at that time.

This study was registered at ISRCTN (https://​doi.​org/​10.​1186/​ISRCTN41009897) on 19 November 2019 and ClinicalTrials.gov (https://​clinicaltrials.​gov/​ct2/​show/​NCT04181879) on 02 December 2019. At the time of resubmission of the revised manuscript (NI protocol version 4.0; date, 20 November 2020; ROI protocol version 5.0; date, 30 March 2021), 9 GP practices had been retained in the study following three drop-outs due to logistical issues arising from the COVID-19 pandemic, and 58 patients have been recruited and retained in the study.