The integrated care programme was delivered to patients over the age of 65 with at least one long-term condition, and we recruited these patients to the CLASSIC cohort [
34]. FARSITE is a software package (
http://nweh.co.uk/products/farsite) that enables centralised searching of general practitioner (GP) records. FARSITE was used to generate a list of eligible patients in each practice, and the results were provided to general practices to allow them to remove any patients meeting the exclusion criteria (patients in palliative care or with reduced capacity to consent) prior to asking them for consent. A total of 12,989 patients were eligible between November 2014 and February 2015. If they did not respond, they were sent a reminder 3 weeks later. Participants were offered an incentive of a £10 voucher. At baseline, 4377 people (34.2%) returned a questionnaire. We did not have access to data on non-respondents.
For inclusion in PROTECTS, patients had to have 2 or more self-reported long-term conditions from a list of 15 [
35], and must have been assessed as needing some assistance with self-management, defined via scores on the Patient Activation Measure (PAM) [
36]. The PAM allows activation to be categorised into four levels. Level 1 includes passive recipients of care, level 2 includes those who lack the basic knowledge and confidence to self-manage, level 3 is those who have the basic knowledge but lack the confidence and skills to engage in self-management and level 4 is those who have the knowledge, confidence and skills and may only require support during times of stress [
36]. We included patients in PROTECTS whose scores placed them in level 2 or 3 of activation, because these patients showed some evidence of self-management which could be improved by health coaching.
Procedures
The intervention was health coaching, as defined earlier. The content of the health coaching was based on three core mechanisms:
1.
Telephone health coaching involved support and encouragement to the patient to promote healthy behaviours around diet, exercise, smoking and alcohol, through provision of information and motivation for long-term conditions. The core health coaching materials include telephone and associated patient tracking and management software, and health coaching scripts for lifestyle support.
2.
Social prescribing involved links to resources in the wider community through the community and voluntary sector [
37,
38]. Access to local resources was provided through either PLANS (
http://www.plansforyourhealth.org/, a self-assessment tool for users to assess their health and social needs, with links to relevant community resources and local support) or the Ways to Well-being site (on-line resources and information, no longer available in the form used in the trial).
3.
Low-intensity support for low mood included assessment of common mental health problems, simple lifestyle advice and behavioural techniques to manage mood, and use of appropriate risk assessment protocols [
39,
40].
Six monthly phone calls to participants were planned. The receipt of four out of the six planned calls was considered a complete ‘dose’ of the intervention.
The PROTECTS intervention was delivered by a ‘health advisor’ (a National Health Service (NHS) Agenda for Change Band 4 worker) with skills in information technology and communication, as well as experience in working with the general public. Advisors already had experience with coaching for diabetes and use of social prescribing. The health advisor attended 3 days of training specific to working with low mood. They were given a manual which outlined the key elements of the low-intensity intervention used (behavioural activation, cognitive restructuring, problem solving). They also received monthly group clinical supervision which focussed on working with low mood. The health advisor were further supported by a specialist nurse manager and received additional advice on mental health and social prescribing (i.e. referral to relevant community resources) from the research team. Patients routinely had continuity in their coach for the duration of their treatment. There were no formal links with primary care as part of the intervention. The health coaching was delivered via telephone from a central NHS facility. Proactive, monthly calls of around 20 min were made for a period of 6 months, with the option for additional calls to deal with complex patients or issues of risk. Health coaching staff were trained to customize calls to the individual patient. Provision of support for low mood and social prescribing were made where appropriate.
The design meant that the comparator for patients meeting the eligibility criteria who were not selected for the intervention was usual NHS care. We collected details of that care for the economic evaluation.
Outcomes
PROTECTS was nested within the CLASSIC cohort, which used a wide range of measures, varying at different time points. A pre-specified subgroup of primary outcomes were used in PROTECTS. All outcomes were collected via postal survey at four time points across the study: at baseline, then at 6, 12 and 20 months. The protocol was registered and updated in a registry (ISRCTN 12286422).
The primary outcome measures were:
-
- Self-management. The PAM is a self-report measure of patient knowledge, skills and confidence in self-management for long-term conditions [
22,
36,
41]. We used the short 13-item version. The score is categorised into four levels for eligibility determination, although we used the continuous score in the analyses.
-
- Quality of life. The World Health Organization Quality of Life brief measure (WHOQOL-BREF) is a 26-item measure of global quality of life (QOL), which has been validated in a large international population with physical and mental long-term conditions. QOL is measured across four domains: physical, psychological, social and environmental, as well as a single-item scale for QOL [
42]. We used the physical domain score as the most relevant in relation to the PROTECTS intervention.
Secondary outcome measures were:
-
- Depression. The Mental Health Inventory (MHI-5) is a 5-item scale which measures general mental health [
43]. This measure is well validated for identifying depression symptoms, with a higher score indicating better mental health [
44,
45]. The recommended cutoff score of 60 was used to indicate the presence of ‘probable depression’ [
45], although we used the continuous score in the analyses.
-
- Self-care. The Summary of Diabetes Self-Care Activities (SDSCA) is a 7-item measure assessing the number of days per week respondents engage in healthy and unhealthy behaviours (i.e. eating fruits and vegetables, eating red meat, undertaking exercise, drinking alcohol and smoking) [
46].
Power and statistical analysis
At the time of study development, there were no bespoke methods for powering this TWiCs design, and we used conventional methods [
47]. We powered the study to have 80% power (alpha 5%) to detect a standardised effect size of 0.25 on any continuous outcome measure. Allowing for 25% attrition amongst participants — and assuming that outcome measures at baseline correlate 0.5 with their respective follow-ups — 504 patients were indicated, with 252 randomised to treatment. The CLASSIC cohort included 1306 patients eligible for PROTECTS, and we randomly selected 252 to be offered the intervention. The uptake rate was lower than anticipated, and we therefore offered the intervention to a further 252 patients. This resulted in a final intervention group of 504 of which 207 consented to the intervention, with the remaining 802 as controls. However, under the TWiCs framework, all 504 patients offered treatment remain in the treatment group in analysis, including those who declined. In consequence, the eventual effect size detectable at 80% power was 0.39 amongst the subsample consenting to treatment.
The analysis followed intention-to-treat principles and a pre-specified analysis plan. In summary, we report the trial and analysis according to updated CONSORT standards and utilising the extension for pragmatic trials [
48]. The main hypothesis test of the intervention was that the overall effect of the intervention is zero. The primary analysis used complete cases only. Condition group was used as a binary variable. All outcomes were treated as though continuous and normally distributed (in all cases both skewness and kurtosis were < =1.0) and analysed using linear multiple regression. Baseline values of outcomes and a set of pre-specified covariates considered prognostic of outcome were included in all analyses: gender, age (categorised as 65–69, 0–79, 80–98), health literacy [
49], social support [
50], patient activation, depression and quality of life (physical health domain). Robust estimates of variance were used accounting for the clustering of patients within practices.
We ran two sensitivity analyses. The first repeated the primary analyses using multiple imputation to include cases with missing baseline or follow-up data. Missing data values were imputed using chained-equation multiple imputation and scores on all available outcome measures and patient demographics at baseline and follow-up. Twenty multiple imputation sets were used to ensure stability of results. The second sensitivity analysis assessed the robustness of the primary analysis results to removal of the pre-specified covariates from the model (not including the outcome at baseline).
Health coaching in the trial was delivered by an existing service managing other patients outside the trial, rather than a bespoke service. This, combined with the time taken to administer and analyse the cohort and randomly select the groups, meant that no patient was offered treatment until 6 months after the baseline assessment for the CLASSIC cohort, and for some the offer was not made until month 12 or later. This caused variations in the duration of time before start of the treatment (range 259 to 513 days after baseline assessment). Length of follow-up from end of treatment to 20 months follow-up was similarly variable. Thus, the trial is considered to have run over 20 months, with patients receiving treatment at any time after the initial 6 months. As these implementation delays were not anticipated, the pre-specified analysis plan stated that the primary analysis would assess the change in outcomes between baseline and 20 months follow-up.
The design provides an estimate of the mean effect in people offered treatment. Compared to a pragmatic trial, which provides an estimate of the mean effect in people agreeing to treatment, the effect is ‘diluted’ by the proportion of patients in the treatment arm who do not consent to treatment. An estimate of the treatment effect in those patients consenting to treatment was derived through application of a complier average causal effect (CACE) analysis [
51,
52]. The CACE estimator was obtained by dividing the mean effect estimate by the proportion giving consent [
51]. The CACE estimate is typically larger, but the power to detect an effect is not greater, since the variance of the estimate increases proportionately [
53].
Cost-effectiveness analysis
The primary outcome measure for the economic evaluation was the EuroQOL 5-Dimension 5-Level (EQ-5D-5L) [
54], a generic measure of health-related QOL covering five domains (mobility, self-care, usual activities, pain/discomfort, anxiety/depression). This new version was developed due to concerns over the lack of sensitivity to change of the original scale, and consists of five severity levels for each domain. Published English general population preference weightings were used to convert responses to a single utility index [
55].
The perspective of the economic analysis was that of the English NHS. Individual patient-level health care resource utilisation over the trial period was collected from two sources. The number of GP contacts in the previous 6 months was collected from self-report data at 6-monthly intervals. Hospital utilisation was extracted from linked administrative patient records provided by the NHS, divided into emergency admissions (short stays ≤5, long stays > 5 days), elective admissions, elective day cases, outpatient attendances and accident and emergency (A&E) department attendances.
The economic analysis assessed the incremental cost-effectiveness of the offer of health coaching compared with usual care from the perspective of the NHS. EQ-5D-5L data were combined with in-hospital mortality information from the secondary care utilisation data, applying a utility value of 0 upon death. Quality-adjusted life years (QALYs) were calculated using the area under the curve method assuming linear extrapolation of utility between time points. QALYs in the second year of the trial were discounted at an annual rate of 3.5% as specified by NICE [
56].
Intervention costs were estimated combining the cost of training and supervision, written materials and delivery of the health coaching sessions. The intervention was offered to all participants selected, although only 189 received at least one call. Only patients receiving at least one call were assigned treatment costs, and the intervention costs were therefore estimated based on these 189 participants.
Patient-level resource utilisation data were combined with relevant unit cost data for the price year 2014–2015 to calculate total costs. Unit costs not available for this price year were inflated to 2014/2015 prices using the consumer price index [
57]. Costs occurring in the second year were discounted at a rate of 3.5% [
56]. Unit cost figures were sourced from the Personal Social Services Research Unit’s unit costs of Health and Social Care 2015 and national NHS Reference Costs [
58,
59].
Follow-up questionnaire completion dates were missing in a small number of cases (
n = 2). In these instances, dates were imputed using the mean length of time between baseline and follow-up for the sample for the purpose of QALY and cost calculations. Missing information on age and gender were sourced from the linked hospital administrative data, where available (gender
n = 6, age
n = 35). For the remaining individuals with missing age (
n = 30) or missing baseline EQ-5D-5L (
n = 29), mean imputation was used to ensure independence from treatment allocation [
60].
For missing EQ-5D-5L and resource use data, we used multiple imputation by chained equations (ICE) to generate 50 imputed datasets assuming the data were missing at random. The independent variables specified in the imputation models were age, gender, treatment arm and baseline EQ-5D-5L. To account for non-normality, predictive mean matching was used which forces imputations to only take values observed in the original dataset. Multiple imputation (MI) was conducted using Stata’s ICE package, and analysis using Stata’s MI package.
The incremental cost-effectiveness ratio (ICER) was calculated, adjusting for age, gender, and baseline EQ-5D-5L index score [
61]. To assess uncertainty surrounding the estimates and to account for the typically skewed nature of cost data, incremental costs and QALYs were bootstrapped using pairwise bootstrapping with replacement using 10,000 replications. Cost-effectiveness planes plot these 10,000 bootstrap replications of the ICER estimates to illustrate the uncertainty around the point estimate of the ICER in probabilistic terms. Finally, cost-effectiveness acceptability curves (CEACs) were plotted to graphically represent the probability of the intervention being cost-effective across a range of cost-effectiveness thresholds.
The primary economic analysis was based on a comparison on the full sample with MI. A sensitivity analysis was performed using only the complete case sample for which there were no missing data. We also took advantage of the implementation delays to perform a further sensitivity analysis separating the trial period into two parts: baseline to 6 months follow-up, where no treatment had yet been received; and 6 months to 20 months follow-up, where we expect any treatment effects to occur. Stata version 14 was used in the analysis.