Evaluating complex interventions in End of Life Care: the MORECare Statement on good practice generated by a synthesis of transparent expert consultations and systematic reviews

There are 57 million deaths each year. Despite being a core business of medicine, end of life care (EoLC) is neglected [1]. Although some people have excellent EoLC, many do not die as they would wish [2]. A major barrier is the lack of quality research; treatments, clinical guidelines and services are limited by a lack of evidence [3, 4]. Surveys, qualitative studies and reviews recommend that EoLC research is feasible and ethical [4] but, funding of EoLC research is poor [1, 5] and lacks common research guidance. Thus, randomised trials of EoLC treatments and services remain rare, often limited by poor recruitment, high attrition, bias, confounding and small sample sizes [6‐8]. There are challenges capturing relevant outcomes in frail patients who may lack capacity, raising ethical reservations [3, 6, 7, 9, 10]. Research evaluating EoLC is characterised as too slow, too expensive and frequently not producing useful results [2]. There is a need to improve research methods to evaluate models of service delivery and complex service level interventions in EoLC and identify good research practices to aid future studies. In response, the Methods Of Researching End of Life Care (MORECare) collaboration was established by the UK Medical Research Council (MRC) and National Institutes of Health Research (NIHR) to identify, appraise and synthesise ‘best practice’ methods for research evaluating EoLC. This paper reports the total integrated results from MORECare and the resulting guidance statement.

The literature reviews and scoping together identified 15,695 papers, of these 62 were included in the three systematic reviews [16‐18]. The results of the scoping and expert panel identified five main areas of contention/uncertainty that required TEC on: ethics [20], statistics (managing missing data, attrition, and response shift), [21] outcome measurement, [22] mixed method research, [23] and health economics [24]. Attendees of the five TECs totalled 140, with 133 responses to the five online consultations (Table 3). The three stakeholder workshops included 19 patients/carers and 12 clinicians. The integrated top ranked recommendations and synthesis with the literature formed: the MORECare statement detailing 36 best practice solutions for research in EoLC to improve study quality and set a standard for research in the future (Table 4); and 13 national/international MORECare recommendations to improve the environment for the development and evaluation of interventions in EoLC (Table 5).

This is the first comprehensive research specifically aimed at producing evidence-based guidance for researching treatments and services in EoLC. Our findings propose using randomised trials and other quasi-experimental or observational designs, which may be appropriate when randomisation is not appropriate. Alternative designs would build on traditional RCT methodology and the MRC framework by integrating observational or natural experiment methods, and taking account of implementation aspects, rather than taking a totally different approach. Mixed methods can be employed at all phases of development and evaluation. We found that patients and families value participation in research. Consumer or patient collaboration in developing studies can be valuable in ensuring ethical methods and in addressing the concerns of ethics committees. MORECare also concluded that it is ethically desirable to offer patients and families the opportunity to take part in research and it may be unethical not to offer this opportunity purely on the grounds of progressive disease. Outcome measures should be short, responsive to change and ideally used for both clinical practice and research. More controversially we propose that attrition and missing data should be expected in studies, and does not indicate poor design – indeed, a lack of attrition may mean that the wrong population has been studied. Attrition should be planned for in advance. A new classification of attrition and missing data was developed. Implications for implementation need to be considered at all stages of the project.

MORECare identified the need to involve relevant methodologists, researchers familiar with the challenges in EoLC and consumers/patients/families in studies. The multiple problems of patients mean that interventions are complex, combining symptom relief and physical, emotional, social and spiritual care. The teams required to conduct research in this field, therefore, may also need to be large and complex. Such teams require management, and funding bodies may need to take account of the costs involved.

Our conclusions on the ethical issues raised by EoLC research challenge earlier thinking, especially that randomisation is unethical [17]. There is growing support for the need for research into EoLC to improve practice. The conclusion from the ethics’ TEC was that it can be unethical not to offer research to this group of individuals. Concerns about approaching patients and families who are distressed or very ill are understandable, and this has often led ethics committees or others to raise concerns regarding research in EoLC. However, the vulnerability of patients and families is often simplistically understood. Koffman et al. identified five aspects: (i) communicative; (ii) institutional; (iii) deferential; (iv) medical; and (v) social vulnerability, which are relevant in EoLC and other situations, and might provide a broader framework for assessment [26].

The new MORECare classification for attrition: ADD – attrition due to death; ADI attrition due to illness; and AaR attrition at random – is novel, as is our statement that attrition should not be seen as an indication of poor research. Traditionally, guidelines propose that attrition of 5% or lower is inconsequential, whereas 20% or greater is unacceptable because of bias [27]. However, such a guideline fails to distinguish the reasons for attrition. Data which are missed because patients have died is very different to that missed because a patient has withdrawn consent or because they are symptomatic. To impute data such as a quality of life score for a patient who has died seems inappropriate. Whereas imputing data for patients who have moved away or are missed at random would be different. Thus, we designed a classification system for attrition to extend the commonly used classification in clinical trials of missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR). We envisage our classification could be used as an adjunct to understand trial data better. Further, while attrition introduces potential for bias, our argument is that a lack of attrition may indicate a different bias, that a less relevant population has been included. In theory, attrition can introduce selection bias in randomized trials. Conversely, a recent secondary analysis from 10 trials evaluating treatment of musculoskeletal disorders, challenged this; the authors found no indication that attrition altered the results in favour of either treatment or control [28]. Work is needed to explore the effect of attrition on bias in EoLC studies, and the best ways to impute data. We believe that our proposed classification will help to clarify reporting and may well be applicable in other populations where attrition is high, for example those who are elderly or frail.

Our findings that outcome measures should be short and easy to use support and further develop conclusions from a large European Network on outcome measurement in palliative and EoLC [29]. We propose further that outcome measurement should be timed to balance the effect of the intervention and loss of data through attrition. Tang and McCorkle proposed an alternative approach, of conducting weekly interviews to ensure adequate data in end of life care studies [30]. While this can be appropriate in some circumstances, it may cause undue interview burden, and we believe the MORECare recommendation of careful timing is more appropriate.

Our proposal that outcome measures used in research should also be valuable in clinical practice is novel, as this is not a usual requirement when assessing outcome measures, although it relates to aspects in the COSMIN (COnsensus-based Standards for the selection of health status Measurement INstruments) checklist of face validity and responsiveness to change [31]. In EoLC there are now many different outcome measures. The European survey identified more than 100 different outcome measures in palliative care research, but 94 of these were used fewer than 10 times [32]. There is a need for standardisation around the few best validated short scales which are widely used, perhaps with core and add on modules [33], so that in the future results from studies may be pooled.

Policy makers, clinicians and patients responding to the MORECare consultation raised the need for research results to be timely to influence service developments. MORECare concluded that robust evaluation data can be found beyond RCTs. This is increasingly raised as an option in research generally [34] and in the most recent formats of the MRC guidance [11]. Secondary analysis of existing data sets, including data collected nationally or in routine clinical practice, and quasi-experimental, epidemiological and qualitative or especially mixed methods can be helpful, especially if the original data is of high quality [34]. Some good examples of secondary analysis of data are available in the USA, where data on hospital activity and costs are routinely available [35].

We attempted to identify the key areas of methodological difficulty in EoLC research; however, we were limited to conducting only five TECs and three systematic reviews, and ideally would have conducted more, especially regarding more specific recommendations on recruitment methods, alternatives to the standard RCT (such as the use of cluster [36] or fast-track trials [37]) and the use of quasi-experimental designs. We see the MORECare statement as a first step, which ideally will be expanded and refined through further testing. Arguably we could have conducted a more traditional Delphi consultation rather than TEC, but the TEC approach allowed a more interactive discussion by allowing novel and sometimes challenging proposals. It did limit our international membership – and only the outcomes summit (which was conducted alongside an international congress) had truly international participation. A particular strength was the involvement of patients and caregivers in all our TECs and throughout the MORECare project, which is uncommon in the development of guidance on good research practice. This involvement resulted in novel proposals, for example, the recommendation for researchers to attend ethics committee meetings with patients, caregivers or consumers came from a patient.

This research study, which integrated data from three systematic reviews and five TECs, resulted in a statement (the MORECare Statement) of 36 best practice solutions for immediate practice and 13 wider recommendations for national and international consideration. The results show how ethical research is possible, what is required of outcome measures, the need for clinical and academic collaborations and how mixed method research can be reported. Some points in the statements challenge current research practice, for example with new recommendations regarding anticipating, planning for and managing attrition and missing data. Other points require longer term change, for example legal change to permit advanced consent. The MORECare Statement sets clear standards on good research practice in evaluating services and treatments in EoLC. The Statement is relevant to those designing, funding and reviewing studies and should be used alongside existing statements. It provides a first step in setting common, much needed standards for evaluative research in EoLC.