Development, modelling, and pilot testing of a complex intervention to support end-of-life care provided by Danish general practitioners

Two narrative literature searches were performed: one focused barriers to end-of-life care and another on facilitators. The following medical databases were searched: biblioteket.dk, SweMed, PubMed, Embase, Cochrane Library, Cinahl and PsyhINFO.

In the pilot-testing, we adapted ideas and terms from the MRC guideline [26] and the process evaluation described by Grol et al. [18]. The purpose of process evaluation is to systematically assess the components in the intervention that could have an impact on the outcome of a pilot study. There is no standard process evaluation, but assessment is suggested to include: 1) the fidelity, 2) the quality and 3) the context of the intervention [18, 26]. We assessed the CME meeting and the EDS separately.

Adherence to the blueprint examines the extent to which the intervention components were delivered as intended, including whether development of the components succeeded and how well the components were implemented. To ensure adherence to blueprint in the CME meeting and delivery of similar content in each CME meeting, a test run was performed. All persons engaged in teaching at the CME meeting were present and received a copy of a detailed plan.

The implementation plan primarily included using existing newsletters which were sent to all GPs from two different senders. The Quality Unit for Cancer care in general practice in Central Region Denmark invited and reminded the GPs to participate in a CME meeting (free of charge) in their catchment area. The GPs were informed about the EDS through the regular DAK-E newsletter. Furthermore, the EDS was demonstrated at the CME meetings and briefly presented in a trade journal for Danish GPs [30].

The reach of the intervention was assessed by number of GPs attending CME meetings or signed up for the EDS compared to those who did either one or none of them. Background characteristics were retrieved for all GPs in the region to allow comparisons and to clarify if the intervention targeted specific subgroups of GPs i.e. younger GPs, urban GPs or female.

The quality of the CME meeting was investigated by using the attending GPs’ experiences of the meeting and the impact of the meeting on provision of end-of-life care. GPs’ experiences were assessed by using a mixed-methods approach carried out by an external evaluation unit in the Central Denmark Region [31]. The evaluation was done using questionnaires and interviews. After each meeting, a questionnaire was handed out to all participating GPs for themselves to fill in. The questionnaire consisted of seven items concerning benefits of attending the CME meeting, applied teaching methods, suggestions for improvements and if/how the CME meeting might affect their approach to end-of-life care in the future. Two of the items were answered on a 5-point Likert scale ranging from one to five and the remaining five questions were answered by free text comments (see Additional file 1 to see the questionnaire).

In addition to the questionnaire, three group interviews with fixed questions were conducted. Each interview was performed with a group of three GPs and carried out immediately after three of the six CME meetings (i.e. a total of nine interviewed GPs). The interviews focused on three topics: teaching methods, benefits from the CME meeting and possible improvements for future educational meetings. Each interview took approximately 15 min.

The short-term impact of the CME meeting on the GPs’ attitude was assessed by an email sent three months after the CME meeting to all participating GPs. They were asked: Have you changed anything in your approach to palliative care since the CME? (If yes: then what?; if no: then why not?).

To assess participants’ experience with the EDS, a postal questionnaire was planned to be send one year after the implementation to the GPs. The questionnaire contained items about relevance and functionality of the EDS. Furthermore, the specific function of the EDS that identified patients with potential palliative needs (more details are available in the results section), was to be adjusted retrospectively by using register-based data on deceased patients to possibly adjust the criterions for identification. These register-based data were also to be compared to how often the GPs ticked the pop-up window as irrelevant. Finally, as a proxy for usage of the EDS, data on how many times a pop-up was opened by the individual GP were to be retrieved.

The impact of the intervention on provision of end-of-life care was to be fully assessed after one year. The assessment would focus on patient-related outcomes on practice level, e.g. number of terminal declarations (a declaration releasing medical reimbursement for end-of-life care), frequency of prescription of anticipatory medication used in the terminal phase and number of home deaths. These data were to be retrieved from national registers using the unique identification number of every Danish citizens and their exact linkage to a general practice. The figures were to be compared before and after the intervention.

The overall context was the Danish health care system, which is tax financed and provides free access for all residents to health care services. More than 98% of the Danes are registered with a specific general practice, and the GPs are responsible for the health care provision to their listed patients [3]. If symptom relief or problem solving is too complex or not possible in primary care, the GPs can get advice from specialists or refer to specialist treatment [3]. The GPs are remunerated by Danish Regions according to a nationally negotiated scheme. Continuing medical education (CME) was not compulsory for Danish GPs until 2015, but they could receive remuneration for five days a year to cover education expenses and loss of earnings [32]. This study was performed in the spring 2014 in the Central Denmark Region with 843 GPs organized in 407 practices covering a population of approx. 1.3 million inhabitants.

Concerning the CME meeting, all participating GPs were registered (provider number, qualified GP/trainee/other, municipality, place of participation) upon arrival at the meeting for evaluation of the implementation of the CME meeting. GP characteristics (provider number, gender, number of GPs in different areas) were retrieved from the Central Denmark Region to compare participating and non-participating GPs. The external evaluator, who carried out the evaluation of the CME meeting, registered the answers on a Likert scale and collected the answers to the open questions in full wording for each topic and each meeting separately. The interviews were recorded and summarized in themes grouped according to the three overall topics (teaching methods, benefits from the CME meeting and possible improvements of the meeting) for each of the three interviews and the points were illustrated with typical statements from the interviewed GPs.

Data about EDS sign-up were retrieved from the DAK-E database. The patient-related data we had planned to use to fully assess the intervention was to be retrieved from national registers and DAK-E database.

Descriptive analysis regarding the attending GPs and their answers to the questionnaires were made. The Likert Scale answers were presented as percentages. The remaining five items with open answers were described qualitatively on the basis of trends and frequency of issues within each topic. Excel® was used for the descriptive statistics. As the analysis of the interview data were performed by an external evaluator no further information can be provided.

The framing of the CME meeting was based on adult learning theory with a problem-based approach to emphasise the relevance to clinical work [28]. The framing paid attention to the independent way in which most GPs work and their preferences for guidance [43, 44]. The content of the CME meeting was supported by research findings on GPs and end-of-life care, and an updated national guideline on palliative care for general practice published by the Danish College of General Practitioners (Table 2) [17].

The case-based teaching alternated between lectures and discussions. Three short films were produced for the meetings based on research findings to cover the topics in the meetings (Table 2). The films were used to facilitate the discussions between the GPs. Engaging with peers, participating actively and sharing own experiences with end-of-life care all aimed at increasing the effect of a CME meeting [18, 28, 44, 45]. GPs with special interest in end-of-life care were teaching together with a local palliative care specialist. Six identical CME meetings were held throughout the region based on the catchment area of the specialist palliative care teams.

The final EDS consisted of two connected parts: i) a pop-up window in the patient’s medical record (Fig. 1: The EDS pop-up window generated in the medical records to be filled in by GP) and ii) a list of patients with end-of-life needs and key elements in their care (Fig. 2: The list of all patients with palliative needs in the practice divided into patients with cancer and COPD, respectively).

The pop-up window in the patient’s medical record had four functions: an identifier of the patient’s potential end-of-life needs, a reminder to the GP of the patients and actions to take, a provider of medical advice, and a checklist of palliative tasks to consider during the end-of-life trajectory (Fig. 1).

To serve as an identifier, the pop-up window was triggered on the first time the GP opened the patient’s medical record if at least one of the following codes were registered in the electronic patient record (EPR): diagnosis of malignancy, palliative diagnosis or COPD with either MRC dyspnoea scale = 5 [46], body mass index < 18 or forced expiratory volume in 1 s < 30 (see Additional file 2 for exact list of diagnosis). The trigger diagnoses were chosen from existing identification tools [47‐49] and available patient information in the EPR. The aim was to identify patients with an estimated remaining life span of 12 months or less. The GP was asked to confirm if the patient was in the end-of-life phase. Additionally, the GP had to indicate the subsequent trigger: either next time the patient would be present in the GP practice or a specific date. This procedure was chosen so that the pop-up could work as a reminder, thereby making it easier for the GP to assume a proactive approach.

The pop-up window provided symptom-specific medical suggestions based on the recommendations from the national clinical guideline [17]. This function of the pop-up was integrated into the medicine module in the EPR to allow quick comparison between medical recommendations and prescribed medications. This easy access to medical advice was made to counterbalance any inadequate medical skills or doubts concerning end-of-life care among the GPs.

The checklist functionality in the pop-up window showed important issues to consider at some point of the palliative trajectory, e.g. making a terminal declaration, prescription of “in-case” box (anticipatory medicine), or registering performance status (Fig. 1). The checklist was linked directly to printable forms and assessment tools to minimize the time spent on administration and paper work. The checklists were also linked directly to the corresponding section in the online version of the national clinical guideline [17], where each issue was explained in detail.

The other part of the EDS was the list of the all patients identified by the GP as being in the end-of-life phase; this list was designed to help organise the care and promote a proactive approach. Two entries into the list were possible: either if the GP filled in anything in the pop-up (apart from ‘irrelevant’) or registered the patient as being in end of life by applying the International Classification of Primary Care diagnosis code “A99” in the EPR. The list showed key elements in end-of-life care (e.g. receiving specialised palliative care, palliative phase, date for next contact) for each patient listed (Fig. 2). If the GP was uncertain about a heading, the cursor could be dragged to the heading and an explanation would appear.

The information about key elements was automatically retrieved from data in the pop-up window and shown on the list using colour codes and simple explanations. The list was divided into two tabs to allow different information for different patient groups: one for cancer, one for COPD (Fig. 2). Another use of the list was to support the GPs in monitoring the clinical work. Additional suggestions as to how to use the data for this purpose were available for the GPs at the homepage of DAK-E [29].

It was possible to model the intervention to address identified barriers to end-of-life care in general practice and integrated facilitators to enhance the effect of the intervention. Although the participation rate was only 15%, the pilot-testing showed that the CME meeting was well received among the attending GPs. The meeting had an immediate impact on the GPs and addressed the identified barriers, which suggests high quality. Evaluation of the EDS could not be performed due to early shutdown. The process evaluation of the pilot-test revealed a need to look further into how the intervention in full scale could be designed to reach more GPs.

One barrier we intended to address in both the CME meeting and the EDS was identification of patients with palliative needs [33, 50, 51]. A Dutch intervention by Thoosen et al. found that patients with palliative needs identified by their GP had more contact with their GP, less hospital admissions, and were more likely to die at home than not identified patients [52]. These findings underline the importance of the GP’s awareness and identification of patients with palliative needs. In the study by Thoosen et al., the GPs had to apply an identification tool (RADPAC) by going through their patients manually. Hence, the identification was still dependent on the GP’s awareness of palliative needs. Mason et al. made a computerised tool to identify patients with deteriorating health due to advanced conditions [24]. They found that some GPs were reluctant to register the computer-identified patients as “palliative” due to associations to death. The resistance against using the term “palliative care” earlier in the disease trajectory compromised the effect of the tool. This underlines the need for a change in attitude alongside the implementation of an EDS.

The attendance rate in the CME meeting was low in our study compared to other Danish studies. A disease management programme in 2010 in the same region had an attendance rate of 69% [53]. However, these GPs were remunerated for participating, and the programme formed part of a regional initiative aiming to prioritise and optimise chronic care management. Another Danish study with 1-h CME meetings in 2012 focusing on lung cancer diagnosis had an attendance rate of 49% [54]. The low attendance rate in this study could have several explanations: lack of interest, no need for education in end-of-life care, bad timing of the intervention, or poor implementation of the CME meeting.

The low attendance rate in our study is unlikely to reflect a lack of need for education in end-of-life care. A prior Danish study identified a need of improvement of palliative care skills and a lack of confidence in providing end-of-life care among GPs [10, 38]. Furthermore, a British study reported that most GPs wanted training focusing on different care issues when asked about their educational preferences in palliative care [39]. The timing of the intervention may have adversely affected the attendance rate due to the disagreement between Danish Regions and the GPs. Finally, despite that the newsletter is the common way to invite GPs to participate in continuing medical education, we do not know if they actually read the invitation. Hence, the effect of different methods of inviting the GPs should be investigated in future studies.

The increased longevity in the population may result in rising incidence of cancer and more terminally ill patients with non-malignant disease. Most of these patients wish to be cared for at home. Hence, there is a growing need for GPs who are skilled and confident in providing palliative care.

The evaluation of the intervention revealed increased awareness among GPs of potential palliative needs in other patient groups than cancer patients. The findings also suggest that the GPs should take a more proactive role and organise the care. Although the participating GPs suggested higher prioritisation of demonstration of practical skills in general in the CME, we recommend maintaining the current balance between practice and theory. The reason for this is increased awareness and broadening of understanding of end-of-life care together with improved skills are prerequisites for optimising care, which would benefit all groups of patients.

As the CME meeting was well received by the GPs and addressed the main barriers to end-of-life care, it is ready to be used in a full-scale study assessing the effect of the intervention on patient-related outcomes. However, the EDS needs to be evaluated before used in full-scale study. Furthermore, it could be beneficial to assess the implementation itself, as the participation rate was low in the set-up tested in this study.

One of the strengths of the study was the systematic development of the complex intervention in accordance with the MRC guideline [46], which facilitated integration of evaluation in the design. This allowed analyses of the different steps and elements of the complex intervention and enabled us to investigate if the different components in the intervention worked as intended. This approach generally improves the applicability of tailored interventions to other settings as it makes it easier to adapt relevant components.

Another strength of the study was the inclusion of stakeholders from an early stage of the modelling of the intervention; this increased the applicability and facilitated the implementation. The reach of the intervention was evaluated using register data, which allowed comparisons between attending GPs and all GPs in the region. The external evaluation carried out in the CME meeting reduced the risk of bias, especially with regard to the interviews after the meetings. On the other hand important information might be lost from the interviews as the data was interpreted by an external evaluator. However, the external evaluator had participated in a meeting prior to the evaluation to ensure appropriate focus. The use of questionnaires for evaluation of the CME allowed us to assess both the immediate and the three-month self-perceived effect of the CME meeting on the GPs. However, the low participation in the follow-up compromised the generalisability of the three-month effect.

The use of narrative literature search to identify the barriers to end-of-life care introduced a risk of missing information, as the search was not exhaustive However, as the aim with the literature search only was to investigate barriers this was not a methodological concern in this study.

A limitation of using the guideline was the lack of standard process evaluation [18].

For the impact of the intervention the early shutdown of the EDS was a major limitation. This prevented evaluation of the reach, the participants’ experience, and the quality and of the EDS. Another limitation of the study was the low attendance rate at the CME meetings and the lack of possibility to examine attendance further in the current design.