Developing and testing a nursing home end -of -life care chart audit tool

The purpose of phase one of the study was to generate a list of possible end of life data elements for a chart audit tool that would capture best practices in the provision of care to dying residents. Drawing on the expertise of our research team, along with a critical review of the literature on best practices in end-of-life care for residents dying in NHs, the areas described in a framework for a good death by Bosek and colleagues [16] were used to determine broad headings under which items to include in the chart audit would be placed. These headings included pain and symptom management, clear decision making, preparation for death, and affirmation of the whole person. Discussion concerning the time frame in which to evaluate end-of-life care was guided by our review of the literature. The team also reviewed existing audit tools currently used within the local regional health authority in order to have a template for the range of items, the manner in which questions can be formatted, and the types of data that are best collected using an audit tool.

Following the methodology proposed by Gearing and colleagues [17], the information collected in phase one was collectively coded, categorized and synthesized to ensure items generated for inclusion on the draft audit tool were: 1) representative of the experience of a good death in the NH setting, common occurrences, and areas of significant concern,; 2) grouped under the broad conceptual domains that emerged in the review of the literature,; and 3) included both quantitative and qualitative items in order to capture the care delivered in the last month and week of life. For example, simple yes/no questions were used to indicate the presence of a specific symptom, and larger free text areas were provided to capture the type of psychosocial care provided to family members. At this point, by consensus of the research team, and guided by the review of the literature, it was decided that the audit tool would evaluate the care the resident received in their last month of life. The audit tool was formatted following the advice of Allison [5] and Banks [18] who stipulate that a well-designed audit tool is designed to promote accuracy of data capture and to limit the likelihood of missing data.

Phase two saw the development, validation, and refinement of the chart audit tool. A convenience sample of key decision makers, gerontological and palliative care clinicians involved in providing care to NH residents (either direct clinical care or administrative services) were recruited to participate in a focus group in order to provide feedback regarding the clarity, content, and scope of the audit tool.

The administrative staff at a large urban regional health authority long-term care program in central Canada, sent a letter of invitation to potential focus group participants via email. The invitation instructed them to email the study’s research assistant directly to indicate their interest in taking part. The research assistant determined if the participant met eligibility criteria, clarified any questions regarding the study, and provided the details of the focus group such as date, time and location. The research assistant also provided participants with a copy of the draft chart audit tool to review prior to the focus group. Focus group participants signed a consent form and completed a demographic questionnaire prior to the start of the discussion.

During the focus groups, participants were invited to share any insights and concerns they had about the audit tool. Each question on the audit tool was systematically discussed to assess its clarity, relevance, and importance. The focus group was facilitated by the first author while detailed notes of the discussion were recorded by the research assistant. The focus group lasted 45 min. The feedback obtained from participants regarding the clarity, relevance, and importance of the items along with any insights or concerns raised during the focus group was used to revise the tool. The final version of the tool was vetted by the research team prior to pilot testing, to review the items, and formatting of the draft audit tool.

In the course of conducting focus groups, critical appraisal of the draft audit tool was completed by gerontological and palliative care health care providers and administrators (n = 12). Focus group participants were on average 47 years old (median 49 years), with 23.95 years of experience in their profession (range 8–37 years) and 11.95 years working in long-term care (range 2–26 years). The group was comprised of registered nurses (n = 5), clinical nurse specialists (n = 2), a social worker (n = 1), a pharmacist (n = 1) and long-term care administrators (n = 3). All items on the audit tool were highly endorsed by the group, including the assessment period (i.e. last month of life). Minor revisions to the instrument questions were made to correct terminology (e.g., advance care planning levels of care) and minor corrections to the formatting of the tool were made based on comments derived from the research team.

In the final phase of this study, a brief trial of the audit tool was conducted in order to assess the reliability, time to complete, and the acceptability and utility of the instrument (See Fig. 1). Four NH facilities participated in the testing of the tool from which a random sample of 20% (n = 90) of their deceased residents’ charts from between January and December 2012 were audited; a sample size recommended for chart audits [19]. Two chart auditors with clinical experience in palliative care/gerontology were trained in the use of the audit tool for this study. The auditors independently extracted data from four of the same charts derived from the overall sample, in order to assess for ambiguous or confusing information on the audit tool. The abstracted data was compared for evidence of agreement on major variables. Every instance of variance was reviewed, using information in the original chart as the reference for further discussion [20]. The main challenge identified was that assessing end of life symptom management in the last month of life required too much data abstraction from the chart, contributing to an unacceptably long completion time of the tool (on average 73 min). Based on discussions between the auditors and research team, the period of symptom management assessment was changed to focus on the last week of life, while the remainder of the items on the tool continued to assess care provided in the last month of life.

The reliability of the chart audit tool was assessed through the evaluation of percentage agreement. This method was chosen since calculation of inter-rater reliability using Cohen’s kappa (Kappa) ratings is dependent upon the distribution of the data; when data are skewed, for example by raters selecting the same response categories, reliability measures are low [21]. The percentage agreement was calculated from the number of ratings with agreement on the 18 charts coded at mid-point. For each item on the instrument, a percentage agreement of 80% or greater was considered to indicate reliability. The average time to complete a chart audit was computed based on the total time required to extract data from each chart and the number of charts reviewed. The journal notes kept by the auditors and from the meetings with the research team were coded for major themes using content analysis and constant comparative techniques to assess the acceptability, utility, and ease of using the chart audit tool [22].