Background
In England, as in many other countries, there has been increasing emphasis on the importance of considering the impact of services on outcomes and quality of life in health and social care policy, practice and research [
1‐
4]. Despite the move towards community care, care homes remain one of the most expensive provisions of long-term care for frail older people in England [
5], with local authorities spending £4,960 million on care homes for adults over the age of 65 in 2013–14 [
5]. Ensuring that these services are providing good quality care is therefore high on the agenda of commissioners.
As part of an ambitious government agenda to change and improve adult social care, the 2014 Care Act [
6] places responsibility on local authorities to ensure the quality of services they are commissioning and promote the well-being of people using them. The corresponding guidance for local authorities [
7] specifically requires authorities to do this with reference to both the regulator’s minimum standards and the population outcomes outlined in the Adult Social Care Outcomes Framework (ASCOF) [
8]. This emphasises the parallels being drawn between well-being, as outlined in the Act, and the quality of life of people with care and support needs; the overarching indicator for which is social-care related quality of life (SCRQoL), as measured by the Adult Social Care Outcomes Toolkit (ASCOT) ([
8] p32).
Historically, local authorities have focused on monitoring the care provided to publicly-funded residents only. However, in a significant change of emphasis, local authorities now have a duty to facilitate local care markets to offer continuously improving, high quality services to all users, including those who fund their own care and do not rely on financial support from the local authority (self-funders) [
9,
10,
7]. This wider remit lends itself to a more inclusive, home-level view of quality monitoring and may mean local authorities take a closer look at homes who care predominantly for self-funding residents.
There is very little literature providing a national picture of English local authorities’ quality assurance systems and audits. Think Local Act Personal (TLAP), a national partnership of organisations focused on transforming health and social care through personalisation, surveyed local authorities to try and find out more about their systems and the way in which their quality data is used. However, they only received 12 responses, representing 22 authorities; 11 individual authorities and one consortium of a further 11 [
11]. Even with the consortium, given there are 152 local authorities with social services responsibilities in England, this represents only 14 % of local authorities. In total, seven reported conducting monitoring visits to assess care home quality, which included some observation of practice [
11] and five maintain an online directory of some kind to provide information to the public about the services they accredit. Beyond the information made available by the regulator, historically, relatively little information about the quality of individual care homes in England has been made available to the public (see for a review [
12]). Providers are generally considered opposed to the publication of individual provider performance data [
13]. Although some local authorities have used information gathered during quality audits to indicate ‘preferred providers’ and used their own ratings or those of the regulator to incentivise providers through ‘payments by results’ [
14,
15], this quality information is rarely shared with the public and so has typically not been able to support user choice.
There are an increasing number of ‘care ratings websites’ being provided by a variety of organisations, including the Social Care Institute for Excellence [
16] and a care home sector led voluntary compact [
17]. To aid choices about care and incentivise quality improvement through the provision of comparative information at provider level, the government launched a portal (
www.nhs.uk) [
2], which aims to draw together ‘high-quality’ information on the quality and effectiveness of individual care homes. Existing and planned quality marks, such as those of My Home Life [
18], the Dementia Care and Support Compact [
19] and NICE (National Institute for Clinical Excellence) quality standards [
20] are intended to be included, along with the views of service users and their families, and specific information from providers on issues such as falls, staff training and turnover, medication errors and pressure sores [
21].
A key source of information about the quality of health and social care is the regulator. Until 2010, star ratings (poor to excellent) were awarded to care homes by the then health and social care regulator, the Commission for Social Care inspection (CSCI). However, when CSCI was replaced by the Care Quality Commission (CQC), it withdrew the quality ratings and began a system of monitoring compliance against essential standards instead. Following the recommendations of reports into high profile abuse scandals [
22,
23], a review of the value of quality ratings [
24] and an independent review of how they carry out inspections [
25], CQC carried out consultations on a new strategy for inspection [
26]. This was followed six months later by another report, A Fresh Start [
27], which outlined the feedback from the consultations and the proposed changes to the regulatory approach to collecting evidence, conducting inspections and judging quality, including the re-introduction of a quality ratings system (outstanding, good, requires improvement, inadequate).
In line with the Nuffield review of the value of quality ratings [
24], CQC will now ask whether services are; safe, caring, effective, responsive and well-led and rated accordingly, with a view to; increasing accountability, aiding choice, improving performance, spotting failure, and reassuring the public [
2,
28,
24]. CQC will gather evidence relating to these key areas through ‘intelligent monitoring’ [
29], including evidence gathered by providers and others about the service. However, there is currently no way for providers, commissioners or researchers to reliably measure, evaluate and report quality of life outcomes at the provider/organisation level. As well as providing helpful information for CQC and addressing the emphasis placed on quality of life and well-being in the Care Act, recent research suggests such an indicator would be valued by the public, with relatives and carers identifying a measure of residents’ quality of life among their top three most useful indicators of care home quality [
30].
The adult social care outcomes toolkit (ASCOT) currently offers a multi-method approach to establishing outcomes for individual care home residents based on eight domains of social care related quality of life (SCRQoL) (
www.pssru.ac.uk/ascot). ASCOT was derived through a series of studies [
31] and to date is the only measure focusing specifically on the areas of quality of life that can reasonably be attributed to social care services. The domains cover the basic (personal cleanliness and comfort, accommodation cleanliness and comfort, food and drink, and feeling safe) and higher order (social participation, occupation and control over daily life) aspects of SCRQoL, and there is also a domain to measure how the way the care and support is delivered impacts on service user’s self-esteem (dignity). ASCOT includes a care homes tool (CH3) which uses a multi-method approach (observation and individual interviews) to score the social care related quality of life (SCRQoL) of individual care home residents based on these domains (
www.pssru.ac.uk/ascot).
The multi-method approach was developed due to the challenges of gathering self-report information from care home residents. Around two-thirds of care home residents in the UK have dementia [
32,
33] and engaging people with cognitive impairment through surveys poses many challenges [
34]. Observations have long been used as an ethnographic method of research in care homes [
35] and can be particularly helpful when researchers are interested in the daily routines and interpersonal communications of residents and staff [
36,
37]. As noted by Luff et al. [
35], “while self-report scales and questionnaires are traditionally ‘quick and easy’ forms of data collection, this may not be the case when working with people living in care homes” (p.25), owing to the high levels of physical and cognitive frailty [
38]. Furthermore, there is evidence that it is inappropriate to rely solely on the kind of self-report information collected through surveys/questionnaires when making judgements about the quality of a service [
39,
40,
31].
The ASCOT toolkit, and in particular the care home interview and observation instruments, are cited as data sources for local data collection about quality measures identified by NICE for care homes for older people [
41]. However, mixed-methods approaches to data collection are more time-consuming and resource intensive than self-completion surveys or interviews alone. This is justified and indeed appropriate when the goal is to measure the outcomes of individuals lacking the capacity to participate in other ways but does not lend itself to a ‘whole home’ approach. At the home level, individual ratings would need to be aggregated to either an average score for the home or reported at the domain level, indicating the distribution of outcomes in each. This kind of information is very sensitive to changes in the current population of residents, however, and as such may be better used by providers to profile residents and identify unmet needs and potential training issues. It is also a resource-intensive method of collecting data that relatively few would be able to undertake as part of their routine quality monitoring activities, let alone keep up to date. A home level (rather than individual level) measure may be able to fill the gap for a reliable outcomes-based approach to quality monitoring, assurance and improvement [
42], and if made available to the public has potential to help people compare and choose an individual care home. However, before a new measure is developed it is important that the purposes are clear and the measure viewed as useful to potential users.
This paper presents the results of an exploratory study that sought to develop a new measure of care home quality, based on residents’ quality of life outcomes. We describe the consultations with stakeholders, outline how these fed into the conceptual development of the measure and end by reviewing the feedback from the feasibility testing in one local authority quality monitoring team.
Methods
Three broad interlinked activities were carried out: consultations with stakeholders, tool development and feasibility testing with a quality monitoring team.
Key professional and lay stakeholders’ views were sought through a variety of methods: a one-day workshop, face-to-face interviews and focus groups. Each set out to identify views about potential use of the measure and associated methodological issues and to gather feedback on how the ASCOT domains and definitions might be adapted to work at a whole home level.
Professionals were invited to the workshop using opportunistic sampling of those who were already aware of and interested in ASCOT and had signed up to the ASCOT mailing list. This included; local authorities, care providers and their representatives, academics and voluntary organisations. We also purposively invited stakeholders that did not attend the workshop to take part in face-to-face meetings. During the workshop, small group consultation sessions, led by different members of the research team, were tape recorded and transcribed for later analysis. During the face-to-face meetings, participants did not wish to be tape recorded, so the research team took detailed notes instead.
Potential lay users of care home quality information were invited to take part in focus groups to complement the consultations with professionals. Ethical approval for this phase of the study was granted by the national Social Care Research Ethics Committee (SCREC) in June 2013. The lay groups aimed to include (1) relatives and carers of older people living in care homes, (2) relatives and carers of older people who have experience of social care services and support and (3) adults aged between 45 and 75 who may have to help arrange residential care for an older relative in the future, but as yet have no experience of choosing a care home. They were recruited through existing local groups in one local authority. Three local voluntary sector carer organisations agreed to help recruit groups 1 and 2. A University run database of members of the public willing to be research participants was accessed to recruit participants for group 3, however to address low response rates an advertisement was also posted on an online University website.
Following the consultation phase, the research team undertook a review of the findings and drew out the main messages for the development of the draft measure. Sometimes different stakeholders had different priorities and views of the proposed measure and these were discussed and reviewed on an iterative basis throughout the conceptual development phase. Using the ASCOT care homes toolkit as a starting point, and drawing on previous work undertaken for one local authority’s quality monitoring team, we began the conceptual and descriptive adaptation of the quality of life domain headings, descriptions and ratings system. We also drafted some provisional guidance and wrote training materials for the final phase.
In the final phase we explored the feasibility of quality monitoring (QM) officers using the new tool as part of their monitoring visits. This stage of the research was granted ethical approval from the national Social Care Research Ethics Committee (SCREC) in November 2013. We sent an email via the ASCOT mailing list asking for one local authority to pilot the draft measure in two homes for older people. Three local authorities expressed an interest and we recruited the one able to work within the time frames of the project. QM officers were trained to use the draft measure and then, working in pairs, they spent a day in each home collecting outcomes-focused data through: a 2–hour period of structured observation in communal areas (including lunch) and semi-structured interviews with staff, residents and family members, if available. The day after the visit, they each individually rated the home they had visited drawing on the evidence collected and the guidance and training we had given them. A face-to-face debrief meeting was held within a week of the visits to gather feedback on the data collection processes and ratings-system and explore whether they had disagreed about any of the ratings, and if so, why? We also explored the face-validity of the measure by asking the team to reflect on whether their final ratings gave an accurate depiction of their own views of the homes, drawing on their professional experience as quality monitoring officers.
All participants gave informed consent to participate in the research. Consent for the consultation phase was given verbally by professional stakeholders who voluntarily agreed to take part in interviews or attend the workshop. For everyone else, including the quality monitoring team piloting the draft toolkit, consent was given in writing.
Discussion
This paper has explored the demand for a new care home quality measure based on residents’ outcomes and presented early development and feasibility testing. Feedback from the consultations with stakeholders and preliminary testing in one local authority indicates that there is a use for this measure, especially in local authority quality monitoring teams, and that the ASCOT domains work at the care home level. However, it also highlighted the skilled nature of collecting data about quality of life through structured observations and interviews and the importance of thorough information gathering to inform ratings. The observational element of the draft toolkit worked well but training would be required to ensure a consistent approach. If used in quality monitoring, the interviews require further work to fit with the time constraints of those collecting the data and ensure adequate information is collected from a variety of sources. Alternative modes of engagement, including postal questionnaires for family members, should be explored.
The potential for this measure to aid user choice arose several times during the consultation phase and reflects a wider political and cultural shift towards encouraging the public to adopt a consumer-approach to long-term care [
53,
54]. Certainly previous research and the result of our own consultation indicates that the public would value a quality indicator based on residents’ outcomes [
30] but this raises the issue of who would be responsible for collecting the data and making it public. Professional stakeholders suggested Healthwatch might be best placed to fulfil this role and this might be something than individual authorities and local Healthwatch teams might explore in the future. However, for such information to be truly of use to prospective residents and their families, ratings would need to be available on all homes in their area. With around 10, 087 homes for older people in England [
55], this is not a small task and would require substantial resources, even with the use of trained volunteers. Furthermore, research evidence suggests the information may not be widely used by the public. The decision to move into a care home is often made at a time of crisis and constrained by the availability of places [
56] and greater weight is often given to ‘word of mouth’ or the reviews of people who know the services [
15,
57]. The previous regulator, the Commission for Social Care Inspection (CSCI), found that less than 1 % of social care users said they used the previous star ratings when making a decision about which home to move to [
15].
Our own consultations with members of the public suggested a preference for information about the quality of homes to be grounded in the views of residents and their families. This is in line with the recent increase in ‘care ratings websites’ [see 30 for a review]. On its website, Your Care Rating states that it aims to: give care home residents a voice; promote continuous quality improvement and provide an “authoritative source of information for existing and prospective customers” (
http://www.yourcarerating.org/about-us/). However, providers have to opt-in to the survey and in 2013 it was only sent out to 1, 123 homes, or approximately 11 % of care homes for older adults in England [
43]. Thus, it is unlikely that the data held on this website will be used to help prospective users find a home. Furthermore, although participants are asked about important aspects of their quality of life (e.g. having visitors when they want, having their own possessions around them, taking part in activities), unlike ASCOT, the survey does not measure the outcomes of social care, which was something the public said they would find relevant and helpful. It is this outcomes-focused approach that makes the measure different from anything else currently available, including the ratings made by the care regulator. Although CQC’s decision to re-introduce quality ratings goes beyond only inspecting compliance, it is notable that the Care Act guidance for local authorities suggests that they use the “definition that underpin the CQC’s fundamental standards of care as a minimum” (p.44) but consider the outcomes included in the ASCOF when promoting quality [
7].
Clearly a measure of care home quality based on ASCOT has the potential to be used in different ways by different stakeholders. However, without better information sharing between organisations, it might not be possible for the measure to do all of the things highlighted by stakeholders in this study. As Warmington [
58] notes, there needs to be a collective accountability for the quality of care and better information sharing is a key part of this. For example, a key tension for this measure is whether, and how, ratings are made available to the public to aid user choice. For good coverage, consistency and authority, local authorities appear best placed and most interested in using the toolkit to carry out ‘enter and view’ visits. However, feasibility testing indicated they may not have the resources required to collect the experiences of relatives and visitors, which the public seem to value particularly highly. Perhaps there is scope for partnership work between local Healthwatch and quality monitoring teams, with Healthwatch collecting information about relatives’ views of the CH4-HL domains and the local authority conducting the monitoring visits? As local authorities were reluctant to make their final ratings available, but there was a clear desire for such information from members of the public, perhaps there is scope for only the results of the potential survey of relatives to be made publicly available to inform user choice? Under the Care Act (2014) authorities are required to provide prospective users, including self-funders, with information about the homes in their area. Information such as this could be shared upon such enquiries being made, thus avoiding the need for relatives and frail older people to ‘data mine’ themselves for reliable, relevant and current information.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
AT – conceived of the study and new measure, led on design, ethical submissions and manuscript drafting. Contributed to data collection and analysis of workshops with professionals; trained QM team and gathered their feedback after piloting. JH – contributed to conception and design of the study, the ethical submissions and design, data collection and analysis of the workshops with professionals. She designed and carried out the focus groups, reviewed the literature and contributed to the development of the new measure and manuscript drafting. NS – contributed to the conception and design of the study and measure. Contributed to data collection and analysis of workshops with professionals; trained QM team and gathered their feedback after piloting. Contributed to revisions of manuscript drafts. TC - recruited participants for and assisted with running of two focus groups. Conducted the analysis of all three focus groups, wrote up the analysis and contributed to revisions of manuscript drafts. AN – contributed to conceptual development of new measure and rating levels, assisted with consultations with professionals and critically revised the content of the manuscript.EW - Contributed to the ethical submission for the focus groups and assisted with the design of research materials. Recruited focus group participants and assisted with the running of one group. Contributed to revisions of manuscript.GC – assisted with the collection of professional stakeholder views and contributed to revisions of manuscript. All authors read and approved the final manuscript.