Development and validation of a French questionnaire concerning patients’ perspectives of the quality of palliative care: the QUALI-PALLI-Patient

For the Qualli-Palli research project, a multi-professional workgroup (physicians, nurses, psychologists, social workers, and volunteers) conducted a literature review to identify research, indicators, and instruments that measure quality of life and quality of care for patients receiving palliative care. We selected a first set of 42 items exploring six domains, based on the major themes that emerged from the palliative literature, as well as from general purpose quality of care questionnaires, and from content analysis of our qualitative study [23, 25]. The six domains consisted of: the quality of information about health status and care (three items), availability of physicians and caregivers (10 items), attitudes of caregivers (six items), physical well-being and symptom management (10 items), functioning of the unit, including the handling of existential issues (nine items), and information and reception of relatives (four items) (Table 1). A four-point Likert scale was displayed after each question with the following ordered responses: no, not at all; not really; almost; yes, exactly, as well as a “does not apply to me” response (not applicable). The last (43rd) item assessed global satisfaction, according to four categories: not satisfied at all; partially satisfied; satisfied; very satisfied. This was followed by an open-ended question about any unmentioned aspects that seemed important to the patient.

The questionnaire needed to be brief, understandable, and easy to complete for hospitalized palliative-care patients and was designed to be self-administered or completed with informal assistance. This questionnaire was presented to 12 patients from three different hospitals to verify comprehensibility and acceptability of the items and response patterns. The patients were asked to complete the first version and to rate each item for clarity in wording, understanding, and relevance to the quality of care [26].

A cross-sectional study was conducted in the Paris metropolitan region in seven settings, encompassing three different care structures: two hospital-based palliative care units (Centre Hospitalier de Pontoise; Groupe Hospitalier Diaconesses Croix Saint-Simon, Paris), one palliative care hospital (Maison Médicale Jeanne Garnier; Paris), and four other standard medical units with mobile palliative care (Institut Curie, Paris, Hôpital Ambroise Paré, Boulogne, Hôpital Saint Antoine, Paris, Hôpital Saint Joseph, Paris). Eligible patients were consecutively recruited according to the following criteria: older than 18 years, hospitalized for palliative care for more than 72 h, a life expectancy under 3 months, and able to understand and communicate well in French and to respond to questionnaires.

One of two trained research assistants at each participating site communicated with the staff to identify potentially eligible patients and excluded those who were considered to be cognitively impaired. After assessing eligibility, the research assistant approached the patients, provided written and oral information about the study, invited them to participate, reassured them about confidentiality concerns and the independence of the study from the caregivers and the hospital, and received their consent. The research assistant gave the questionnaire to the patients informing them that they could take as much time as they needed to complete it or helped them to complete it if they were too tired. For validation purposes, patients completed the French version of the MDASI, which is a reliable assessment tool containing 13 items for the assessment of symptoms and six items to measure their global impact on their daily living and quality of life [27]. The completed questionnaires were directly transmitted to the coordinating investigator or addressed to him in a sealed envelope, without any interaction or supervision by professionals of the structure. Demographic (age, sex) and medical (pathology, stage) data were obtained from the caregivers who also informed us of changes in the health status of the patients (worsened, stable, improved) at the retest visit, relative to the beginning. The functional status of the patients was assessed using the French version 2 of the Palliative Performance Status (PPS) [28, 29].

In this multicenter prospective study, we created and validated a French questionnaire – Quali-Palli-Pat – that evaluates the quality of palliative care and the satisfaction with care from the patients’ perspective. We started with an initial questionnaire of 43 items. After analysis, the final questionnaire includes 35 items that explore the eight following domains: availability of caregivers, serenity, quality of information, pain management, willingness to listen to patients, psychosocial and spiritual aspects, right of refusal, and respect of the patients. The questionnaire concludes with a 36th question concerning general satisfaction. These eight dimensions, obtained after PCA, are in accordance with the six initially expected, but also include the theme of pain management and separate items which initially belonged to the relationship with doctors and caregivers. These results highlight major themes previously described in the palliative care literature and finally correspond to patient priorities, such as the availability of caregivers or the need for serenity [3, 14, 39‐42]. By exploring these eight dimensions, Quali-Palli-Pat differs from unidimensional satisfaction questionnaires, such as Famcare Pat [19, 43]. The stability of these domains was maintained across different sub-groups of patients (sex, age, PPS) and healthcare settings.

Our results indicate that global satisfaction is linked primarily to the global score of the MDASI symptom assessment scale and secondly to the interference of symptoms on the quality of life score. An important factor, such as the availability of caregivers, was linked both to the symptom score and interference score. However, Quali-Palli-Pat is neither a questionnaire of symptom assessment nor of quality of life. Satisfaction with care and quality of care are not interchangeable measures [44]. Our results indicate that all but one (right to refuse) of the explored dimensions positively correlated with the score of general satisfaction and probably directly contributed to it, as previously suggested [39, 41, 44, 45]. As quality of care may be some externally assessed measure, satisfaction with care deals not only with quality from a technical perspective but also with discrepancies between patients’ attempts and provision of care. [45]. Health status, quality of care, satisfaction with care, and quality of life appear to be in a dynamic loop [17]. We observed that a relative well-being (MDASI score) is associated to the satisfaction, related to obvious efforts of the caregivers mediated through interpersonal communication, as postulated by Stewart et al. [45]. However, according to the integrative response shift model proposed by Sprangers and Schwartz, a worsening of health status may modify the tolerance of the patient, and thus paradoxically maintain the level of the quality of life [46]. A similar process may have affected our measures of quality of care and satisfaction, found to be stable irrespective of the PPS, such as the recently reported independence between quality of life and ECOG performance status [37].

The strength of the method used to create and validate Quali-Palli-Pat makes it a theoretically sound and valid instrument, based on scoring of measurement properties as formalized in the COSMIN checklist (www.​cosmin.​nl/​), which includes internal consistency, reliability, measurement error, content validity, construct validity, criterion validity, and responsiveness [47, 48]. Quali-Palli-Pat meets most of these criteria. Except for reliability, measurement error, and responsiveness whose evaluation was hampered by the rapid deterioration of patients, all other criteria were excellent or good. Sample size and the methods appeared to be good both for dimensionality and internal consistency analysis, while also managing missing data. Concerning sample size, we had an initial sample-to-variable ratio of 5:1 and final variable-to-factor ratio of 4:1. Our sample size thus appeared to be adequate, considering the mean variable-to factor ratio and the relatively high value of the loadings (mean = 0.55) [49]. Content validity was built on a qualitative study among patients, and was confirmed by the easy interpretability of the final subscales. Construct validity was based on exploratory factor analysis and we verified the stability of the structure. Criterion validity mainly confirmed a priori hypotheses, facing global satisfaction or relation with the impact of the disease. Finally, the questionnaire was administered in various settings, to patients of various ages and physical status, thus enhancing the generalization of the study findings, although future validation should include home hospitalized patients.

We conceived Quali-Palli-patient as a diagnostic tool that provides a quality profile at the service level, highlighting domains which need improvement, rather than a unique and absolute score, which is difficult to interpret. Evaluation at the patient level is not appropriate, as evaluating and comparing individuals within a setting requires very high internal consistency, high precision and adjustment on patient status while there is the problem of the lack of independence of the patient from the caregiver. Our instrument may be used as a pre- and post-measurement tool in quality improvement interventions, as recommended by the e-satis program of the French National Authority for Health with periodic audits of satisfaction of hospitalized patients [50]. Periodically using this tool to compare various settings of palliative care could provide the means to better adapt public health policies and funding.

Quali-Palli-Pat largely explores the communication between patients and health providers. This is an extremely important domain, especially when deciding to stop specific treatments, such as chemotherapy, and at the beginning of exclusive palliative care. Teno et al. previously highlighted this domain and developed a specific tool focusing on information needs, especially at the time of diagnosis or making decisions concerning treatment [41]. In our study, this domain showed the lowest values, highlighting the necessity of further improvement, especially in palliative care units. A single item concerned spirituality (“I can talk to someone about philosophical or religious issues if I wish”) but with a rate of 27% of NA, despite its harmless formulation, this item would have been deleted according to our rules. However, it was retained by the working group because spirituality is an essential component of palliative care [51, 52]. The NA response can be explained by the specificity of French secularism, which considers religious or spiritual domains as private; French patients cannot understand the relationship between care, quality of care, and spirituality [53].

Our study had limitations. First, we used a sample of inpatients hospitalized in palliative care units or academic hospitals involved in quality of palliative care. This choice was probably not representative of the general population of palliative care inpatients, nor of the numerous settings tagged with palliative care services, ours being volunteers. This could explain the ceiling effect frequently observed and justifies retaining some items, despite this effect. The choice of retaining some items, despite their ceiling effect, should be confirmed in a future confirmatory study including a more diverse population of structures and patients.

Second, reproducibility could not be assessed due to the rapid evolution of the health status of the patients. Testing reliability requires a test/retest in which patients remain stable for a reasonable period, perhaps a week [1]. This short timeframe is far too long for hospitalized palliative care patients, because the median overall survival in palliative care units is approximately 12 days, during which patients quickly and inevitably get worse. Thus, it was not possible to test reliability and sensitivity to change, as in other studies [1]. Quali-Palli-Pat, like all instruments of this kind, can only be used by fully communicative patients, which does not reflect the palliative care population, especially at the end of life [54]. Thus, we plan to create and validate the Quali-Palli-Fam questionnaire, intended for relatives. We did not estimate the relative weight of each of the eight domains, as tested in other settings [42]. We believe that it would be too difficult for terminally ill tired patients to disentangle the two related questions of perceived quality and subjective importance, item by item [43]. This could eventually be done in a subsequent confirmatory survey, which could also verify the existence of these eight domains in other contexts.

Although the questionnaire was created to be as brief, understandable, and easy as possible, 80% of patients required assistance to complete it. Response bias was reduced by the fact that the interviewer was a clinical research associate, independent from the caregivers of the hospitals [55]. This raises the question of who will assist in the completion of the questionnaire for periodic assessments. In our recent experience, it is possible to involve volunteers. However, some patients with delirium or vigilance disorders will never complete it, explaining the lack of patients with brain tumors in our data. The Quali-Palli-Fam, which explores the family perspective, should at least partially compensate for this.