Do Rural Residents in China Understand EQ-5D-5L as Intended? Evidence From a Qualitative Study

Four regions (North, South, East and West) across China were selected on the basis of geographic location involving the cities of Tianjin (municipality), Guangzhou (Guangdong province), Nanjing (Jiangsu province) and Guiyang (Guizhou province) and then one county (rural area) from each region was selected for data collection. Participants from each county were recruited through convenience sampling. Rural residents who had lived in the county for the last 3 years and made a living by agricultural operations were eligible.

To obtain a representative sample, we used the national figures on the population distribution [20] and sought advice from an expert panel. Healthcare researchers from four universities in the four cities (Tianjin, Guangzhou, Nanjing and Guiyang) with sufficient survey experience in rural China were invited by the lead authors via personal contact and a total of 15 members were included in the expert panel. Teleconferences were held to achieve consensus between members. It was agreed that age, sex and education level should be taken into account, but not socioeconomic status considering that the survey was self-reported and the responses to socioeconomic status may not be reliable. Additionally, we did not include disease status as there is no clear evidence showing that the disease status could affect respondents’ understanding of the EQ-5D-5L and chronic disease is usually under-diagnosed among rural Chinese, so respondents may not be able to report accurate information. Disease status was not considered in quota sampling in the previous EQ-5D-5L valuation studies such as in the US [24] and in China [25].

In line with the recommendation for qualitative research that sample sizes of between 5 and 15 are typical [26], the target sample size for each county was 15. A total of eight interviewers were involved in the data collection, with two in each county. They were the junior members of the expert panel who were postgraduate students with experience in conducting surveys. Before the data collection, the interviewers received training from the team leaders and conducted pilot interviews. Training included briefing on the questionnaire, addressing queries and interviewers conducting pilot interviews with others. The feedback on the interviewing process was collected and then used to improve the interviews in the main data collection. Data were collected from September to October 2018 and involved semi-structured, one-on-one, face-to-face interviews, administered by trained interviewers using a paper-and-pencil questionnaire. The interviews were conducted at the participant’s home or other locations that allowed quietness and privacy. Participants provided written informed consent prior to the interview. During data collection, all participants were allocated with one study ID and their responses to the questions were recorded with this study ID only. No personal information was collected to protect privacy and we recorded the participants’ age as age group (18–35 years/36–55 years/56 years and above). Ethical approval was obtained by the Safety and Ethics Committee of the School of Pharmaceutical Science and Technology (SPST2018-03) in Tianjin University.

The survey instrument consisted of two tasks. The first task presented a cognitive debriefing procedure [27] to elicit participants’ feedback on the content of the EQ-5D-5L. In cognitive debriefing, each domain was assessed in terms of comprehensibility, relevance, clarity and comprehensiveness.

Comprehensibility was evaluated by examining whether the questions could be understood by participants as intended. A series of questions were asked about their interpretations of the questions, e.g., “could you give an example of usual activities”. For the two composite domains (pain/discomfort, anxiety/depression), one previous study reported the significant effect of splitting the anxiety/depression domain on self-reported health and health states valuation [28], and therefore, in this study, participants were asked to explain their interpretations of both terms and we also asked them whether there were differences between the two terms.

Relevance was assessed by examining whether the domains of EQ-5D were health-related and whether participants’ response could reflect their health condition using the questions “is the domain relevant to your health” and “do you choose the response level based on your health”.

Comprehensiveness was assessed by examining whether there were important aspects of health not covered by the five domains of EQ-5D-5L using an open-ended question. The five domains in EQ-5D were developed based on European populations, so it may not be able to fully capture what matters for the Chinese population, especially for rural residents.

In the first task, participants were asked to choose a response level to each domain of the EQ-5D-5L according to their own health condition before and after debriefing, to check whether the debriefing procedure helped them in understanding and answering the questions. The second survey task required participants to select the most and least important domains related to their health among the five EQ-5D-5L domains. This task could also reflect their understanding of the concepts. Details of the instrument are available in the Appendix (see electronic supplementary material).

All interviews were audio-recorded and transcribed, and then the transcripts were analysed using QSR NVivo 12 software following the content analysis framework [29].

A total of sixty-two participants were recruited, with 27.4% aged 18–35 years, 51.6% aged 36–55 years and 21.0% aged 56 years and above. Among all participants, 51.6% were female; 25.8% participants had primary education or lower, 50.0% had junior secondary education and 24.2% had senior secondary education or higher (Table 1). Participants from different regions were slightly different in terms of age, gender and education level (Table 1).

Before the debriefing task, the mean (standard deviation) EQ-5D-5L score was 0.939 (0.088) using the Chinese EQ-5D-5L value set [25] (Table 1 and Fig. 1a). Participants from different regions reported slightly different pre-debriefing EQ-5D-5L scores with people from the North reporting the highest while those from the West reported the lowest (North: 0.981; South: 0.948; East: 0.925 and West: 0.897) (Table 1 and Fig. 1c).

The debriefing task took 38.8 min to complete on average and participants from the West spent the longest time (North: 32.6; South: 32.4; East: 41.0 and West: 49.9) (Table 1). After debriefing, five (8%) participants (4 from the West and 1 from the East) changed their responses, and the changes were in both directions of the response level. Two changed their response for ‘anxiety/depression’ from ‘no’ to ‘slight’ (Fig. 2a), one changed the response for ‘self-care’ from ‘no’ to ‘slight’ and the response for ‘pain/discomfort’ from ‘moderate’ to ‘slight’ (Fig. 2b), one changed the responses for ‘anxiety/depression’ from ‘severe’ to ‘no’ (Fig. 2c) and one changed the response for ‘mobility’ from ‘no’ to ‘slight’ and the response for ‘anxiety/depression’ from ‘no’ to ‘slight’ (Fig. 2d). After debriefing, the overall EQ-5D-5L score was identical but the score of participants from the East dropped from 0.925 to 0.918 while that of patients from the West increased from 0.897 to 0.907 (Table 1).

For the ‘mobility’, ‘self-care’ and ‘usual activities’ domains, all participants reported that they could understand the domains well by giving their interpretations.

“Mobility is the physical activity, such as do house work and go to the farmland.” (18–35 years, male).

“Self-care means that as an adult, I could handle my own life well, not like a child.” (18–35 years, female).

“Usual activities include going to community centres to meet friends, grocery shopping in markets, etc.” (36–55 years, female).

For the ‘pain/discomfort’ domain, all participants could give their interpretations, but when asked the differences between the two terms, forty-two (67.7%) participants indicated that although the two terms shared similar meaning, there were differences between ‘pain’ (Chinese: 疼痛) and ‘discomfort’ (Chinese: 不舒服). These differences were mainly related to the severity level and the range that each term covers.

“They are different. If you have pain, you also have discomfort, but discomfort does not necessarily relate to pain, for example, it can be sore waist or bloating.” (36–55 years, female).

“Discomfort can also be emotional, for example, unhappy, but without pain.” (18–35 years, male).

“Pain relates to physical feelings, but discomfort covers a wider range, including emotions.” (36–55 years, male).

For the ‘anxiety/depression’ domain, six (10.0%) participants could not understand ‘anxiety’ (Chinese: 焦虑) and nine (14.5%) participants mentioned they did not understand ‘depression’ (Chinese: 沮丧). This was mainly due to the wording used being too formal for them to understand. Two (3.2%) participants suggested some revisions to make the question more straightforward, such as using the terms ‘unhappiness’ or ‘sadness’ (Chinese: 不开心/难过).

“Just ask it simply, like whether you are happy or not.” (36–55 years, female).

“I like the question asking whether I am in a good mood rather than anxious or depressed.” (18–35 years, male).

Similar to the ‘pain/discomfort’ domain, thirty-five (56.5%) participants reported differences between ‘anxiety’ and ‘depression’. These differences may confuse them when trying to select the appropriate response level.

“They (anxiety and depression) are different. Anxiety means there is something that puzzles you and you could not stop thinking about it. Depression is in a bad mood, similar to sadness.” (18–35 years, male).

“Anxiety is worry, mainly emotionally, but depression means lack of vitality.” (18–35 years, female).

All participants reported that the five domains were relevant to their health and they chose the response level based on their own health, supporting the use of EQ-5D-5L to measure health status.

“Yes, the domains are related to my health and I can understand them. I chose my answer based on my health.” (18–35 years, male).

When asked whether they could distinguish the five response levels, all participants gave positive answers. When asked to recall the levels, thirty-eight (61.3%) participants could not recall all levels and the majority of them could only recall four levels.

“I only remember some of the items, not all.” (18–35 years, female).

“Eh, no problems at all, moderate problems, severe problems, like these.” (36–55 years, female).

“No problems, severe problems, moderate problems. Could not recall others.” (56 years and above, male).

Five (8.1%) participants suggested reducing the response levels to three or four.

“Three should represent almost all the conditions, that is, no, some and severe.” (18–35 years, male).

“I like four categories, no, some, moderate and severe.” (36–55 years, female).

Furthermore, three (4.8%) participants suggested that the Chinese wording of ‘some’ (一些) should be used instead of ‘only a little’ (一点) because ‘only a little’ in Chinese could not accurately describe the health state between ‘no problem’ and ‘moderate’.

“I always have only slight problems, such as joint pain, but I report no problems. The ‘only a little’ seems useless to me. If my pain gets worse, I think it would be ‘some’ problems, but not as severe as ‘moderate’.” (36–55 years, male).

“I don’t understand ‘only a little’ and we don’t use it here (the region), ‘some’ may be better.” (36–55 years, female).

When asked about aspects of health not covered by the five domains of EQ-5D, fatigue and appetite were raised by some participants (n = 4, 6.5%).

“Fatigue should be included. If I feel fatigue, I could not do the daily activities as usual.” (56 years and above, male).

“Appetite is important for my health.” (56 years and above, female).

In the task of relative importance assessment, all participants provided their answers to the question of the most important domain but two participants did not answer the question of the least important one. The item ‘mobility’ was selected as the most important by 24 (38.7%) participants while ‘anxiety/depression’ was selected as the least important by 37 (59.7%) participants (Table 2).