Background
Improving the quality of medical care is a high priority in Japan, as well as in other countries. There are many tools worldwide for evaluating the quality of primary care, including the Components of Primary Care Index [
1], Primary Care Assessment Survey (PCAS) [
2,
3], Primary Care Assessment Tool (PCAT) [
4], EUROPEP questionnaire [
5], General Practice Assessment Survey (GPAS) [
6,
7], and General Practice Assessment Questionnaire (GPAQ) [
8]. Among these tools, the GPAQ, was originally developed in the United States as the PCAS; then, in collaboration with the Health Institute in Boston and the National Primary Care Research and Development Centre at the University of Manchester, in the United Kingdom, it was modified for use in British general practice and became the GPAS, which has been widely used [
3,
6,
7,
9]. A brief version of the GPAS (the GPAQ) was also used for increased accessibility to patients and with a revised scoring system [
8]. The GPAQ has been broadly used as a quality survey for general practice and has been translated into and validated in many languages, including Chinese, Somali, and Thai [
10‐
12]. The goal of this questionnaire is to evaluate various aspects of care quality, such as receptionists’ demeanour, access to primary care, continuity of care, physicians’ communication skills, and enablement (understanding of self-care and of one’s own health after consultation).
It has been noted that effective communication skills among physicians are critical for patients to develop trust in them [
13]. Factors such as maintaining confidentiality, respecting patients’ dignity, and involving them in treatment decisions are also highly valued [
14]. In addition, systematic patient feedback about physicians’ interpersonal skills has been shown to help improve physicians’ interpersonal performance [
14,
15]. Thus, the GPAQ is used to assess how patients feel about their primary care and how they cope with their problems or illnesses during and after consultations, making it an enablement scale [
16].
In Japan, however, no study has used the GPAQ in primary care; instead, a Japanese version of the PCAT (JPCAT) has been developed and used [
17]. Previous studies using the JPCAT have reported that patients’ experience in primary care affects their health-associated behaviour or ability in Japan. Patient experience of primary care was positively related to breast cancer screening uptake in Japanese women [
18], while longitudinality and comprehensiveness of primary care service were related to higher health literacy [
19]. On the other hand, Japanese family physicians (FPs) and general practitioners (GPs) require additional education. An Organisation for Economic Co-operation and Development (OECD) review noted that Japanese primary care has been provided by a cadre of semi-generalists/semi-specialists, those who were working as generalists in the community with an unspecified amount of time to train as generalists (no compulsory further training) after they had left hospital practice [
20]. Therefore, the former Japanese Academy of Family Medicine (JAFM) (currently the Japan Primary Care Association (JPCA)) started the certification system of a residency training programme for FPs in 2006 [
21]. Moreover, The Japanese Medical Specialty Board established primary care as a distinct new medical specialty beginning in 2018 [
22]. Patients in European countries are generally provided consultation by GPs who completed specialist training in family medicine, and some of those countries have a system for registering patients for primary care [
23]. A registration system with a primary care practitioner is suggested to possibly bring more effective primary care such as providing lifestyle advise, coordinating an individual’s care, and strengthening the relationship between patients and their primary care physicians [
23]. There is no such registration system in Japan; however, Japan’s medical system is gradually beginning to recognise primary care physicians as medical specialists. For example the Japanese Medical Specialty Board recognised family medicine as a new specialty in 2017 [
22]. The GPAQ is a tool to assess physicians’ interpersonal skills; thus, we expected its results to be useful not only for improving the quality of care but for educating new FPs/GPs.
Compared with the PCAT, the GPAQ features a more detailed evaluation of practitioners’ interpersonal and communication skills in relation to patients. To take advantage of this affordance, we focused on the development and validation of a Japanese version of the tool (GPAQ-J). To verify the robustness of the GPAQ-J, we also referred to the results of the JPCAT, an established primary care assessment tool.
Thus, the purpose of the current study was to develop the GPAQ-J, a self-administered questionnaire, from the original GPAQ, using a formatted translation process, and to conduct a full validation survey. We established the utility of this new tool through statistical analyses. We also analysed responses to questionnaire items on physicians’ interpersonal skills in their practice. The results elucidate the characteristics of Japanese primary care and show the differences between this new tool and an existing alternative—the JPCAT—for the Japanese context.
Results
We developed the GPAQ-J through the seven steps described above. Patients (N = 252) responded to the GPAQ-J (approximately 50 participants per clinic). Respondents were Japanese patients who had visited any of the five target primary care centres from May to July 2017. These respondents accounted for approximately 6.7 ± 2.1% of outpatients aged ≥20 years in each clinic. Patients (N = 234) also responded to the JPCAT at the same clinics during the same period; however, 19 did not identify a primary care provider. Therefore, 215 responses were included in the final analyses.
The process of the expert review and pilot test are in Additional file
1. The item ‘Waiting time at the practice for each consultation’ required revision in Steps 2 and 3. In addition, items about patients wanting to see a particular physician/any physician and about urgent consultations were removed in Step 2. Additionally, we added a question in Step 2, and revised it in Steps 3 and 4, about telephone management by physicians and nurses; because, in Japan, when patients telephone for medical advice, it is often nurses who handle the call. In Step 2, for cultural reasons, questions about ethnicity were removed, as there are few sizable ethnic groups in Japan. Ultimately, the GPAQ-J was finalized (see Additional file
2, content in Japanese). The main procedure of this study was not modified during the research.
Across all clinics, 18 physicians (5.1 ± 2.4 physicians per clinic) were evaluated for consultation skills. We found no floor or ceiling effects for the GPAQ-J scales; mild ceiling effects were found for some individual items on the GPAQ-J, and moderate ones on the JPCAT (see Additional files
3 and
4). Three-fourths of patients reported having seen a physician seven or more times in the previous 12 months. Regarding physicians’ communication skills, 84% of patients responded to the item ‘How well the physician listened to what patients had to say’ with ‘very well’ (for other items see Additional file
3).
Table
1 shows respondents’ demographic characteristics. All respondents who agreed to participate answered the questionnaire.
Table 1
Patients’ characteristics (N = 252)
Age (years) |
Mean ± SD | 68.3 ± 12.3 | |
Range (years) | 28–95 | |
n | 230 | |
Sex |
Male | 99 | |
Female | 132 | |
n | 231 | |
Number of primary care visits |
Once/year | 1 | 0.4 |
2–6 times/year | 59 | 24.5 |
7–12 times/year | 121 | 50.2 |
13 or more times/year | 60 | 24.9 |
n | 241 | |
Chronic illnessa |
Yes | 134 | 68.7 |
No | 61 | 31.3 |
n | 195 | |
Occupation |
Employee/public servant | 28 | 12.6 |
Employer | 33 | 14.9 |
Student | 0 | 0 |
Part-time worker | 30 | 13.5 |
Unemployed | 102 | 45.9 |
Other | 29 | 13.1 |
n | 222 | |
Table
2 shows the central tendency, dispersion, and mean scores for all scales and patients’ overall satisfaction with the GPAQ-J.
Table 2
General Practice Assessment Score in five primary care centres using the GPAQ-J (N = 252)
Receptionists | 1 | 74.9 | 20.1 | 60 | 80 | 80 | 1.6 | 20.9 | 0–100 |
Access | 5 | 64.9 | 16.1 | 55 | 66.7 | 76 | 0 | 3.7 | 20–100 |
Continuity of care | 1 | 67.5 | 21.9 | 60 | 80 | 80 | 1.0 | 13.6 | 0–100 |
Communication | 8 | 81.6 | 13.4 | 74.5 | 80 | 95 | 0 | 15.6 | 45–100 |
Enablement | 2 | 65.8 | 22.7 | 50 | 66.6 | 83.3 | 0 | 19.5 | 33.3–100 |
Table
3 shows the results of validity and reliability testing of the GPAQ-J. All item-scale correlations exceeded 0.6, and all scales demonstrated a narrow range of item-scale correlations. The reliability test was implemented to confirm item consistency for access to care, communication, and enablement. Cronbach’s alpha for all scales was > 0.70. Inter-scale correlations were lower than each value of Cronbach’s alpha. Item-other scales correlations were significantly lower than item-own hypothesised scale correlations for access, communication, and enablement (detailed statistics in Additional file
5). The scaling success rate was 100% for all scales. Moreover, all scales except continuity of care were significantly correlated with overall satisfaction using Spearman’s correlation coefficient (
p < .001).
Table 3
Results of validity testing and internal consistency from GPAQ-J (N = 252)
Receptionists | 1 | NA | 0.26–0.53 | NA | 0.26–0.54 | NA | 0.38 |
Access | 5 | 0.66–0.81 | 0.15–0.51 | 25/25 (100) | 0.37–0.54 | 0.79 | 0.38 |
Continuity of care | 1 | NA | 0.28–0.45 | NA | 0.28–0.45 | NA | 0.17 |
Communication | 8 | 0.77–0.87 | 0.23–0.45 | 40/40 (100) | 0.36–0.51 | 0.92 | 0.37 |
Enablement | 2 | 0.92–0.93 | 0.24–0.41 | 10/10 (100) | 0.26–0.41 | 0.83 | 0.31 |
Total | 17 | | | | 0.9 | |
Table
4 shows the results of validity and internal reliability testing of the JPCAT. Cronbach’s alpha was > 0.70 for all scales, item-scale correlations were > 0.3 for all scales, and item-other scales correlations were significantly lower than item-own hypothesised scale correlations, except for the community orientation scale. Longitudinality, comprehensiveness (services available), and community orientation were significantly correlated with overall satisfaction.
Table 4
Results of validity testing and internal consistency from JPCAT (N = 215)
First contact (Access) | 3 | 0.62–0.92 | 0.009–0.34 | 0.78 | 0.22 |
Longitudinality | 5 | 0.74–0.88 | 0.09–0.57 | 0.86 | 0.38 |
Coordination | 6 | 0.73–0.86 | 0.09–0.52 | 0.89 | 0.23 |
Comprehensiveness (services available) | 5 | 0.81–0.89 | 0.03–0.57 | 0.91 | 0.40 |
Comprehensiveness (services provided) | 5 | 0.58–0.82 | −0.04–0.40 | 0.81 | 0.26 |
Community orientation | 5 | 0.50–0.81 | −0.11–0.45 | 0.78 | 0.44 |
Total | 28 | | | 0.90 | 0.47 |
Table
5 shows the principal component analysis using a rotated factor matrix. Analysis of the scree plot identified three factors, which accounted for 62.4% of the variance. Item loadings > 0.3 on a factor are considered substantive and are shown in bold in Table
5. Component 1 included only the communication scale, which explained 31.1%. Component 2 included the receptionists scale, access scale, and continuity of care scale, which explained 20.7%. Component 3 included only the enablement scale, which explained 10.6%. In component 2, the continuity of care scale was weakly correlated with access (Spearman’s correlation coefficient: 0.29); the access scale was strongly correlated with the receptionists scale (Spearman’s correlation coefficient: 0.53).
Table 5
Rotated factor matrix of the analysis of the GPAQ-J
2 | Receptionists | 0.179 |
0.633
| 0.081 |
3a | Availability of surgery (any GP) | 0.247 |
0.529
| −0.008 |
4b | Waiting times at surgery | 0.108 |
0.665
| −0.044 |
5a | Quality of phone call (length of time, etc.) | 0.137 |
0.775
| 0.175 |
5b | Availability of doctors on the phone | 0.245 |
0.702
| 0.245 |
5c | Doctor’s or nurse’s phone support | 0.139 |
0.795
| 0.127 |
6b | Continuity of care | 0.223 |
0.495
| 0.136 |
7a | GP questioning |
0.706
| 0.202 | 0.121 |
7b | GP attention |
0.832
| 0.167 | 0.109 |
7c | GP putting you at ease |
0.817
| 0.291 | 0.032 |
7d | GP involving you in decisions |
0.84
| 0.188 | 0.179 |
7e | GP explanations |
0.794
| 0.194 | 0.209 |
7f | GP spending time with you |
0.631
| 0.263 | 0.078 |
7 g | GP patience |
0.848
| 0.198 | 0.134 |
7 h | GP caring and concern |
0.779
| 0.154 | 0.152 |
8a | Able to understand your problem | 0.266 | 0.196 |
0.857
|
8b | Able to keep yourself healthy | 0.192 | 0.15 |
0.885
|
Table
6 shows inter-country comparisons between our data and GPAQ results for Thailand, New Zealand, Australia, and the UK [
10,
43,
44]. There are a few differences in content and in GPAQ format across the studies, such as a consultation version (administered in the primary care facility) or postal version (administered by mail). Results showed that mean scores from our study were higher than Thailand and the UK, and lower than New Zealand and Australia.
Table 6
Inter-country comparison (mean score)
Receptionists | 74.9 | 63.5 | 85.7 | 81.8 | 69.1 |
Access | 64.9 | 55.1 | 76.6 | 68.6 | 58.3 |
Continuity of care | 67.5 | 65.8 | 80.4 | 76.5 | 66.1 |
Communication | 81.6 | 69.0 | 89.1 | 84 | 75.9 |
Enablement | 65.8 | 80.8 | NA | NA | NA |
Overall satisfaction | 80.4 | 80.6 | 86.3 | 81.6 | 76.5 |
Moreover, we examined the correlations between each item and overall satisfaction. In the GPAQ-J, items ‘Does the doctor care and show concern for you?’ ‘How about the service of the receptionist?’ ‘Did the doctor explain your problems and treatment sufficiently?’ and ‘Did the doctor listen to your questions or worries with patience?’ were significantly correlated with overall satisfaction (Spearman’s correlation coefficients were 0.40, 0.38, 0.37, and 0.36, respectively; all ps < .001).
Discussion
The GPAQ-J demonstrated good reliability and validity, which confirms that it is an acceptable instrument to evaluate primary care in Japan. We analysed the data based on the original version of the GPAQ, and we did not amend the main scales. Based on the original scale classification, we analysed reliability and validity, and applied principal component analysis. Since the GPAQ has been widely used since 2004 [
8,
11,
45], we did not modify its basic structure. In addition, during the development of other versions of the GPAQ, factor analysis was not conducted as part of the initial analysis [
8,
10]. Therefore, we conducted factor analysis to confirm whether those categories of GPAQ-J such as receptionists, access, continuity of care, communication, and enablement were compatible with our data to reconfirm robustness.
As for the researchers’ role, two researchers were included in the expert review in Step 5. One author was not a principal researcher but took part in the qualitative review in Step 2. The principal researcher contacted all members and compiled their comments in Step 2. One of the authors worked at one of the primary care centres, and another had previously worked there. To avoid participant or experimenter bias, no researcher was a research assistant. The principal researchers analysed all data, including detailed factor analysis, which might cause potential bias. However, that approach was taken based on previous GPAQ studies [
8,
11].
Almost all researchers had worked in rural areas, and thus recognized the need to evaluate the quality of primary care in rural areas. The proportion of elderly people in rural areas in Japan is increasing, so the results might potentially be particularly important for eldercare—a growing concern in Japan as the population ages. The study population was mainly elderly people, which affected participant recruitment. In fact, the elderly population in the areas we surveyed was higher than the overall Japanese elderly population (32.7% vs. 25.1%) [
46,
47].
For selection of respondents, staff who were currently working or had previously worked in each facility selected patients they knew in that facility to complete the questionnaire. Since staff selected patients, selection bias is a possible limitation. Subsequently, research assistants explained that collected questionnaires would be managed and analysed outside the facility, to maintain confidentiality; they also conducted the informed consent process. Questionnaires were collected by volunteers after being posted by patients into a box placed in the facility. To avoid the possibility that some patients might feel nervous or constrained in their comments because the informed consent sheet included the names of researchers who had worked with some of them before, the questionnaire did not gather any personal information such as age or sex.
The GPAQ-J demonstrated good reliability and validity, which indicates that it is an acceptable instrument to evaluate primary care in Japan. Owing to differences between medical systems, it was necessary to customise several items from the original GPAQ to the GPAQ-J. For instance, in Japan, it is common for patients to visit a primary care facility without making an appointment. Several physician consultations per person have been reported in Japan compared to other countries by the OECD (Japan, 12.7/year; OECD average, 6.9/year) [
48]. In Japan, patients can consult a primary care physician on a work day with ease. This can be both positive and negative; on the one hand, it provides access for patients who urgently want to consult a physician; on the other hand, it is likely that there will be a longer wait time in the facility even for patients with scheduled appointments. Therefore, the item ‘Waiting time at the practice for each consultation’ required repeated discussion and revision. Further, items about patients wanting to see a particular physician/any physician or receive urgent consultation were removed because of the above characteristic of the Japanese medical system.
Although we revised some items, the GPAQ-J retained a Cronbach’s alpha of 0.90. Our principal component analysis showed comparable results to previous research in the UK [
8]. Regarding convergent validity, item-total correlations were found to be high (> 0.6) for access, physicians’ communication skills, and enablement. Discriminant validity was demonstrated by the scaling assumption, whereby the correlation of an item with its own hypothesised scale was significantly greater than that with other scales.
Table
5 shows the results for the three major components. A previous study indicated the same categories [
8], supporting the compatibility of our scale categories with the data. Factor analysis is used to identify externally meaningful dimensions or tendencies by exploring variables with high factor loading [
49]. These were categorised into three dimensions: physicians’ communication skills; demeanour of receptionists, access, and continuity of care; and enablement. The Physicians’ communication skills component had the highest contribution rate in the questionnaire, suggesting that the GPAQ-J is adequate to evaluate physicians’ communication performance. Receptionists were included in a composite component with the access and continuity of care scales, similar to a previous study [
8]. Access was strongly correlated with the demeanour of receptionists, which implies that receptionists’ service was a more critical component related to access to primary care facilities than was continuity of care with one physician.
The results for the GPAQ-J were similar to those for the JPCAT regarding total Cronbach’s alpha, construct validity, overall satisfaction, and correlations between each scale and patients’ overall satisfaction. As the questionnaires were not completed by the same participants, these results do not directly validate the GPAQ-J; however, the questionnaires were administered to similar respondents (patients in the same primary care centres during the same period), which indicates external validity with the existing tool, the JPCAT—although the GPAQ-J and JPCAT have different items and scales. Therefore, it is difficult to conduct an exact comparison. On the GPAQ-J, receptionists’ demeanour, access, and physicians’ communication skills were positively related to overall satisfaction, as were, on the JPCAT, community orientation, comprehensiveness, and longitudinality. Although the JPCAT involves a comprehensive primary care evaluation, some items about communication, showing a correlation with overall satisfaction, are included only in the GPAQ-J. For the communication scale, Cronbach’s alpha was high, and item-scale correlations were narrow. The GPAQ-J can, therefore, be considered an appropriate and useful tool for evaluating physicians’ communication skills. Potentially, it could also be applied to evaluate communication training of medical students or trainees or to survey Japanese patients’ satisfaction with their physicians’ communication skills.
For inter-country comparison, we compared our data with those from Thailand, New Zealand, Australia, and the UK [
10,
43,
44]. Almost every scale in our data was highly rated compared with Thailand and the UK, but lower-rated than New Zealand and Australia. Our sample and data included older patients who were noted to rate highly receptionists, access, continuity of care, and overall satisfaction compared to younger people [
43]. Regarding the difference in access between Japan and Thailand, Thailand data were investigated in a hospital; hospital respondents possibly affected the lower access score compared with primary care facility respondents in Japan. UK surveys were conducted by mail (postal version) which may account for the differences compared to the Thailand and UK data. As for Australia and New Zealand, sample size and the history of family medicine in the respective countries may influence differences in factors such as continuity of care.
Our data were mostly within predictions, except on two points. First, the continuity of care scale was less related to overall satisfaction than the research team expected. We discussed above that the continuity of care scale was based only on evaluation of the consultation with a main physician; in contrast, the continuity of care concept involves understanding the whole patient, including detailed medical history, medication, job, and family condition. Although items covering these factors were included in the communication scale, the continuity of care scale needs to be revised to address them. Second, patients’ overall satisfaction was very high. A previous GPAQ study reported that older respondents tended to express greater satisfaction with their care than did younger ones [
43]; therefore, the older population may have influenced the high score.
The study did, however, have some limitations. First, we used only a single back translator. Although some previous studies showed appropriate results with one back translator [
10,
17,
24], it may nevertheless be insufficient for comparing the original version with our Japanese version. Second, the GPAQ-J was administered to patients in the waiting room, just after consulting a physician. Immediate evaluation is advantageous to elicit fresh assessments without risk of forgetting; however, it could cause respondents to overestimate satisfaction. In our study, however, overall satisfaction on the GPAQ-J, which included demographic data, was lower than on the JPCAT (without demographic data). Although the questionnaires have some differences, this result suggests that the GPAQ-J did not overestimate satisfaction. Third, concerning comparing the JPCAT and GPAQ-J, some patients responded to both questionnaires sequentially on the same day, some patients on different days, and other patients responded to only one questionnaire. In sequential administrations, the time commitment may increase response burden and lower response rate [
50]. Fourth, this survey focused on rural areas, and had fewer participants than in previous GPAQ studies [
8,
44]. However, factor loadings in the principal component analysis sufficiently identified items in three components, and the coefficients between scales and overall satisfaction also indicated substantial differences; hence, the sample size was sufficient for our statistical analyses. Fifth, the communication skills of only a limited number of physicians were evaluated; thus, it is possible that there was evaluation bias. Finally, this study was limited to a small percentage of patients who visited one of the five targeted family medicine training centres and were selected mainly by staff working at the facility. There may have been biases at play towards respondents who communicated well with staff and had good feelings about the facility; that is, we may have under-sampled patients who were not satisfied. Further research in a variety of areas and facilities, with larger numbers of physicians, and using a postal method are needed to generalise the results to Japanese primary care.
Acknowledgements
We thank all the patients who completed the questionnaire and all primary care centre staff who cooperated in data collection. This project relied on contributions from collaborators and interviewers in Shimane and Okayama prefectures. We are particularly grateful to the following people: Prof. Eisaku Taniguchi, Prof. Sachiko Hara, Prof. Jun Iwata, Dr. Kenji Abe, Dr. Satoshi Takahashi, Dr. Kazushige Fujiwara, Dr. Yuko Fujiwara, Dr. Makoto Sato, Dr. Yoshiro Iijima, Dr. Yuichi Miyamoto, Dr. Yuko Sato, Ms. Samantha Kawakami, Ms. Ayako Iyasu, Ms. Mayu Fushitani, Ms. Eri Ishii, and Ms. Masae Toyofuku.