Background
In recent decades, public reporting has been used as a vehicle to improve the performance at the institutional and individual levels [
1]. Hospital-level public reporting is currently familiar to most clinicians, whereas physician-level reporting is rapidly progressing and becoming increasingly prevalent [
2]. In the US, the outcomes data for individual cardiac surgeons have been publicly available in the states of New York, Pennsylvania, and New Jersey since 1991, 1992, and 1994, respectively [
3]. Thereafter, an independent newsroom in the US published the nationwide complication rates of nearly 17,000 surgeons in 2005 [
1]. In 2005, the Society for Cardiothoracic Surgery (SCTS) in Great Britain and Ireland published publicly accessible performance data of individual hospitals and surgeons [
4]. In 2013, England became the first country to mandate the release of the names of surgeons and outcome data associated with surgeons across nine surgical specialties [
1].
The public reporting of health care performance data offers several potential benefits. First, public reporting enables patients to identify the best physicians and hospitals, as well as to make informed choices [
5]. Second, public reporting will increase competition within the health care system and offer physicians and hospitals incentives to improve quality [
2,
6]. Third, public reporting provides policymakers and third-party payers with knowledge for informed decisions on payment, including rewarding high or penalizing low performers [
7,
8]. Public reporting does, however, have several unintended consequences. For example, providers may become risk averse, clinical priorities may become distorted, and staff morale may be reduced [
9]. In addition, the inadequate risk adjustment, lack of valid quality metrics and validated data are cited by critics [
6].
Although public reporting is being extensively studied, the perceptions of physicians on the public reporting of hospital- and physician-level performance data are limited. Ross et al. conducted qualitative interviews to explore the perceptions of physicians on public reporting, and indicated that the interviewees expressed skepticism regarding public reporting because of concerns on the ability of patients to comprehend the performance data. However, Ross et al. were only focused on the perceptions of physicians on the hospital level performance data reporting [
10]. Sherman et al. evaluated the perceptions of surgeons on the public reporting of the hospital- and individual-level surgical qualities. The investigated surgeons expressed concerns regarding public reporting, particularly the reporting of individual performance, and were more in favor of public reporting of hospital level outcomes data [
6]. Nevertheless, Sherman’s study focused on the perceptions of surgeons, and perceptions may not be generalizable to other medical workers, such as the general practitioners (GPs) in primary care institutions. GPs play a gatekeeping role in the health care system [
11]. Therefore, gaining an improved understanding of the perceptions of GPs on the public reporting of the hospital- and individual-level performance data may facilitate the acceptance of public reporting, improve performance, and enrich the evidence of public reporting research.
This study intends to answer the following questions: (1) Do GPs have different perceptions on the public reporting of the institution- and individual-level medicine prescribing data in primary care institutions? (2) What is the difference among the perceptions of GPs on the necessity, methodological rigor, and impact of medicine prescribing data reporting?
Methods
Setting and participants
This study was conducted in Q city, Hubei Province, and involved 10 primary care institutions. Q city is located in central Hubei, with a land area of 2004 km2 and a population of 0.95 million. The annual GDP of Q city reached 49.3 billion (RMB yuan) in 2013, just above the average level of all cities in Hubei. The majority of inhabitants in Q city are of Han ethnicity, with about 8000 from Hui, Tujia, and other ethnic groups. The population serviced by the 10 primary care institutions was about 383, 000; outpatient visits was about 49,108; and inpatient was about 1, 482 per year. The majority of patients (about 60 %) of the 10 primary care institutions were covered by New Cooperative Medical Scheme (NCMS); about 20 % patients were covered by other medical insurances (basic medical insurance for urban employees, basic medical insurance for urban residents); and the rest 20 % patients paid at their own expense. For the NCMS, the reimbursement rate is about 85 % for inpatients and about 50 % for outpatient in the Q city.
We selected Q city because the institutional and individual medicine prescribing data have been released to public in November 2013 to promote the rational use of medicine. The public reporting had already last for 9 months when we conducted the investigation. Besides, the health authority of Q city was very supportive of this project and it guaranteed the investigation conducted smoothly. Three prescribing indicators using the outpatient prescription data were calculated at the institution- and individual-levels: percentage of prescriptions requiring antibiotics; percentage of prescriptions requiring injections; and average drug expenditure per prescription. These indicators were ranked in order at the institutional level within 10 institutions and the individual prescriber level within an institution on a monthly basis. The ranking results were displayed in a public place (entrance hall).
GPs with prescribing license from the 10 primary care institutions were eligible for this study. We calculated the sample size with the follow formula:
$$ \mathrm{n}=\frac{{\mathrm{Z}}_{1-\alpha /2}^2\left(1-P\right)}{\upvarepsilon^2P} $$
where,
Z1 − α/2 is the value obtained from the standard normal distribution table for 100 (1-α)% confidence level. Its value for 95 % and 99 % confidence level is 1.96 and 2.58 respectively,
P is the anticipated proportion of the condition in the study population and ε is the relative precision [
12].
According previous study, about 79 % of the doctors are aware of public reporting [
13]. The minimum sample size required for this study for relative precision of 10 % and confidence level of 95 % was 103.
All of the 154 eligible GPs of the 10 primary care institutions would be surveyed on considering of response rate. The investigation was conducted in July 2014.
This study was approved by the Ethics Committee of Tongji Medical College, Huazhong University of Science and Technology (NO: IORG 0003571). An introductory letter was attached to the questionnaire to explain the objectives of this study, and verbally informed consent was obtained from the survey participants. Permission was also granted by the local health authority.
Instrument
The items were developed based on Sherman’s survey instrument and expert opinions; these items were modified for applicability to the research site and target population [
6].
The structured instrument included 3 dimensions and 33 items (i.e., 16 pairs of questions on the perceptions of institution- and individual-level data reporting, as well as a question on the strategies on facilitating public reporting). Five-point Likert scale response options were used in the instrument; the higher the score, the more positive response to public reporting.
The first dimension was perceptions on the necessity of public reporting, including whether public reporting is necessary, and whether third-party reporting is necessary (1 = not at all necessary, 2 = slightly necessary, 3 = somewhat necessary, 4 = very necessary, 5 = extremely necessary).
The second dimension covered the perceptions on the methodological rigor of public reporting, that is, whether GPs are concerned with the following issues: patients cannot interpret or recognize data complexity, the public reporting data are unreliable, a lack of appropriate medicine prescribing metrics, have no adequate risk adjustment, have no adequate amount of prescriptions for statistical comparison, a lake of transparency of methodology (1 = extremely concerned, 2 = moderately concerned, 3 = somewhat concerned, 4 = slightly concerned, 5 = not at all concerned).
The third dimension was perceptions on the impact of public reporting, covering positive impact perceptions and negative impact perceptions. The positive impact perceptions of public reporting include: improving prescription quality, learning more knowledge on rational prescription of medicines, and providing a sense of achievement to GPs (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree). The negative impact perceptions of public reporting include: causing risk averse, damaging quality, penalizing low performers, losing volumes of patients, and increasing medical disputes (1 = strongly agree, 2 = agree, 3 = neither agree nor disagree, 4 = disagree, 5 = strongly disagree).
The last question of the instrument is about the promotion strategy of public reporting, and it’s a multiple choices. Items mainly based on the follow-up interview thematic analysis outcomes conducted by Sherman, including internal review before public reporting, third-party reporting, educating patients on data interpretation, presenting simplified data to patients, more comprehensive risk stratification and case-mix adjustment, and focusing on process measures and not outcome measures. An additional file shows the details of the instrument (see Additional file
1).
Statistical analysis
The mean Likert score with standard deviation (SD) and frequency of positive responses were calculated for each item at the institution- and individual-levels. One-sample Kolmogorov-Smirnov test was used to determine if a normal distribution of perceptions score was, the null hypothesis is that “sample distribution is normal”, and if the test is significant, the distribution is non-normal. For the normal distribution perceptions score, the difference would be compared using paired t-tests. Otherwise, nonparametric test should be used. An analysis of variance (ANOVA) test was used to analyze the differences of Likert score among the three dimensions (i.e., necessity, methodological rigor, and impact of public reporting). If the ANOVA test showed a significant difference in means between the three dimensions, then the Bonferroni correction method will be employed to determine how they differ. A p-value of < 0.05 was considered significant for all comparisons except for the mean pair-wise analysis with Bonferroni correction. For Bonferroni’s correction method, the adjusted p-level was 0.0167, that is, the 0.05 was divided by 3 (the number of pair-wise comparisons in this study was 3). All statistical analyses were performed using SPSS17.0 (SPSS Inc., Chicago, IL).
Discussion
This study determined that the difference of perceptions of GPs between public reporting of the institution- and individual-level data lacked statistical significance. However, GPs expressed highest concerns regarding the methodological rigor dimension of public reporting. Thus, strategies should be considered to facilitate the public reporting of medicine prescribing data, such as disclosing process measures and educating patients on data interpretation.
First, the lack of significant difference on the perceptions of GPs between the institution- and individual-level data reporting should be explained. Public reporting may result in negative impacts on market share, contracting arrangements, or reputation of health care providers [
14,
15]. Generally speaking, for self-preservation, the positive response of clinicians to individual-level data reporting may be generally lower than that in institution-level data reporting. However, our results did not support this hypothesis. The current transparency policy and medical environment of China may explain this phenomenon. On the one hand, provider incentives played an important role in public reporting [
16], but China’s transparency policy lacks explicit incentives. Regardless whether GPs have high or low performance, they both obtained limited penalty or reward. Therefore, GPs did not demonstrate evident reluctance and concerns on the public reporting of their own individual-level medicine prescribing data in our study. On the other hand, education level and socioeconomic status have been demonstrated to affect the searching habits of patients, and quality information was used frequently by those with high education levels [
17]. Patients in Q city with low levels of education may use limited information to select the providers [
18]. Therefore, GPs’ perceptions of individual-level reporting were no more negative than institution-level reporting because they have no selection pressure coming from patients. Note that the results of Sherman’s study indicated that most surgeons supported the public reporting of quality metrics at the hospital level but opposed individual reporting [
6]. Therefore, the studies on the perceptions of clinicians on the institution- and individual-level data reporting showed mixed results and required further research.
Second, compared with the necessity and impact of public reporting, the methodological rigor was the dimension where GPs placed their utmost concern. This result was consistent with those of previous studies. Barr et al. conducted an interview to explore the view of physicians on public reporting and determined that physicians perceived that public reporting lacked methodological rigor [
19]. Rechel et al. mapped the current approaches of public reporting in 11 high-income countries and learned that the accuracy and reliability of performance data were a main concern of public reporting [
20]. Burns et al. suggested that if public reporting was desirable, the reporting data must be valid, and the main validity concerns included sample size; model performance for risk adjustment; and quality of the data [
1]. Necessity and impact have higher scores than methodological rigor, thereby indicating that GPs viewed public reporting as necessary and has significant influence on prescribing practice. This result was similar to those of other studies. Public reporting of provider quality information was viewed as a necessary building block in the pursuit of value-based health care [
21]. Survey data from physicians indicated that public reporting significantly influences performance improvement [
22]. Recent data from the Wisconsin Collaborative for Healthcare Quality showed that public reporting has an impact on quality improvement [
23,
24].
Finally, for the strategies to facilitate public reporting acceptance, over 80 % of GPs involved deemed that public reporting should focus on process measures and not outcome measures, as well as educate patients on data interpretation. The Agency for Healthcare Research and Quality (AHRO) showed that process measures have been used for public reporting and have the following advantages over the outcome measures: (1) process measures may be easier and less costly to measure, (2) process measures can be advantageous when outcomes of interest are rare or sample sizes are small, (3) and well-specified process measures do not need case-mix or risk adjustment [
25]. The Process-Based Quality Improvement Manual stressed that process quality measures expand the domains of quality measurement and were important to promote health care performance [
26]. Similar to emphasizing the importance of process measures, several studies stressed the significance of patient education during public reporting. Yang indicated that consumer education was required to maximize the impact of public reporting [
27]. Stephanie et al. suggested that education was required to help assist patients interpret quality data, and an improved public education on quality data was required to increase the use of reporting data in the future [
28].
The current study has a few limitations. First, as the study was conducted in primary care institutions, the conclusions drawn from this research must be carefully generalized to other types of healthcare institutions. Second, this study was undertaken in just one city. Since socio-economic and cultural may have an effect of GPs public reporting perceptions, the GPs of our study may not be representative of all GPs that with different economic and cultural types. We may expand the sample size in the future when other medical institutions public reporting institution- and individual-level medicine use information.
Acknowledgements
This research is supported by the National Natural Science Foundation of China (71373092). The authors would like to thank the health authority of city Q. Lianping Yang, Chunyan Yang Xiaopeng Zhang, Lijun Wang, Xi Yin, Shiru Yang, Yuqi Xiong, Fangying Zhong, Xiaofei Zheng, Chenxi Liu, Meng Wu, Jie Wan, Chao Yan, Fei Cai and Jing Ye participated in the intervention program and helped with data collection.