Impact of public hospital pricing reform on medical expenditure structure in Jiangsu, China: a synthetic control analysis

Real-world health policy evaluations often rely on evidence derived from natural experiments or observational studies where randomized control trials (RCTs) as the gold standard are ethically infeasible or practically unavailable [1]. Causal inferences are subject to potential confounding effects if non-RCT evidences are used; difficulties remain in choosing the proper comparison control groups and constructing the counterfactual that best resembles the characteristics of the case of interest [2].

Traditional time-series analysis lacks a control on the unobserved confounding factors, such as any policies/programs that occur after the target policy, and thus might incur bias in effect estimations [3]. Difference-in-differences (DiD) method is also commonly used to estimate the effect of a health policy, by contrasting the change in outcomes pre- and post-intervention, assuming the counterfactual treatment groups follow a parallel trend with the control groups [4, 5]. Nevertheless, such ‘parallel trend’ is sometimes criticized as an implausible assumption and thus might introduce bias to the results [6]. The synthetic control method (SCM) has emerged as an alternative to the DiD method that relaxes the parallel trend assumption. By allowing time-varying effect of unobserved factors, it is useful to evaluate the impact of a treatment or a pilot program of the treatment implemented in some regions or units [2, 6, 7]. This method has been widely applied in a variety of program evaluations worldwide [8‐12]. Another major merit of SCM is that causal inference can be drawn from data collected at the aggregate level (public census and statistical yearbook), when individual-level data are unavailable or difficult to obtain.

China initiated a health system reform in 2009. The central component of the reform was the medical pricing reform, also known as the zero-markup drug policy, which removed the markups adding to the prices of any medications prescribed and sold in public hospitals. The drug mark-up policy could date back to 1955, when the policy allowed the public hospitals to add a 15% markup over the selling prices of any medications over their purchase prices [13]. The policy was issued to compensate the medical services in hospitals (e.g. diagnosis, nursing, surgery), which were underpriced due to the stable wage system back then. Unexpectedly, the policy incurred accumulating complains about over-prescription, especially in antibiotic products [14, 15], indicating the incentives of health providers’ services were distorted. The government therefore took actions to remove all the existing mark-ups and rectify the twisted pricing system of medications and medical services in the 2009 reform, aiming to control the proportion of drug in total medical expenditure and adjust the health providers’ behavioral incentives [16].

The reform of drug mark-up removal was initiated in 2010 nationwide and took on four phases before fully rolling out in September, 2017 [17]. Within each phase a number of cities were chosen to pilot the reform. Jiangsu initiated the reform in 2013, aiming to lower the proportion of drug expenditure while keeping the total medical expenditure per visit stable [18]. Zhenjiang was chosen as the pilot city in Jiangsu Province to implement the pricing reform on January 1, 2013, while all the other cities (11 cities in total except Suqian) in Jiangsu begun the reform on October 30, 2015.

Despite a few studies evaluating the reform [19‐22], a formal unbiased analysis on the policy effects coupled with accurate construction for the counterfactual could identify the causality and ascertain the true values of the effects. In this study, we aim to apply the SCM in estimating the effect of the pilot reform in Zhenjiang by controlling confounding factors and weighting the control cities to construct a more suitable comparison group for Zhenjiang. Another purpose is to illustrate the usefulness of this method and its applicability in the evaluation of health policy in the context of Chinese healthcare system.

Figure 1 shows the longitudinal trend of the observed percentage of drug expenditure among total medical cost per visit in outpatient and inpatient care. Overall, we saw a decreasing trend in the drug expenditure proportion among both outpatient and inpatient care in most cities during the study period. As expected, the drug expenditure proportion witnessed a sharp decline in Zhenjiang in both outpatient and inpatient care in 2013 when the reform was piloted while the trends varied in other cities. Another turning point we noticed was 2016 where the drug expenditure proportion underwent a dramatic decline in all cities except Zhenjiang and Suqian, which was aligned with the fact that the same scale of reform that had been piloted in Zhenjiang was implemented to all the other cities except Suqian in 2015. However, we also noticed that the drug expenditure proportion among inpatient cost in Zhenjiang was significantly and constantly lower than other cities by an average of 10%, causing a synthetic control unable to be constructed from the donor pool, a methodological limitation noted by previous study [3]. Instead, we reduced this drug expenditure proportion by 10% for all cities in the donor pool in inpatient care to ensure comparability of baseline characteristics between the treated and control cities.

Table 1 compares the mean value of the outcome variable and covariates in Zhenjiang and the synthetic control, as well as the average of all cities in the donor pool during the pre-policy period. Overall, the synthetic Zhenjiang provided a better match on the characteristics in pre-policy period for outpatient care as compared to the average of other cities, particularly in reproducing the outcome variable. Similarly, in Table 2, with the adjustment made to the outcome variable, the synthetic Zhenjiang suggested a more suitable comparison group than the average of other cities for inpatient care.

Table 3 shows the weights of each city in the synthetic version of Zhenjiang. The composition differs for outpatient and inpatient synthetic Zhenjiang. The top three contributors to the synthetic Zhenjiang in outpatient care were Nantong (0.619), Wuxi (0.187) and Lianyungang (0.124), while the counterparts in inpatient care were Taizhou (0.412), Wuxi (0.299) and Yangzhou (0.250).

The left panel in Fig. 2 compares the proportion of drug expenditure among average outpatient cost per visit of Zhenjiang and its synthetic counterpart. The right panel demonstrates the intertemporal difference of this proportion between the observed and synthetic Zhenjiang during the study period. The drug expenditure proportion of the synthetic Zhenjiang remained stable from 2008 to 2015, while the actual data showed a decrease of 6.9% (from 46.4 to 39.5%). The mean reform effect was estimated to be 7.7% (absolute value in gap) during the post-policy period (2013–2015), which amounted to 16.3% of its 2012 level (47.0%).

Figure 3 shows the comparison (left panel) and intertemporal difference (right panel) of drug expenditure proportion among average inpatient cost per visit between Zhenjiang and the synthetic Zhenjiang. The proportion dropped in both groups, whereas the decline was greater in the observed Zhenjiang (9.0%, from 38.3 to 29.3% between 2008 and 2015). The results suggested an estimated mean reform effect of 3.2% in absolute value (2013–2015) or 9.2% decrease proportional to its 2012 level (34.8%), suggesting that the reform might pose a larger impact on outpatient care medical expenditure than on inpatient care.

None of the placebo effects were found to be as great as the estimated effect in Zhenjiang in outpatient outcomes (Fig. 4, left panel), while there was only one city, Suqian, whose placebo effect was greater that Zhenjiang in inpatient scenario (Fig. 5, left panel), indicating a significant policy effect of the medical reform. To note that, in conformity with the standard practice of Abadie et al. [7], we excluded cities whose pre-policy mean squared prediction error (MSPE) of the drug expenditure proportion that were greater than 5 times the MSPE that we found in Zhenjiang in inpatient scenario in the placebo test analysis. We interpreted the seemingly large placebo effects in these excluded cities as a result of a lack of fit, rather than of the effect of the reform. As a result, placebo test results for Wuxi and Nantong were excluded in our analysis for inpatient care. We also found that the trajectories of the placebo outcomes were distributed symmetrically around zero for both outpatient and inpatient care (Fig. 4, right panel and Fig. 5, right panel), suggesting that the synthetic control method would be expected to estimate a zero effect when the true effect was zero.

This synthetic control analysis has shown significant impact of the medical pricing reform on medical expenditure structure in Zhenjiang in reducing the proportion of drug expenditure among total medical cost per visit in both outpatient and inpatient care. The results are consistent with previous studies. For instance, in 2014, Wang et al. forecasted that there would be a decrease in the proportion of drug expenditure over the per-visit in-patient care expenditure from 56.8 to 47.8% using data from one pilot district of Nanjing [27]. In 2017, Hu et al. analyzed the per-visit drug expenditure proportion in out-patient care in 5 cities in Jiangsu, and found consistent trend of decrease in this proportion: Nanjing (60.9 to 55.7%), Xuzhou (62.4 to 57.0%), Suzhou (54.2 to 48.5%), Nantong (50.5 to 45.5%) and Huai’an (50.0 to 43.6%), ranging from 5.0 to 5.7% [28]. In 2015, Dai et al. stated that the drug expenditure proportion of the per-visit expenditure in out-patient and in-patient care was estimated to drop by 6.0% (46.3 to 40.3%) and 4.8% (34.5 to 29.7%) from 2012 to 2013, respectively, in Zhenjiang [29]. It is worth noting that the studies mentioned above might have limitations of not controlling for individual trends or confounders. On the contrary, our study explicitly accounted for socioeconomic confounders to construct a more suitable counterfactual for comparison, and we estimated the effects to be 7.7% (outpatient) and 3.2% (inpatient), indicating that previous studies presented potentially biased effects of the pricing reform.

However, this study is not without limitations. First, in some control cities, the reform was implemented at scattered levels (i.e., in some regional hospitals) during the pre-policy period (2012–2015), which would inevitably reduce the estimated impact. However, we found significant results from such conservative estimation, further reinforcing the reform effect. Second, as shown in Fig. 1 (lower panel), Zhenjiang displayed strong uniqueness in inpatient care expenditure structure, due to which a synthetic counterfactual was unable to construct from the donor pool [3]. We instead reduced the proportion of inpatient drug expenditure for control cities by a fixed number to enable comparability between the treatment and control groups. However, bias could be introduced by the adjustment with respect to the magnitude of the estimated effect. To address this potential bias, we conducted an additional cross-validation with a DiD approach that confirmed the estimated effect size.

This study contributes to literature in the following ways. First, the evaluation of health policies and reforms in China has long been limited by data unavailability. The access of individual-level data, as the gold standard of data source, was rather restricted by health authorities and hospitals. The synthetic control analysis of this study in the context of Chinese healthcare system provides an example of how to utilize publicly available population-level data and reduce the reliance on individual level data on to evaluate the effect of health policies and reforms. Second, some previous evaluation studies had less control over confounders, which might over- or under-estimate the impacts of certain policies. This study provides an analytic framework to account for a series of covariates to exclude potential confounding effects. Third, our conservative results suggest that the impact of pricing reform in Jiangsu was potentially underestimated, which might arouse the needs to further examine the impacts of similar reforms in other Chinese provinces. Last, we hope this study could provide meaningful implications for the applicability of the synthetic control analysis method in future policy evaluation and decision-making in China.

We demonstrate the applicability of the SCM by studying the effects of Jiangsu’s medical pricing reform on medical expenditure structure through a case study in the city of Zhenjiang where the reform was piloted. This study suggests that the medical pricing reform in urban public hospitals has significantly reduced drug expenditure incurred in both outpatient and inpatient care. While our results are generally consistent with other studies, this SCM analysis contributes to the literature by rigorously selecting comparison units, constructing counterfactual scenarios and controlling for confounding effects. This study also highlights the value of population-based approach that can be constructed from publicly available census data on the evaluation of health policy and reform.