Start spreading the news: A structural estimate of the effects of New York hospital report cards

doi:10.1016/j.jhealeco.2008.03.001

Journal of Health Economics

Volume 27, Issue 5, September 2008, Pages 1201-1207

https://doi.org/10.1016/j.jhealeco.2008.03.001 Get rights and content

Abstract

Research on the effects of publicly reported hospital quality report cards on patient market shares is mixed. Higher-ranking hospitals do not consistently experience increases in market share. We argue that this may be because the report cards do not always convey “news” about quality; in some cases the rankings conform with prior beliefs about quality. We develop a structural model of the “news” in report cards and estimate the model using data from New York State in 1989–1991. We show hospitals with negative news in the original 1990 report cards experienced a decrease in market share, but that a misspecified model might continue to find no report card effect.

Introduction

One of the key strategies for improving health care quality is the release of “report cards” on everything from enrollee satisfaction with their health plans to cardiovascular surgery mortality. Report cards may improve the performance of health care markets in at least two ways. First, they may enable enrollees and patients to identify and patronize higher quality payers and providers. Second, they may encourage payers and providers to further improve quality so as to increase demand.¹ The latter is predicated on the former, because improvements in quality will not increase demand if consumers are unaware of them. Thus, if report cards fail to influence demand, both of these benefits may be lost.

As we describe below, several researchers have studied whether report cards affect demand, with mixed results. Dafny and Dranove (2005) suggest that one reason for the mixed findings may be that report card rankings could comport with prior beliefs about quality. For example, it seems doubtful that any positive report card could elevate the reputation of the Mayo Clinic above its current stature. Thus, even a glowing report might not increase Mayo's market share. Report cards will likely have the largest impact on market shares when the results are contrary to prior beliefs.

In this study we propose and implement a methodology to assess the effectiveness of the “news” that report cards provide to the market. Studying the immediate aftermath of the introduction of New York's cardiovascular surgery report cards in December 1990, we find that higher-ranking hospitals did not appear to gain market share. However, once we control for prior beliefs about quality, we find that when report card scores differ from prior beliefs, patients do respond by changing their choice of hospital. We conclude that report cards may be valuable in precisely those situations where they are needed most, when the facts about quality differ from preconceptions. This effect is not symmetric, however—hospitals whose report card rankings were lower than prior beliefs experienced a statistically significant decrease in demand, but hospitals with higher-than-expected scores appear to reap no benefits from the positive news. This is consistent with an earlier finding by Scanlon et al. (2002) who found a similar asymmetric response to health insurance report cards. Based on these findings as well as anecdotal conversations with health care executives, we speculate that this asymmetric response might prove to be a consistent expression of health consumer behavior.

Section snippets

Brief background on report cards

Detailed discussions of health care provider report cards appear elsewhere, so we will summarize some of the key facts, especially those pertaining to the New York report cards that we study.² The Health Care Financing Administration (now the Center for Medicare and Medicaid Services) released the first publicly disseminated provider report cards in 1984. These reported hospital mortality rates for a wide range of conditions and procedures. The HCFA report

Previous literature

Numerous researchers have attempted to study the effects of report cards on market shares. Unless otherwise mentioned, none of these studies account for patient prior beliefs.

Mennemeyer et al. (1997) examined the effects of the report cards published by HCFA between 1986 and 1992. They found that hospitals with higher than expected mortality rates (where the expectations were based on patient characteristics, not market perceptions of quality) experienced almost no effect on market share, but

Why report cards?

The additional information afforded by report cards might provide two critical benefits⁴:

(B1) Report cards could enable patients

Model

In this section we present a highly stylized model that shows how to incorporate prior beliefs into estimates of the impact of report cards. This approach is similar to that of Chernew et al. (in press), although the empirical implementation is somewhat different. Suppose that an individual i who visits hospital j expects to obtain utility U_ij based on their expectation of the hospital's quality Q_j, their travel time, T_ij, and a random component η_ij distributed according to the type-I extreme

Data and methods

Our data come primarily from New York state hospital inpatient records for the years 1989 to 1991. Data from 1990 to 1991 were collected as part of the Healthcare Cost and Utilization Project State Inpatient Databases (SID). Data from 1989 were collected as part of New York State's SPARCS program. The two data sources are equivalent: they are collected from the same hospital records and contain the same information. The data include diagnoses, procedures performed, hospital identifiers, and

Results

Table 3 contains the estimate of Eq. (4). For comparison sake, we also include an estimate of “naïve” Eq. (5). Both models include interactions between age, race, sex, and private insurance and the hospital fixed effects. We also include the travel time to the hospital, the travel time interacted with patient characteristics, and the estimated fixed effect by hospital and year. Because these hospital-year effects were estimated in a previous step, we adjusted the standard errors of the

What is an improved report card score worth?

The results of model (5) indicate that a hospital receiving negative news can expect to gain approximately 7% in market share for each one standard deviation increase in report card score. To get a feel for what a good report card is worth, consider a hospital that currently performs 500 surgeries, was perceived to be two standard deviations below average in quality, but receives a mean report card score. That hospital would perform an additional 70 surgeries as a result of the report card.

Conclusions

In this study, we showed that when hospital report cards provide information that differs from patients’ prior beliefs, patients respond to this information by moving to higher-quality hospitals. We also showed that this effect is primarily due to shifting away from hospitals with negative news, rather than shifting towards hospitals with positive news. This finding may explain why hospital executives with whom we have spoken indicate that their hospital governing boards pay considerable

References (17)

G.Z. Jin et al.
Information and consumer choice: the value of publicized health plan ratings
Journal of Health Economics
(2006)
D. Scanlon et al.
The impact of health plan report cards on managed care enrollment
Journal of Health Economics
(2002)
Altman, L., 1990. Heart-surgery Death Rates Decline in New York. The New York Times, December 5, section...
Altman, L., 1992. Surgical Scorecards: Can Doctors be Rated Just Like Ballplayers? The New York Times, January 14,...
Chernew, M., Gowrisankaran, G., Scanlon, D., in press. Learning and the value of information: the case of hospital...
D. Cutler et al.
The role of information in medical markets: an analysis of publicly reported outcomes in cardiac surgery
American Economic Review
(2004)
Dafny, D., Dranove, D., 2005. Do Report Cards Tell Patients Anything They Don’t Already Know? The Case of Medicare HMO...
Davis, H., 1992. Heart surgery mortality declines 36% across state; improvement in WNY matches the average. Buffalo...

There are more references available in the full text version of this article.

Cited by (99)

Does quality affect choice of family physician? Evidence from patients changing general practice without changing address
2023, Economic Modelling
We examine whether quality affects the choice of practice by patients who are most likely to be well informed about quality: those who leave their practice and join another local practice without changing their address (non-movers). Using 2006/7–2010/11 data on 6766 English practices we estimate fixed effects panel count data models of the number of non-movers leaving a practice and of the number joining. Fewer non-movers leave a practice after increases in clinical quality, the proportion of patients satisfied with access, doctors per patient, and the proportion of doctors qualified in the UK. More join after increases in patient satisfaction with access and doctors per patient. A 10% increase in opening hours satisfaction is associated with a 5.75% reduction in non-movers leaving and a 2.9% increase in non-movers joining. Our results imply that improving information on quality will increase practice incentives to raise quality when competing for patients.
Effects of disclosing inspection scores of health facilities
2022, Socio-Economic Planning Sciences
Health services are highly regulated and targeted for in-depth inspections by government officials in major part of the countries, aiming at the protection of patients from malpractice. In Brazil, the inspection of regulated professions costs approximately 3.3 billion reais (630 million US dollars) annually. Despite the substantial costs, there is lack of evidence on the quality of the public expenditures on regulation of health services. The Brazilian legal framework poses additional challenges to account for policy effectiveness, since professional regulatory bodies may adopt its own inspection strategy. A methodological alternative to deal with the complexity of public policies refers to the adoption of complex systems approach, considering the interaction between agents, the influence of diverse factors, and the possibility of emergence of patterns from bottom-up. Thus, the objective of the study was to present an agent-based model to investigate two inspection strategies applied to health services: one strategy omits inspection scores from patients, whilst the other discloses the inspection scores. The results of the interactions between patients and clinics under different inspection strategies through computational simulation showed that transparency in inspection scores had no significant impact neither on patients' health nor on clinics’ compliance rate. Conclusions drawn in the light of public policy effectiveness indicate the possibility of adaptation of the model to other regulatory environments.
Measuring quality effects in equilibrium
2022, Journal of Health Economics
Unlike demand studies in other industries, models of provider demand in health care often must omit a price, or any other factor that equilibrates the market such as a waiting time. Estimates of the consumer response to quality may consequently be attenuated, if the limited capacity of individual physicians prevents some consumers from obtaining higher quality. We propose a tractable method to address this problem by adding a congestion effect to standard discrete-choice models. We show analytically how this can improve forecasts of the consumer response to quality. We then apply this method to the market for heart surgery, and find that the attenuation bias in estimated quality effects can be important empirically.
Hospital report cards: Quality competition and patient selection
2021, Journal of Health Economics
Hospital ‘report cards’ policies involve governments publishing information about hospital quality. Such policies often aim to improve hospital quality by stimulating competition between hospitals. Previous empirical literature lacks a comprehensive theoretical framework for analysing the effects of report cards. We model a report card policy in a market where two hospitals compete for patients on quality under regulated prices. The report card policy improves the accuracy of the quality signal observed by patients. Hospitals may improve their published quality scores by costly quality improvement or by selecting healthier patients to treat. We show that increasing information through report cards always increases quality and only sometimes induces selection. Report cards are more likely to increase patient welfare when quality scores are well risk-adjusted, where the cost of selecting patients is high, and the cost of increasing quality is low.
What do consumers learn from regulator ratings? Evidence from restaurant hygiene quality disclosures
2021, Journal of Economic Behavior and Organization
Regulators use ratings to inform consumers about firms and products across a range of industries, but little is known about what consumers learn from such ratings. We propose and directly study two pathways through which consumers form beliefs about products and firms based on these ratings: inference about the absolute implications of a rating (e.g., does a good rating mean a firm is of high quality?) and inference about the relative implications of a rating (e.g., do few firms have a rating this good?). In the context of restaurant hygiene ratings, we find that consumers form incorrect beliefs along both pathways and that their mis-calibrated beliefs are strongly related to their willingness to pay for a restaurant meal. We also find that their misperceptions can be partially reduced with informational interventions that impact their willingness to pay as well.
The industrial organization of health care markets
2021, Handbook of Industrial Organization
In this chapter we outline the tools that have been developed to model and analyze competition and regulation in health care markets, and describe particular papers that apply them to policy-relevant questions. We focus particularly on the I.O. models and empirical methods and analyses that researchers have formulated to address policy-relevant questions, although we also provide an overview of the institutional facts and findings that inform them. We divide the chapter into two broad sections: (i) papers considering competition and price-setting among insurers and providers and (ii) papers focused specifically on insurance and market design. The former set of papers is largely concerned with models of oligopolistic competition; it is often focused on the U.S. commercial insurance market where prices are market-determined rather than being set administratively. The latter focuses on insurance market design with an emphasis on issues raised by asymmetric information, leading to adverse selection and moral hazard. In addition, we discuss the literature on consumer choice frictions in this market and the significant implications of those frictions for I.O. questions.

View all citing articles on Scopus

View full text

Start spreading the news: A structural estimate of the effects of New York hospital report cards

Abstract

Introduction

Section snippets

Brief background on report cards

Previous literature

Why report cards?

Model

Data and methods

Results

What is an improved report card score worth?

Conclusions

Journal of Health Economics

Journal of Health Economics

The role of information in medical markets: an analysis of publicly reported outcomes in cardiac surgery

American Economic Review