Skip to main content
Erschienen in: Journal of General Internal Medicine 12/2017

12.09.2017 | Original Research

Reliability of Physician-Level Measures of Patient Experience in Primary Care

verfasst von: Joshua J. Fenton, MD, MPH, Anthony Jerant, MD, Richard L. Kravitz, MD, MSPH, Klea D. Bertakis, MD, MPH, Daniel J. Tancredi, PhD, Elizabeth M. Magnan, MD, PhD, Peter Franks, MD

Erschienen in: Journal of General Internal Medicine | Ausgabe 12/2017

Einloggen, um Zugang zu erhalten

Abstract

Background

Patient experience measures are widely used to compare performance at the individual physician level.

Objective

To assess the impact of unmeasured patient characteristics on visit-level patient experience measures and the sample sizes required to reliably measure patient experience at the primary care physician (PCP) level.

Design

Repeated cross-sectional design.

Setting

Academic family medicine practice in California.

Participants

One thousand one hundred forty-one adult patients attending 1319 visits with 56 PCPs (including 45 resident and 11 faculty physicians).

Measurements

Post-visit patient experience surveys including patient measures used for standard adjustment as recommend by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Consortium and additional patient characteristics used for expanded adjustment (including attitudes toward healthcare, global life satisfaction, patient personality, current symptom bother, and marital status).

Results

The amount of variance in patient experience explained doubled with expanded adjustment for patient characteristics compared with standard adjustment (R2 = 20.0% vs. 9.6%, respectively). With expanded adjustment, the amount of variance attributable to the PCP dropped from 6.1% to 3.4% and the required sample size to achieve a reliability of 0.90 in the physician-level patient experience measure increased from 138 to 255 patients per physician. After ranking of the 56 PCPs by average patient experience, 8 were reclassified into or out of the top or bottom quartiles of average experience with expanded as compared to standard adjustment [14.3% (95% CI: 7.0–25.2%)].

Conclusions

Widely used methods for measuring PCP-level patient experience may not account sufficiently for influential patient characteristics. If methods were adapted to account for these characteristics, patient sample sizes for reliable between-physician comparisons may be too large for most practices to obtain.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Anhang Price R, Elliott MN, Cleary PD, Zaslavsky AM, Hays RD. Should health care providers be accountable for patients’ care experiences? J Gen Intern Med. 2015;30(2):253–256.CrossRefPubMed Anhang Price R, Elliott MN, Cleary PD, Zaslavsky AM, Hays RD. Should health care providers be accountable for patients’ care experiences? J Gen Intern Med. 2015;30(2):253–256.CrossRefPubMed
2.
Zurück zum Zitat Ding VY, Hubbard RA, Rutter CM, Simon GE. Assessing the accuracy of profiling methods for identifying top providers: performance of mental health care providers. Health Serv Outcome Res Methodol. 2013;13(1):1–17.CrossRef Ding VY, Hubbard RA, Rutter CM, Simon GE. Assessing the accuracy of profiling methods for identifying top providers: performance of mental health care providers. Health Serv Outcome Res Methodol. 2013;13(1):1–17.CrossRef
3.
Zurück zum Zitat Browne K, Roseman D, Shaller D, Edgman-Levitan S. Analysis & commentary. Measuring patient experience as a strategy for improving primary care. Health Aff (Millwood). 2010;29(5):921–925.CrossRef Browne K, Roseman D, Shaller D, Edgman-Levitan S. Analysis & commentary. Measuring patient experience as a strategy for improving primary care. Health Aff (Millwood). 2010;29(5):921–925.CrossRef
4.
Zurück zum Zitat Hofer TP, Hayward RA, Greenfield S, Wagner EH, Kaplan SH, Manning WG. The unreliability of individual physician "report cards" for assessing the costs and quality of care of a chronic disease. JAMA. 1999;281(22):2098–2105.CrossRefPubMed Hofer TP, Hayward RA, Greenfield S, Wagner EH, Kaplan SH, Manning WG. The unreliability of individual physician "report cards" for assessing the costs and quality of care of a chronic disease. JAMA. 1999;281(22):2098–2105.CrossRefPubMed
5.
Zurück zum Zitat Nyweide DJ, Weeks WB, Gottlieb DJ, Casalino LP, Fisher ES. Relationship of primary care physicians’ patient caseload with measurement of quality and cost performance. JAMA. 2009;302(22):2444–2450.CrossRefPubMedPubMedCentral Nyweide DJ, Weeks WB, Gottlieb DJ, Casalino LP, Fisher ES. Relationship of primary care physicians’ patient caseload with measurement of quality and cost performance. JAMA. 2009;302(22):2444–2450.CrossRefPubMedPubMedCentral
7.
Zurück zum Zitat CAHPS Surveys and Instructions. Instructions for analyzing data from CAHPS surveys: Using the CAHPS analysis program version 4.1. Agency for Healthcare Quality and Research; Document No. 2015v2, 2012. CAHPS Surveys and Instructions. Instructions for analyzing data from CAHPS surveys: Using the CAHPS analysis program version 4.1. Agency for Healthcare Quality and Research; Document No. 2015v2, 2012.
8.
Zurück zum Zitat Berwick DM, Murphy JM, Goldman PA, Ware JE, Jr., Barsky AJ, Weinstein MC. Performance of a five-item mental health screening test. Med Care. 1991;29(2):169–176.CrossRefPubMed Berwick DM, Murphy JM, Goldman PA, Ware JE, Jr., Barsky AJ, Weinstein MC. Performance of a five-item mental health screening test. Med Care. 1991;29(2):169–176.CrossRefPubMed
9.
Zurück zum Zitat Hewitt B, Turrell G, Giskes K. Marital loss, mental health and the role of perceived social support: findings from six waves of an Australian population based panel study. J Epidemiol Community Health. 2012;66(4):308–314.CrossRefPubMed Hewitt B, Turrell G, Giskes K. Marital loss, mental health and the role of perceived social support: findings from six waves of an Australian population based panel study. J Epidemiol Community Health. 2012;66(4):308–314.CrossRefPubMed
10.
Zurück zum Zitat Rodriguez HP, Rogers WH, Marshall RE, Safran DG. The effects of primary care physician visit continuity on patients’ experiences with care. J Gen Intern Med. 2007;22(6):787–793.CrossRefPubMedPubMedCentral Rodriguez HP, Rogers WH, Marshall RE, Safran DG. The effects of primary care physician visit continuity on patients’ experiences with care. J Gen Intern Med. 2007;22(6):787–793.CrossRefPubMedPubMedCentral
11.
Zurück zum Zitat Hinchey SA, Jackson JL. A cohort study assessing difficult patient encounters in a walk-in primary care clinic, predictors and outcomes. J Gen Intern Med. 2011;26(6):588–594.CrossRefPubMedPubMedCentral Hinchey SA, Jackson JL. A cohort study assessing difficult patient encounters in a walk-in primary care clinic, predictors and outcomes. J Gen Intern Med. 2011;26(6):588–594.CrossRefPubMedPubMedCentral
12.
Zurück zum Zitat Fong RL, Bertakis KD, Franks P. Association between obesity and patient satisfaction. Obesity. 2006;14(8):1402–1411.CrossRefPubMed Fong RL, Bertakis KD, Franks P. Association between obesity and patient satisfaction. Obesity. 2006;14(8):1402–1411.CrossRefPubMed
13.
Zurück zum Zitat Fiscella K, Franks P, Clancy CM. Skepticism toward medical care and health care utilization. Med Care. 1998;36(2):180–189.CrossRefPubMed Fiscella K, Franks P, Clancy CM. Skepticism toward medical care and health care utilization. Med Care. 1998;36(2):180–189.CrossRefPubMed
14.
Zurück zum Zitat Diener E, Emmons RA, Larsen RJ, Griffin S. The satisfaction with life scale. J Pers Assess. 1985;49(1):71–75.CrossRefPubMed Diener E, Emmons RA, Larsen RJ, Griffin S. The satisfaction with life scale. J Pers Assess. 1985;49(1):71–75.CrossRefPubMed
15.
Zurück zum Zitat Digman JM. Personality structure: emergence of the five-factor model. Annu Rev Psychol. 1990;41:417–440.CrossRef Digman JM. Personality structure: emergence of the five-factor model. Annu Rev Psychol. 1990;41:417–440.CrossRef
16.
Zurück zum Zitat Costello BA, McLeod TG, Locke GR, 3rd, Dierkhising RA, Offord KP, Colligan RC. Pessimism and hostility scores as predictors of patient satisfaction ratings by medical out-patients. Int J Health Care Qual Assur. 2008;21(1):39–49.CrossRefPubMed Costello BA, McLeod TG, Locke GR, 3rd, Dierkhising RA, Offord KP, Colligan RC. Pessimism and hostility scores as predictors of patient satisfaction ratings by medical out-patients. Int J Health Care Qual Assur. 2008;21(1):39–49.CrossRefPubMed
17.
Zurück zum Zitat Conover WJ, Iman RL. Rank transformations as a bridge between parametric and nonparametric statistics. Am Stat. 1981;35(3):124–129. Conover WJ, Iman RL. Rank transformations as a bridge between parametric and nonparametric statistics. Am Stat. 1981;35(3):124–129.
18.
Zurück zum Zitat Carrasco JL, Jover L. Estimating the generalized concordance correlation coefficient through variance components. Biometrics. 2003;59(4):849–858.CrossRefPubMed Carrasco JL, Jover L. Estimating the generalized concordance correlation coefficient through variance components. Biometrics. 2003;59(4):849–858.CrossRefPubMed
19.
Zurück zum Zitat Kraemer HC. Correlation coefficients in medical research: from product moment correlation to the odds ratio. Stat Methods Med Res. 2006;15(6):525–545.CrossRefPubMed Kraemer HC. Correlation coefficients in medical research: from product moment correlation to the odds ratio. Stat Methods Med Res. 2006;15(6):525–545.CrossRefPubMed
20.
Zurück zum Zitat Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–428.CrossRefPubMed Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–428.CrossRefPubMed
21.
Zurück zum Zitat Carmines EG, Zeller RA, eds. Reliability and validity assessment. In: MS L-B, ed. Sage University Paper Series on Quantitative Applications in the Social Sciences. Newbury Park, Calif.: Sage Publications; 1979. No. Series no. 07–001. Carmines EG, Zeller RA, eds. Reliability and validity assessment. In: MS L-B, ed. Sage University Paper Series on Quantitative Applications in the Social Sciences. Newbury Park, Calif.: Sage Publications; 1979. No. Series no. 07–001.
22.
Zurück zum Zitat Datta GS, Ghosh M. Bayesian prediction in linear models: Applications to small area estimation. Ann Stat. 1991;19(4):1748–1770.CrossRef Datta GS, Ghosh M. Bayesian prediction in linear models: Applications to small area estimation. Ann Stat. 1991;19(4):1748–1770.CrossRef
23.
Zurück zum Zitat Shaller D, Yount N, Rauch J, Li S, Corrothers M. 2015 Chartbook: What Patients Say About Their Healthcare Providers and Practices. Rockville, MD: Agency for Healthcare Policy and Research; 2016. Shaller D, Yount N, Rauch J, Li S, Corrothers M. 2015 Chartbook: What Patients Say About Their Healthcare Providers and Practices. Rockville, MD: Agency for Healthcare Policy and Research; 2016.
24.
Zurück zum Zitat Rodriguez HP, von Glahn T, Chang H, Rogers WH, Safran DG. Measuring patients’ experiences with individual specialist physicians and their practices. Am J Med Qual. 2009;24(1):35–44.CrossRefPubMed Rodriguez HP, von Glahn T, Chang H, Rogers WH, Safran DG. Measuring patients’ experiences with individual specialist physicians and their practices. Am J Med Qual. 2009;24(1):35–44.CrossRefPubMed
25.
Zurück zum Zitat Safran DG, Karp M, Coltin K, et al. Measuring patients’ experiences with individual primary care physicians. Results of a statewide demonstration project. J Gen Intern Med. 2006;21(1):13–21.CrossRefPubMedPubMedCentral Safran DG, Karp M, Coltin K, et al. Measuring patients’ experiences with individual primary care physicians. Results of a statewide demonstration project. J Gen Intern Med. 2006;21(1):13–21.CrossRefPubMedPubMedCentral
26.
Zurück zum Zitat Selby JV, Schmittdiel JA, Lee J, et al. Meaningful variation in performance: what does variation in quality tell us about improving quality? Med Care. 2010;48(2):133–139.CrossRefPubMed Selby JV, Schmittdiel JA, Lee J, et al. Meaningful variation in performance: what does variation in quality tell us about improving quality? Med Care. 2010;48(2):133–139.CrossRefPubMed
27.
Zurück zum Zitat Fielding the CAHPS Clinician & Group Survey. In CAHPS Clinician & Group Survey Instructions. Bethesda, MD: Agency for Healthcare Quality and Research; Document No. 2033, 2016. Fielding the CAHPS Clinician & Group Survey. In CAHPS Clinician & Group Survey Instructions. Bethesda, MD: Agency for Healthcare Quality and Research; Document No. 2033, 2016.
28.
Zurück zum Zitat Dyer N, Sorra JS, Smith SA, Cleary PD, Hays RD. Psychometric properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS(R)) Clinician and Group Adult Visit Survey. Med Care. 2012;50 Suppl:S28–34.CrossRefPubMedPubMedCentral Dyer N, Sorra JS, Smith SA, Cleary PD, Hays RD. Psychometric properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS(R)) Clinician and Group Adult Visit Survey. Med Care. 2012;50 Suppl:S28–34.CrossRefPubMedPubMedCentral
29.
Zurück zum Zitat Hays RD, Chong K, Brown J, Spritzer KL, Horne K. Patient reports and ratings of individual physicians: an evaluation of the DoctorGuide and Consumer Assessment of Health Plans Study provider-level surveys. Am J Med Qual. 2003;18(5):190–196.CrossRefPubMed Hays RD, Chong K, Brown J, Spritzer KL, Horne K. Patient reports and ratings of individual physicians: an evaluation of the DoctorGuide and Consumer Assessment of Health Plans Study provider-level surveys. Am J Med Qual. 2003;18(5):190–196.CrossRefPubMed
30.
Zurück zum Zitat Nelson EC, Gentry MA, Mook KH, Spritzer KL, Higgins JH, Hays RD. How many patients are needed to provide reliable evaluations of individual clinicians? Med Care. 2004;42(3):259–266.CrossRefPubMed Nelson EC, Gentry MA, Mook KH, Spritzer KL, Higgins JH, Hays RD. How many patients are needed to provide reliable evaluations of individual clinicians? Med Care. 2004;42(3):259–266.CrossRefPubMed
31.
Zurück zum Zitat McDowell I. Measuring Health: A Guide to Rating Scales and Questionnaires. 3rd ed. New York, NY: Oxford University Press; 2006.CrossRef McDowell I. Measuring Health: A Guide to Rating Scales and Questionnaires. 3rd ed. New York, NY: Oxford University Press; 2006.CrossRef
32.
Zurück zum Zitat Franciosi M, Pellegrini F, De Berardis G, et al. Correlates of satisfaction for the relationship with their physician in type 2 diabetic patients. Diabetes Res Clin Pract. 2004;66(3):277–286.CrossRefPubMed Franciosi M, Pellegrini F, De Berardis G, et al. Correlates of satisfaction for the relationship with their physician in type 2 diabetic patients. Diabetes Res Clin Pract. 2004;66(3):277–286.CrossRefPubMed
34.
Zurück zum Zitat Sixma HJ, Spreeuwenberg PM, van der Pasch MA. Patient satisfaction with the general practitioner: a two-level analysis. Med Care. 1998;36(2):212–229.CrossRefPubMed Sixma HJ, Spreeuwenberg PM, van der Pasch MA. Patient satisfaction with the general practitioner: a two-level analysis. Med Care. 1998;36(2):212–229.CrossRefPubMed
35.
Zurück zum Zitat Bjorngaard JH, Ruud T, Garratt A, Hatling T. Patients’ experiences and clinicians’ ratings of the quality of outpatient teams in psychiatric care units in Norway. Psychiatr Serv (Washington, D.C.). 2007;58(8):1102–1107.CrossRef Bjorngaard JH, Ruud T, Garratt A, Hatling T. Patients’ experiences and clinicians’ ratings of the quality of outpatient teams in psychiatric care units in Norway. Psychiatr Serv (Washington, D.C.). 2007;58(8):1102–1107.CrossRef
36.
Zurück zum Zitat Hong CS, Atlas SJ, Chang Y, et al. Relationship between patient panel characteristics and primary care physician clinical performance rankings. JAMA. 2010;304(10):1107–1113.CrossRefPubMed Hong CS, Atlas SJ, Chang Y, et al. Relationship between patient panel characteristics and primary care physician clinical performance rankings. JAMA. 2010;304(10):1107–1113.CrossRefPubMed
37.
Zurück zum Zitat Consumer Assessment of Healthcare Providers and Systems. Patient experience measures from the CAHPS clinician and group surveys. Agency for Healthcare Quality and Research; Document No. 1309, 2014. Consumer Assessment of Healthcare Providers and Systems. Patient experience measures from the CAHPS clinician and group surveys. Agency for Healthcare Quality and Research; Document No. 1309, 2014.
38.
Zurück zum Zitat Groves RM. Nonresponse rates and nonresponse bias in household surveys. Pub Opin Q. 2006;70(5):646–675.CrossRef Groves RM. Nonresponse rates and nonresponse bias in household surveys. Pub Opin Q. 2006;70(5):646–675.CrossRef
39.
Zurück zum Zitat Elliott MN, Zaslavsky AM, Goldstein E, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 2009;44(2 Pt 1):501–518.CrossRefPubMedPubMedCentral Elliott MN, Zaslavsky AM, Goldstein E, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 2009;44(2 Pt 1):501–518.CrossRefPubMedPubMedCentral
40.
Zurück zum Zitat Rosenthal MB, Dudley RA. Pay-for-performance: will the latest payment trend improve care? JAMA. 2007;297(7):740–744.CrossRefPubMed Rosenthal MB, Dudley RA. Pay-for-performance: will the latest payment trend improve care? JAMA. 2007;297(7):740–744.CrossRefPubMed
Metadaten
Titel
Reliability of Physician-Level Measures of Patient Experience in Primary Care
verfasst von
Joshua J. Fenton, MD, MPH
Anthony Jerant, MD
Richard L. Kravitz, MD, MSPH
Klea D. Bertakis, MD, MPH
Daniel J. Tancredi, PhD
Elizabeth M. Magnan, MD, PhD
Peter Franks, MD
Publikationsdatum
12.09.2017
Verlag
Springer US
Erschienen in
Journal of General Internal Medicine / Ausgabe 12/2017
Print ISSN: 0884-8734
Elektronische ISSN: 1525-1497
DOI
https://doi.org/10.1007/s11606-017-4175-y

Weitere Artikel der Ausgabe 12/2017

Journal of General Internal Medicine 12/2017 Zur Ausgabe

Leitlinien kompakt für die Innere Medizin

Mit medbee Pocketcards sicher entscheiden.

Seit 2022 gehört die medbee GmbH zum Springer Medizin Verlag

Update Innere Medizin

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.