Developing instruments to measure the quality of decisions: Early results for a set of symptom-driven decisions

Abstract

Objective

To identify a set of critical facts and key goals and concerns for five common medical conditions, benign prostate disease, hip and knee osteoarthritis, herniated disc and spinal stenosis and examine the validity of the method for identifying these items.

Methods

Investigators identified facts and goals through literature reviews and qualitative work with patients and providers. A cross-sectional survey of patients and providers was conducted to examine the accuracy, importance and completeness of the identified items.

Results

42 facts (6–16 per condition) and 31 goals and concerns (4–13 per condition) were identified. 182 responses were obtained from patients (76.5% response rate) and 113 responses from providers (78% response rate). Overall, the facts were accurate, important and complete across all conditions. For one condition (hip osteoarthritis), the goals did not meet the criteria for completeness. There was more disagreement between patients and providers around the ranking of goals than of facts.

Conclusions

Overall, respondents found the identified facts and goals accurate, important and complete. Significant differences between patients’ and providers’ rankings highlight the importance of including both perspectives.

Practice implications

Instruments to measure whether or not patients are informed and the extent to which treatments reflect patients’ goals must balance patients’ and providers’ perspectives when selecting items to include.

Introduction

Are patients well-informed before they make fateful medical decisions? To what extent can geographic variation in treatment rates or apparent disparities in health care delivery be explained by warranted variation in patients’ preferences for different health outcomes? There is a consensus growing around the importance of engaging and informing patients in decisions about their care, and around the critical role of patients’ preferences in treatment decisions. The Institute of Medicine has defined patient-centered care as “healthcare that establishes a partnership among practitioners, patients and their families (when appropriate) to ensure that decisions reflect patients’ wants, needs and preferences and that patients have the education and support they need to make decisions and participate in their own care” [1]. Recent consensus efforts among researchers, policymakers, consumers and developers of decision aids as part of creating the IPDAS standards also found strong support for a definition of decision quality (DQ) as “the match between chosen option and features that matter most to the informed patient” [2].

Currently, there are no systematic data regarding how well patients are informed prior to most major medical decisions. Nor are there any data or even well-validated approaches to determine the extent to which treatment decisions match patients’ preferences [3], [4]. Most approaches that attempt to measure the quality of care focus on documenting adherence to evidence-based guidelines, which is appropriate in situations where there is proven, effective care [5], [6]. Wennberg and others have distinguished between “effective-care” decision situations and “preference-sensitive” decisions [7]. The latter is characterized by the presence of more than one medically appropriate option, where the best treatment can only be determined by how an individual weighs the potential benefits and harms. Traditional measures that examine treatment rates alone, or adherence to guidelines, do not provide any information about the quality of decisions in preference sensitive situations. In these situations, it is critical to determine whether the right person is getting matched with the right treatment.

The authors and colleagues have been working on the development of instruments to measure the quality of decisions for preference-sensitive decisions. The first proposal was described by Sepucha et al. [8] and subsequent publications have detailed the conceptual underpinnings of the approach [9] as well as some pilot results [10]. The underlying conceptual framework builds on the systems approach to shared decision making first outlined by Mulley in 1989 [11], [12]. The framework takes a prescriptive approach to decision making and does not require that patients follow the axioms of decision theory, rather it recognizes that factors other than utilities for health states may influence choices, which the authors refer to in this report as “goals and concerns.”

The DQ instruments are designed to measure the extent to which treatments reflect informed patients’ goals and concerns. The instruments are composed of a set of decision-specific knowledge questions and a set of subjective assessments of patients’ goals and concerns. The goals and concerns include good and bad health outcomes, holistic attitudes toward treatments, and the desired influence of others. To be practical, the investigators have hypothesized that a small set of key facts can be reliably identified that are necessary in order for a patient to be considered “informed.” In addition, a small set of goals and concerns can be identified that can be used to calculate the extent to which patients’ goals are met by the chosen treatment [8].

The purpose of this report is to describe the early development work on DQ instruments for a set of similar symptom-driven conditions and to present results from a set of cross-sectional surveys of patients and providers. These conditions include osteoarthritis of the knee (OAKnee) and hip (OAHip), herniated lumbar disc (HD), spinal stenosis (SS) and benign prostatic hyperplasia (BPH). Each of these conditions cause symptoms that compromise quality of life, symptoms may fluctuate over time, and can interfere with daily activities. Surgery is an option for each of these conditions, and offers a high chance of relief, but carries with it a set of harms, including serious risks and problematic side effects. Non-surgical treatments are also available for each condition, and these have fewer possible side effects, but are generally less effective than surgery in relieving symptoms. These are preference-sensitive decisions as the “best” choice depends on how each patient feels about the likelihood and magnitude of the possible symptom relief and about the likelihood and impact of different harms, including the expected timing of the benefits and harms.

As mentioned before, the investigators believe that there are limited set of key facts that patients need to understand in order to be informed and a limited set of goals that are salient for patients and will help distinguish those who may prefer surgical vs. non-surgical treatment. This paper reports evidence on the validity of the methods for identifying those items for inclusion in a decision quality instrument. In particular, it examines the extent to which the process identified items that were accurate, important and complete, and whether or not it was possible to identify a small, core set of items for each condition.