Using Claims Data to Assess Treatment Quality of First-Episode Psychosis

Growing literature supports that intervening at the moment of a first episode of psychosis is critical for maximizing recovery, resulting in fewer future hospital admissions and better overall functioning (1–4). Coordinated specialty care (CSC), a team-based approach that provides evidence-based pharmacological treatment, individual support, family education, and appropriate care coordination, has demonstrated greater effectiveness than usual care for first-episode psychosis (FEP) (3, 5). Most recently, a landmark study of the Recovery After an Initial Schizophrenia Episode–Early Treatment Program (RAISE-ETP) refined and validated this treatment approach (6).

The goal of RAISE-ETP was to test the effectiveness of CSC versus usual community treatment among individuals experiencing FEP. NAVIGATE, the CSC intervention in RAISE-ETP, used a team-based approach that included shared decision making, a focus on patient strengths and resiliency (using cognitive-behavioral therapy methods), motivational enhancement, and psychoeducation. Core interventions included individual resiliency therapy, supported education and employment, and individual algorithm-based medication management tools to guide medication prescribing. Family education and care coordination were included to ensure appropriate treatment. NAVIGATE participants experienced significantly greater improvements in self-rated quality of life, depression, and symptom severity over a 2-year period compared with the community care group (3). These effects were largely mediated by duration of untreated psychosis.

Despite guidelines supporting best practices, gaps exist between recommended and actual treatment of schizophrenia, which lead to poorer outcomes, increased hospitalizations, and decreased quality of life (5, 7, 8). Clinician factors, such as lack of awareness or familiarity with guidelines, disagreement with guidelines, and inertia of previous practice, may influence prescribing practices (9). Patient factors, including gender, race-ethnicity, age, insurance status, and patient preference, may also influence prescribing practices (10).

As our understanding of how to best treat FEP evolves, support is needed for efforts to facilitate consistent implementation of evidence-based practices across populations. A first step is to develop methods to assess quality of care in a manner that is cost effective and that generates actionable information. For this study, we developed quality measures on the basis of the principles included in the RAISE-ETP, in the Schizophrenia Patient Outcomes Research Team (PORT), and in other sources that characterize best practices in terms of psychopharmacological treatment, provision of psychosocial interventions, coordination of care, and appropriate health monitoring. The primary aim of this exploratory work was to apply these measures to a Medicaid database to characterize the state of treatment for individuals with FEP on the basis of these actionable quality indicators (QIs). The secondary aim was to investigate the relationship between quality of care and patients’ future health care utilization. Our hypothesis was that deviation from best practices would be associated with increased utilization of emergency and hospital services due to suboptimal quality of care.

Methods

Data Collection

We identified individuals between the ages of 15 and 40 by using Medicaid claims obtained from MO HealthNet, Missouri’s Medicaid program, with dates of service ranging from November 2007 through April 2017. Our claims-based definition of FEP was similar to what has previously been used in related literature (11). Participants were required to have been eligible for Medicaid benefits for at least 10 months in the year prior to and the year after the first episode of psychosis and to have had no evidence of a prior psychosis diagnosis, identified through the presence of prior diagnostic codes. (A list of diagnostic codes used to identify individuals with FEP is available in an online supplement.) Additionally, we required evidence of a prescription of antipsychotic medication in the 3-month period after the first qualifying diagnosis of FEP. A prior study reported that this additional criterion greatly enhanced the accuracy of the claims-based FEP diagnoses included in the study (12). A psychotic episode was defined as the occurrence of a claim containing any of several qualifying diagnostic codes (see online supplement). Psychiatric comorbid conditions, including substance abuse, anxiety disorders, and mood disorders, were also identified. Claims information also provided information on medication prescribing (antidepressants and antipsychotics, including both oral medication and long-acting injectables [LAIs]); use of laboratory testing; and psychosocial interventions, including care coordination. Outcome data for health care utilization, defined as behavioral and nonbehavioral emergency department (ED) and hospital visits, were also provided through Medicaid claims. (Lists of the claim codes associated with each best practice measure and of reference dosage ranges for each antipsychotic medication are available in an online supplement.) Institutional review board approval was received from the Center for Outcomes Analysis.

Best Practice Measures

Fourteen best practices for treating FEP were operationalized into QIs (10, 13) (Box 1). The QIs represent deviations from best practices. (A list of the procedure codes that correspond to each QI is available in an online supplement.) A preliminary review of the data showed that the procedure codes did not appear in the claims data for three QIs: no evidence of a medication review (QI 12), no evidence of family psychoeducation (QI 13), and no evidence of vocational intervention (QI 14). We concluded that these treatment activities may be occurring under other generic billing codes or were not billed to Medicaid. One QI—no evidence of use of an antipsychotic medication (QI 11)—was a criterion for inclusion in the study. These four QIs were not included in the analysis or results.

To investigate the gap between CSC best practices and the actual care delivered, the percentage of patients whose treatment met the criteria of each quality measure was calculated for the year after the index diagnosis of FEP. To investigate future health care utilization, we conducted a within-individual model by using a stratified Cox proportional hazards multivariate regression model in which each individual was used as his or her own control. For this model, we compared treatment episodes (i.e., periods between hospitalizations or ED visits, which were the outcomes) for individual patients. This is the preferred approach for comparative effectiveness research in nonrandomized samples (14, 15) because it mitigates confounding by indication (i.e., an intervention looking less efficacious if it is indicated for individuals with worse prognosis). Because the comparisons are between treatment episodes for individual patients, only factors that change as a product of time need to be adjusted. Participants were followed for up to 6 years after index diagnosis of FEP in order to evaluate for QIs and health care utilization. Follow-up time was reset to zero after each outcome event to allow comparison of treatment periods for each individual. Therefore, since each patient was his or her own control, we adjusted only for order of outcome, referring to the number of times the outcome occurred for that participant. With this approach, each individual’s treatment episodes that deviated from best practices were compared with his or her treatment episodes that adhered to best practices in order to predict behavioral and nonbehavioral ED usage and hospital visits. A Bonferroni correction for four outcomes per QI was made to adjust the level of significance to p<0.0125.

Results

Applying the described inclusion criteria, 6,246 participants were identified, and their care was evaluated by using the quality measures, with a mean follow-up time of 4.24 years (range 1–6 years). The mean±SD age of the cohort was 27.54±7.52 years, 48.2% were male, and the cohort had a high prevalence of comorbid substance abuse, depression, and anxiety disorders (Table 1).

State of Treatment for Individuals With FEP on the Basis of QIs

Table 2 shows the percentage of patients whose treatment met criteria for each of the 10 QIs for which data were available. Significant gaps between best practices and actual care were revealed in the appropriate use and monitoring of antipsychotic medications. Of individuals prescribed antipsychotic medication, 5% received prescriptions above recommended daily doses (QI 4), 16% received two or more antipsychotics (QI 7), and 20% were treated with olanzapine or clozapine (QI 3). In the year following initiation of a second-generation antipsychotic, 34% had no evidence of metabolic screening (QI 6).

We observed that 2,758 participants failed to refill a prescription of antipsychotic medication (QI 5). Further investigation found that 335 of these individuals had subsequent claims for LAI antipsychotics. Presuming that the discontinuation of oral antipsychotics was deliberate, these individuals were removed from the “failure to refill” group, yielding a rate of 41.1% of participants whose treatment met the criteria for this measure. Of note, 485 additional participants from the cohort showed evidence of treatment with LAI antipsychotics over the observation period.

Within-Patient Associations Between QIs and Treatment Utilization

Table 3 summarizes the relationship between each of the QIs and utilization of behavioral and nonbehavioral ED and inpatient services. Failure to monitor for substance abuse (QI 9) and failure to monitor for smoking (QI 10) were both associated with increased risk of all forms of utilization, ranging from 39% to 62% and from 105% to 171%, respectively. Lack of psychotherapy (QI 1) and lack of integrated care delivery (QI 8) were also both associated with an increased risk of all forms of utilization, ranging from 29% to 127%. Failure to refill a prescription for an antipsychotic (QI 5) was associated with lower risk of all forms of utilization, ranging from 59% to 65%. Use of high-dose antipsychotics (QI 4) was associated with a 22%−31% decreased risk of behavioral ED and nonbehavioral ED visits and a 24%−38% decreased risk of behavioral hospital and nonbehavioral hospital visits. Use of two or more antipsychotics (QI 7) was associated with a 26%−27% decreased risk of behavioral ED and nonbehavioral ED visits and a 24%−29% decreased risk of behavioral hospital and nonbehavioral hospital visits. Inappropriate use of antidepressants (QI 2) was associated with increased risk of behavioral (10%) and nonbehavioral (16%) ED visits. Participants lacking metabolic monitoring (QI 6) showed reduced utilization ranging from 38% to 59%. Use of olanzapine or clozapine (QI 3) was associated with a reduced risk of all forms of utilization, but these results were not statistically significant.

Discussion

This study evaluated the utility of 10 QIs to assess the quality of care among patients in a Medicaid claims database. Our results highlight the areas of treatment of FEP that most likely diverge from recently updated treatment recommendations. With a few exceptions, our results for the metrics related to medication prescribing were qualitatively similar to those identified by Robinson and colleagues in the RAISE-ETP data set. For example, 5% of patients prescribed antipsychotic medication received it at a higher than recommended dose, which is similar to the 8.8% reported in the RAISE-ETP study (10). Almost 20% of patients were prescribed olanzapine, which is slightly higher than the 16% reported in the RAISE-ETP study (10). Finally, 16% of our cohort was prescribed two or more antipsychotics, whereas the RAISE-ETP study reported 9%. These data suggest that gaps between evidence-based recommendations and usual care are still large and deserve attention from policy makers and administrators. An important qualification is that we did not adjust for concurrent therapeutic interventions nor for the order of treatments. It is possible that use of high-dose antipsychotics, polypharmacy, and olanzapine or clozapine was preceded by more appropriate but less effective treatment.

Psychopharmacology practices that are not recommended for the treatment of FEP (olanzapine as first-line treatment, antipsychotic polypharmacy, and doses higher than recommended) were associated with lower treatment utilization in within-patient analyses. Because the follow-up period for examining utilization began when criteria of a QI were met, this association could reflect a “step up” in treatment strategy after initial approaches using antipsychotic medications failed. Although these practices (except for use of clozapine) have not shown superior treatment efficacy compared with treatment as usual, they could have resulted in decreased need for emergency and inpatient services. It is also possible that a step up in treatment strategy included additional psychosocial interventions that were not captured by our analysis. Further study will be required to better understand these observations.

Our results showed that 44% of patients had at least one episode during the measurement period where they failed to refill antipsychotic medication. These results are comparable with other real-world cohorts. For example, using the Finnish national registry, Tiihonen et al. (16) found that 41.8% of individuals with FEP failed to fill a prescription for antipsychotic medication after hospital discharge, and 54.3% did not do so within 3 months of discharge; other studies have found similar results (17–19). We found, to our surprise, that this QI was associated with reduced ED and inpatient utilization. Individuals with FEP who feel better may feel more comfortable discontinuing treatment because they may feel that they do not need it (20, 21). Data from similar studies suggest that median time to relapse after treatment interruption in this population is longer than a year (22). Therefore, it is possible that individuals who discontinued were less symptomatic initially, with an apparent lower risk of relapse in the short term but greater risk of relapse in the mid- to long term, which was not captured within our observation window. In our sample, 13.1% of participants were treated during the observation period with at least one dose of LAI antipsychotics. Although the patterns of LAI use were not studied, we observed that approximately 12% of participants who did not refill an antipsychotic medication (QI 5) received an LAI in the subsequent 2 months. The low utilization rate of LAIs in this population, well below the average for patients with schizophrenia in general (20), is alarming given that a majority of individuals with FEP relapse, especially when they do not adhere to treatment (21, 22), and that psychosis relapse is associated with detrimental effects on recovery and potentially danger to self or others. LAIs have proven to be effective and well tolerated to treat FEP (23), yet our data illustrate how underutilized they are in this population.

An important aspect of prescribing antipsychotics is the consistent monitoring of weight and metabolic parameters (24). Even though this recommendation has been widely publicized since 2004 (25), 34% of patients receiving second-generation antipsychotics had no evidence of receiving metabolic screening in the 12-month evaluation period. This finding is of concern given the relatively high use of olanzapine. We are not aware of studies reporting metabolic monitoring rates in a population of individuals with FEP. However, in a large general Medicaid population receiving antipsychotics, 20.4% lacked evidence of annual glucose testing, and 58.8% lacked evidence of annual lipid screening (26). Our findings appear to be generally consistent with this report, although we credited a case as having monitoring if any of the individual tests were observed in the follow-up period. This method would be more likely to overestimate the level of monitoring. Lack of monitoring was associated with a lower likelihood of near-term service utilization, which may reflect less concern by providers with “less sick” patients, whereas providers would be more likely to conduct a more thorough assessment of patients at greater risk.

One of the RAISE-ETP treatment benchmarks states that antidepressants should not be prescribed in the absence of a diagnosis of depression or anxiety disorder. We found high rates of depression or anxiety disorder diagnosis (73.1%) and antidepressant use (71.1%), suggesting high rates of comorbid conditions with FEP. Approximately 20% of patients receiving antidepressants had no justifying diagnosis. Although this figure is substantially lower than the 50.4% of patients in the RAISE-ETP study (10), this finding still requires attention by policy makers.

We examined several QIs that were not related to medication. There was no evidence of psychotherapy (QI 1) or integrated care (QI 8) in 71% and 22% of the cohort, respectively. These indicators may be underreported in our data set because psychotherapy or activities related to care coordination could be bundled with other procedure codes. Efforts should be made to increase the use of psychotherapy in treatment of FEP because it has been shown to be effective (27). Although a CSC approach is slightly more costly than usual care, it results in significantly higher quality of life, which offsets the costs (28). Lack of claims evidence for psychotherapy and integrated care delivery were both associated with across-the-board increases in future health care utilization. These data highlight the gap between evidence-based recommendations and real-world utilization of nonpharmacological treatment elements, given the positive impact they may have on outcomes such as treatment utilization.

Comorbid substance abuse in this study was greater than what was reported in RAISE-ETP, with over two-thirds (70.3%) of participants meeting criteria for comorbid substance use. This finding highlights the need to make addiction interventions central to treatment of early psychosis. A finding of great concern is that despite such high incidence of comorbid substance use disorders, monitoring for substance abuse was reported for less than one-fifth of the patients. It is possible that monitoring was being addressed in ways that are not captured by claims data. In our sample, lack of monitoring was associated with greater treatment utilization, which attests to the large need for the treatment of comorbid substance use disorders in the population with FEP.

These results should be interpreted in the context of several methodological limitations. For instance, our sample was derived from Medicaid claims data, which may not be representative for individuals with FEP whose treatment is reimbursed by private insurance. Also, the stratified Cox regression minimizes confounding by indication; however, the prognosis of a single patient may change over the course of follow-up. This change may result in noncomparable risk between the period before and after the criteria of a QI were met. We mitigated this limitation by conducting adjusted analyses for co-occurring treatments when a QI was met and for order of QIs. Therefore, associations may not be explained only by causality. Unmeasured, confounding differences in treatment severity, inaccurate billing of QI-related interventions, or regression to the mean (where patients whose illness was more acute and who had previously received interventions such as antidepressants, olanzapine, or clozapine regressed to the mean over follow-up) could explain these findings as well. These limitations of using claims data are in tension with the advantages of obtaining information for large populations and of limiting selection bias. Therefore, our results should be interpreted in the context of other literature, such as randomized controlled trials. A final limitation was that claims diagnoses may not be as reliable as research diagnoses.

Conclusions

Our results suggest that applying an analytic strategy to claims data is a feasible approach for measuring aspects of FEP treatment. This analysis showed a large gap between delivered care and CSC best practices. These results suggest that policy makers should pay special attention to improving the monitoring of patients’ general medical health and substance misuse in order to maximize fidelity with CSC, as it becomes the new standard of care in FEP treatment.