Development of the National Network of Depression Centers Mood Outcomes Program: A Multisite Platform for Measurement-Based Care

Mood disorders are among the most burdensome public health concerns in the world. The lifetime prevalence of major depression in the United States is 13% to 17% (1, 2), and the World Health Organization (WHO) has ranked major depression among the most disabling disorders on the globe (3). Depression is strongly associated with suicide, particularly in populations at high risk, such as veterans, elderly persons, and those struggling with chronic pain (4, 5). Management of depression in medical specialty care is also critical because depression is strongly associated with morbidity and nonsuicide mortality in chronic conditions such as heart disease and diabetes (6, 7). Mrazek and associates (8) estimated that treatment-resistant depression occurs in 12% to 20% of cases and incurs an annual added societal cost of $29 to $48 billion.

There has been progress during the past 5 decades in improving pharmacologic and psychotherapy treatments of mood disorders. However, despite these gains, the etiology of these disorders remains poorly understood. As a result, patients and providers must often work through multiple medication trials before finding an effective treatment and achieving remission. The Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial found that only 37% of patients responded to the first antidepressant prescribed, and up to four medication changes were needed to achieve a 67% response rate overall (9–11). Sustaining wellness then becomes an incremental challenge.

New approaches are needed to develop personalized treatments that are tailored to the specific needs of individual patients and can achieve more effective and timely remission and maintenance of wellness. Such precision treatments require improved understanding of the multifactorial etiologies and longitudinal course of these complex disorders. To help meet this challenge, large, well-characterized samples of patients with extended follow-up over time are needed to identify more meaningful clinical phenotypes and their underlying biological substrates. Within the United States, the medical communities surrounding cancer and heart disease have successfully developed longitudinal patient cohorts and registries. For example, the Framingham Heart Study (12–15) and registry programs sponsored by the National Heart, Lung, and Blood Institute (16) and the National Cancer Institute (17) have contributed clinical insights that are transforming these formerly acute illnesses with high mortality into chronic illnesses with remarkable recovery, survivorship, and improved functioning.

Comparable efforts are needed to produce similar gains for the millions receiving treatment for mood disorders in psychiatric and primary care settings (18). Nordic and Asian countries have developed longitudinal registries of psychiatric disorders, based on information collected in the context of their national health services (19–22), but the United States does not have a comparable health care infrastructure, making it challenging to support similar nationwide registries. The closest the U.S. has to such registries are efforts to study clinical data from large health systems, such as the Mental Health Research Network (23) or the Veterans Health Administration (5, 24). Longitudinal cohort studies of individuals recruited for research aims, such as the Collaborative Study of Depression (25–27) or the aforementioned STAR*D trial (10) have also contributed enormous insights into mood disorders. However, these studies focused on selected populations, or the sample sizes of these studies were limited because of the logistical challenges and costs of ongoing monitoring. The NIH recently initiated the “All of Us” research program (28), which seeks to overcome the limitation of small sample sizes by establishing a prospective cohort of more than 1 million. However, the depth of clinical information that will be gathered on mental illnesses in this project is unclear.

In the United States, the recent proliferation of electronic health records and physician quality reporting systems tied to the Affordable Care Act (ACA) and the subsequent Medicare Access and CHIP Reauthorization Act (MACRA) (29) have promoted development of a nationwide information technology (IT) infrastructure. Major electronic health record vendors support electronic capture of patient characteristics and health care utilization. Perhaps more important, current mature electronic health records enable electronic assessment of key patient-reported outcomes to inform and help guide clinical decision making and measurement-based care. This infrastructure has the potential to support identification and longitudinal monitoring of patients with mood disorders in real-world clinical contexts. Clinicians can then learn what works best for these patients while providing care and, in iterations, continuously improve the care overall in what is referred to as a “learning health system” (30, 31).

The National Network of Depression Centers (NNDC), founded in 2008, is a nationwide nonprofit consortium of 26 leading clinical and academic member centers in the United States (32). Its overarching mission is to integrate innovative research, clinical translation, education, and public policy to better diagnose and treat depression, bipolar disorder, and related mood disorders. In service of these goals, the NNDC is currently implementing the Mood Outcomes Program whereby participating sites follow a standard protocol to collect patient-reported outcomes in the context of routine clinical care. This systematic data collection is intended to enable measurement-based care (33–35) that can enhance patient and provider decision making at the point of care and has been shown to be effective in improving overall patient outcomes (33). The collected data are then deidentified and gathered from multiple sites into a central repository that can support population health analytics and quality improvement initiatives to define best practices and aid the development of precision health care. The repository also provides a platform for carrying out multisite research through secondary data analyses or embedded studies, such as pragmatic clinical trials. In this way, the Mood Outcomes Program will provide the crucial infrastructure for a nationwide learning health system on mood disorders that can drive sustained improvement in patient outcomes.

This article describes the development of the Mood Outcomes Program as the standard of care in the NNDC and reports on a pilot study of the program at eight NNDC sites. This overview presents descriptive statistics based on the first 24-month period of data collection within the pilot sites. These data are presented to demonstrate the feasibility of collaborative data collection for the creation of a large clinical repository. As the program is expanded and disseminated across all NNDC sites, we anticipate, the repository will be able to support analyses that take advantage of ever more powerful data.

Methods

The planning and development of the Mood Outcomes Program began in 2011 and entailed the following steps: agreeing on the patient-reported outcome measures; specifying clinical workflows for administering these measures; building an IT platform to collect the measures, visualize the results in real time, and share deidentified data with the national repository; and establishing regulatory compliance and data use policies for continuous learning from the data. A pilot of the program was then initiated in 2015 and carried out in eight participating NNDC sites.

Patient-Reported Outcome Measures

Starting in 2011, a group of 20 NNDC members conducted a series of phone conference meetings to discuss and develop consensus on a battery of longitudinal rating measures to be collected as “vital signs” for a new measurement-based care program for patients with mood disorders (36). This became known as the Standard Assessment Package. The aim was to develop a reliable, valid, brief, cost-free, self-rated, standardized, and repeatable clinical tool, not just an extensive research battery, such as those employed in clinical trials. The goal was a tool that efficiently provides the most essential clinical information to inform point-of-care decisions. Initial clinical targets deemed most essential were severity of mood symptoms for both depression and mania, presence and severity of comorbid anxiety, and suicidal ideation/risk.

Balancing these considerations, the following four measures were selected: the nine-item Patient Health Questionnaire (PHQ‐9) (37, 38); the seven-item Generalized Anxiety Disorder Scale (GAD‐7) (39); the five-item Altman Self‐Rating Mania Scale (ASRM) (40); and the seven-item patient‐rated screener version of the Columbia‐Suicide Severity Rating Scale (C-SSRS) (41), with a baseline version to assess both recent and lifetime ideation and attempts and a follow-up version to assess ideation and attempts since the last clinic visit.

Clinical Workflows

The goal was for patients to complete the measures in the clinic waiting rooms prior to their initial and follow-up appointments and for the results to be available in real time for review with their clinicians to inform measurement-based care. The decision to collect the measures at the clinic was made for two reasons. The first was to establish a culture that emphasized the importance of routinely collecting these mental health “vital signs” at each visit as the standard of care (36). The second was to make it easier to address concerns about suicide risk. There were clinical and legal concerns about whether patients should be allowed to complete the assessments before they came to the clinics. For example, if patients were allowed to complete the assessments early, mechanisms would be needed to address situations in which a patient endorsed suicidal ideation or behavior on PHQ-9 item 9 or the CSSR-S. Such mechanisms could add a prohibitive burden to implementing the program. Beyond these requirements, sites were free to adopt whatever workflows worked best for their clinics in order to maximize program adoption.

Health IT Infrastructure

Central IT implementation.

The NNDC collaborated with the Altarum Institute, a nonprofit health informatics organization, to develop a health IT platform (called ePro) to support the implementation of the Mood Outcomes Program. The platform includes an on-line patient portal for collecting the patient-reported outcome measures and a clinician dashboard for reviewing the longitudinal results during the clinical encounter. The online patient portal allows patients to enter the self-rated scales directly into the clinical management system using tablet computers or other similar devices in the clinic waiting rooms. Procedures for setting up the patients’ accounts and logging into the portal were designed to be as simple as possible to minimize demands on support staff. Per best practices, logging in by the patient requires a unique username and password. Once the patient-reported outcome data are entered by the patient into the portal, they are available in real time for viewing by the clinician through an on-line clinician dashboard. Once providers have access to the clinician dashboard, they can quickly navigate to the appropriate patient and view the longitudinal results of the patient-reported outcome assessments in tabular form or displayed as graphs of trends over time. (A figure showing an example of the results is available in an online supplement to this article.) The graphs include annotation to flag values of clinical concern, including if the patient endorses items suggesting suicidal risk that needs to be addressed clinically. The patient-reported outcome results, including individual items, can also be copied and pasted into the patient’s medical record note if desired.

The clinical data are maintained in a multitenant database such that the clinical data from each participating site are virtually partitioned to ensure data security and patient confidentiality. This clinical database is hosted on the Altarum Secure Network, which is fully compliant with all HIPAA standards for a secure environment with an encrypted database. Data are encrypted “at rest” on the server and “in flight” during transmission between the application server and the Web browser. The data stored in this multitenant database support the clinical goals of the measurement-based care program.

The clinical data are automatically uploaded on a near real-time basis into a separate repository of clinical data called the NNDC KnowledgeBase. Data in the repository have been deidentified and support the population health analytics and research goals of the “learning health system.” The NNDC KnowledgeBase is housed in a secure partition of the Altarum Secure Network. A dedicated clinical coordination team at Altarum is responsible for managing the loading of the clinical data into the NNDC KnowledgeBase. The clinical data are deidentified by stripping away all protected health information (PHI), including the 18 designated “Safe Harbor” identifiers (42). Clinic site and patient identification numbers are anonymized, and all dates are randomly shifted using an algorithm that preserves the day of the week. The clinical coordination team at Altarum retains the crosswalk between the anonymous clinic site and patient identification numbers and any personally identifying information.

Local IT implementation.

Many sites have existing local IT platforms that they would prefer to use instead of the Altarum platform. To accommodate such preferences and maximize uptake, a local IT implementation pathway was developed for these sites. A distinct advantage of the local IT implementation pathway is that clinicians do not have to bounce between dueling systems when caring for their patients in the program, which reduces the resistance to adoption. Under the local IT implementation pathway, sites build the assessments and graphical views of the results into their local platforms. The local builds are then reviewed by the Mood Outcomes Program Steering Committee to ensure the implemented measures are consistent with NNDC standards. Once the local builds are approved, local sites upload their data into the NNDC KnowledgeBase on a quarterly basis through a data loader after the data have been suitably deidentified and transformed into the appropriate data model according to NNDC specifications.

Given that more than two-thirds of the NNDC sites use Epic as their electronic health record, the NNDC established a collaboration with Epic to build the Mood Outcomes Program assessment package within Epic’s foundation system to allow for more efficient implementation of the program at these sites. The Mood Outcomes Program assessments can be deployed to patients either through Epic’s welcome self-service check-in and registration module or MyChart (Epic’s patient portal), and the results can be viewed in graphical form by clinicians in the patient synopsis view.

Regulatory Compliance and Data Use Policies

Each NNDC site participating in the program executes a Business Associate Agreement with Altarum to manage the necessary PHI and support the clinical care functions of the program. In addition, each site consults with its institutional review board as required to contribute the clinical data to the NNDC KnowledgeBase for research purposes. Because the clinical data are collected as part of standard of care and are fully deidentified prior to inclusion in any research data set, patient consent is not obtained. This approach to consent was decided upon after extensive consultation with an advisory board that included members of the NNDC Mood Outcomes Program Steering Committee as well as several bioethics experts, institutional review board representatives, and patient advocates from NNDC sites participating in the pilot study. The approach and the discussions were greatly influenced by recent thinking about learning health systems (43, 44).

The NNDC also established a data use policy to govern access to the deidentified clinical data for research purposes. NNDC members from sites that have contributed clinical data to the Mood Outcomes Program may request access to the deidentified research data set by submitting a data request form that must be preapproved by the NNDC Mood Outcomes Steering Committee. Qualified researchers outside the NNDC may also request access to the deidentified data by submitting a data request form that must be approved by the NNDC Mood Outcomes Steering Committee, as well as the NNDC executive committee, which consists of the NNDC leadership, and, as appropriate, representatives from each member site that contributed data.

Pilot Study

Eight NNDC centers participated in the pilot study of the Mood Outcomes Program. These centers were at the University of Michigan, Johns Hopkins University, the Mayo Clinic, the University of Iowa, the University of Louisville, the University of Colorado Anschutz Medical Campus, the University of Illinois Chicago, and the Medical University of South Carolina. The Mood Outcomes Program was implemented as a standard of care at outpatient psychiatry clinics in each of these centers. Patients with a primary diagnosis of any mood disorder were included in the program, and in subsequent analyses they were classified as having either unipolar depression or bipolar disorder. Patients were older than age 18 years, and one center (Johns Hopkins University) additionally recruited patients younger than age 18 years to pilot the program in a pediatric population.

Results

The pilot study of the Mood Outcomes Program at NNDC centers began in August 2015. Figure 1 shows the rate of patient enrollment over time from initiation of the pilot to the first quarter of 2019. The network reached its target of 10,000 unique patients in December 2018. Of the eight sites that contributed data, one used Epic to implement the program and integrate the assessments into the initial evaluation of all patients in its outpatient clinics. This site contributed more than double the number of patients than the next leading site, demonstrating the advantages of the Epic implementation to facilitate adoption of the program. As a result, we anticipate more sites will adopt the Epic implementation as we continue to expand the program.

Table 1 presents the demographic characteristics and summary results on the Mood Outcomes Program measures at the baseline visit for patients enrolled over the first 24-month period of data collection, broken down by bipolar disorder (N=849) versus unipolar depression (N=3,998). The sample is predominantly female, reflecting closely the two-to-one gender ratio reported in both treated and community samples for unipolar mood disorders. Both diagnostic subgroups were middle-aged on average, with a mean age of 42.9±15.7 years for those with bipolar disorder and 40.2±16.4 years for those with unipolar depression.

The mean baseline PHQ-9 and GAD-7 scores fell in the mild to moderate range. They were 12.3±7.1 and 10.5±6.4, respectively, for patients with unipolar depression, and 11.2±7.2 and 9.5±6.5, respectively, for those with bipolar disorder. It is likely that few patients were in manic episodes at the time of assessment, reflected by the mean ASRM scores, which were 3.5±3.6 for those with bipolar disorder and 2.7±2.9 for unipolar depression. Nearly one-third of the patients in both diagnosis subgroups had a lifetime history of planned or actual suicide attempt. It is noteworthy that the baseline visit for the registry did not necessarily occur at treatment initiation; many patients had been in treatment before the baseline assessment.

Figure 2 reflects the longitudinal course of depression for patients who had a PHQ-9 score greater than 10 at the baseline visit and had a follow-up visit at least 30 days after the baseline assessment. A total of 57.5% of the patients with unipolar depression and 58.6% of the patients with bipolar disorder remained symptomatic, as measured by the PHQ-9, at their last follow-up. For patients with unipolar depression, 16.1% achieved remission (a score of less than 5 on the PHQ-9), while 18.7% of the patients with bipolar disorder achieved remission from depression.

Figure 3 presents explorations of the relation between patient characteristics—gender, marital status, diagnosis, and comorbid anxiety—and the longitudinal course of depression symptom severity. For example, the trajectory diverged between men and women at approximately 6 months’ follow-up, with men showing clearly poorer recovery rates at the 12- and 24-month follow-up. Patients who were married showed improved recovery at 12 and 24 months, relative to patients in all other marital categories. Although the trajectory for depression remission was largely comparable between patients with unipolar depression and patients with bipolar disorder, having comorbid anxiety as indicated by a baseline GAD-7 score of 10 or higher, portended a significantly poorer course, with marked divergence at 24-month follow-up.

Discussion and Conclusions

The progress of the NNDC Mood Outcomes Program thus far demonstrates the feasibility and potential of collaborative systematic collection of patient-reported outcomes as part of a measurement-based care program for the treatment of patients with mood disorders. The program has met its initial goals of enrolling more than 10,000 patients (current enrollment is 10,570 patients) and using standard assessments routinely collected through a flexible health informatics infrastructure to inform treatment at the point of care. As we disseminate the program to additional sites across the network and continue to follow the patients with multiple assessments, we anticipate rapid growth of a progressively richer data set to explore the longitudinal course of mood disorders. The pilot study, therefore, provides an important step toward achieving our goal of a fully realized “learning health system” (30, 31) for mood disorders.

Preliminary analysis of the data collected in the pilot study demonstrates that new and more powerful precision-driven treatments, ideally guided by biomarkers, are needed to understand and optimally treat mood disorders. In this sample, nearly 60% of the patients with mood disorder remained symptomatic at follow-up. The analyses of the association between patient characteristics and longitudinal course revealed a poorer course for men, unmarried individuals, and those with comorbid anxiety. This initial snapshot of clinical burden, chronicity, and risk for suicide concurs with the converging public health data from the World Health Organization and others indicating that mood disorders are among the most debilitating health conditions (45, 46).

These results reflect the common experience of treating mood disorders; that is, the best treatments and most skilled clinicians still provide most patients with only partial improvement in their mood disorders (9, 26). Although this should give one pause, the results should be interpreted with several important caveats in mind. The data were collected in a naturalistic setting as patients received usual care. As a result, patients may have entered the program at different points in their episode of care, and those who were doing better may have been less likely to return to the clinic for additional treatment. Thus, with the data collected during this pilot study, we may be observing only a partial snapshot of the full trajectory of illness, especially among patients with more challenging outcomes. We anticipate that as we continue to follow more patients over longer periods of time, we will get a more complete picture of the different illness trajectories, and we will be able to examine how differential follow-up affects the interpretation of the results.

Despite the initial excitement generated by the introduction of selective serotonin reuptake inhibitors and, more recently, by multiple-mechanism antidepressants (47) or the promise of new findings in psychiatric genetics (48), our field has stalled in its ability to significantly reduce the medical burden of a large proportion of patients with mood disorders. Larger, longitudinal, measurement-based databases are essential if we are to advance to more precise and effective treatments. The current pilot study illustrates that collection of such data are achievable, and it provides important lessons that can facilitate the wider implementation of this program and others like it (Table 2).

Encouraged by the success of the pilot study, we are currently working to expand the reach and scope of the Mood Outcomes Program. We are using the lessons of the pilot study to rapidly disseminate the program to other NNDC sites and additional clinics within these sites, which should further accelerate the enrollment of patients into the program. Particularly noteworthy is that we seek to expand the program into primary care, where the majority of patients with mood disorders often receive care, as well as into other specialty clinics, such as women’s mood disorders clinics or brain stimulation clinics. In addition, we are working on plans to collect detailed clinical data extracted from participating sites’ electronic medical records—such as health care utilization, diagnostic information, treatment histories, and laboratory results—to link with the patient-reported outcome measures in the NNDC KnowledgeBase. We envision using the infrastructure of the Mood Outcomes Program and the rich, longitudinal data gathered in the NNDC KnowledgeBase as a platform to support large-scale, multisite quality improvement and research efforts across the NNDC centers. This may involve using big-data analytics with the gathered data, as well as conducting embedded studies in real-world settings to generate new data. The goal will be to rapidly translate what is learned from these efforts back into improved care for our patients and thus achieve the vision of a nationwide learning health system for mood disorders.