Implementation of the epilepsy center of excellence to improve access to and quality of care – protocol for a mixed methods study

The limited research on the quality of epilepsy care in the United States suggests room for improvement both within and outside the VA. Our prior work, among the earliest example of epilepsy quality assessment in the elderly [1‐3] found that over half of older Veterans with new-onset epilepsy received sub-optimal anti-epileptic drugs (AEDs), according to recommendations on appropriate medications for geriatric patients [1, 2]. Studies in non-VA settings found that the time from epilepsy diagnosis to referral for surgery for temporal lobe epilepsy patients who fail two or more first-line AEDs remains relatively unchanged at 18 years [4], despite recommendations for referral after only one year of AED failure by the National Association of Epilepsy Centers [5]. Poorly controlled epilepsy is associated with increased cognitive dysfunction, mortality, injury and lower quality of life [6‐17], and accounts for 80% of the cost of treating epilepsy [18]. Thus, assessing and improving the quality of epilepsy care has implications for patient outcomes and healthcare costs.

We developed the first quality measures for the care of adults with epilepsy, the QUality Indicators for Epilepsy Treatment (QUIET), for use in primary care and general neurology settings using an evidence-based expert consensus process (Appendix) [19]. These measures were used as a foundation for the American Academy of Neurology Epilepsy Quality Indicators [20]. Our work in non-VA settings found that the concordance between recommended and actual care provided ranged from 0% to 99%, with 44.2% of recommended care processes being performed. Data indicated similar proportions of concordance for initial seizure assessment (42%), initial epilepsy diagnosis and treatment (44%), chronic epilepsy care (46%), and aspects of care unique to women (36%). More recently, we adapted these indicators to the unique setting of the integrated VA healthcare system (QUIET-VA). We do not know whether Veterans living with epilepsy are receiving high quality care as defined by the QUIET-VA measure, or if access to care and quality of care have improved since implementation of the ECOE initiative.

In the Center of Excellence (COE) model of care delivery, access and quality are dependent upon the interconnections between individuals working to facilitate care in the hub and spoke system [21]. For example, information must flow effectively between hub and spoke for care to be timely, coordinated and appropriate for the patient. Thus, access and quality are properties that emerge from the hub and spoke system [22]. Assessing the impact of the ECOE initiative on access and quality requires an assessment of the interconnections and interdependencies among the hubs and spokes [23].

The ECOE model is intended to improve coordination among members of the epilepsy care team. For example, greater interaction of epilepsy specialty providers in team meetings and patient consultation may facilitate an awareness of how their work relates to the overall goals of the patient and to others in the epilepsy patient care team. Educational outreach may also increase interaction of epileptologists with primary care and general neurology providers, stimulating a sense of alignment and shared knowledge regarding their role in providing high quality epilepsy care.

Studies over the past decade indicate that the relationships, interactions, communication patterns, and flow of information among the individuals within the healthcare team are important to its functioning [21, 24‐26]. Simply changing structures and processes does not always lead to better outcomes [27, 28]. Rather, changes in the organization and delivery of care that lead to increasing interconnections and to timely, accurate, problem-solving communication should generate better quality of care and patient outcomes. Relational Coordination (RC), mutually reinforcing interactions between communication and relationships within a team for the purpose of task integration [29], has been shown to be an important predictor of a team’s or organization’s ability to provide high quality care [30‐33]. RC describes and measures cross-role or cross-task shared knowledge, shared goals and mutual respect among team members as well as the frequency, timeliness, accuracy, and problem-solving orientation of communication among team members. Furthermore, recent work suggests that team leaders play an essential role in promoting healthy communication, relationships, and learning among work groups that appear to be essential for coordinating and achieving complex goals [34, 35].

In a setting such as the ECOEs where goals are defined broadly and leaders are allowed flexibility in achieving those goals, we believe Relational Coordination will have a significant influence on the quality of care provided.

Objective 1: Describe changes in quality of and access to epilepsy care before and after the ECOE initiative. We expect that the ECOE implementation will lead to significant improvements in the quality of epilepsy care as evidenced by a higher proportion of recommended processes of care delineated in the QUIET measures being performed.

Objective 2: Describe the associations between changes in the structure/ processes of care implemented by the four geographic ECOEs and RC among epilepsy care team members in each ECOE region. We expect that the ECOE regions that enact strategies that enhance RC (e.g., video case-conferences between hub and spoke physicians) will have higher RC scores than ECOEs that rely on traditional referral modalities (e.g., electronic consults) between facilities.

Objective 3: Determine whether variations in quality of and access to VA epilepsy care among ECOEs are associated with variations in RC. We expect that ECOE regions with higher RC scores will have higher QUIET-VA indicator performance ratings.

We are conducting a four-year comparative case study using a mixed-methods approach that incorporates strategies from patient safety/process improvement and implementation science [27, 36]. We are using VA inpatient, outpatient, pharmacy, and chart abstraction data to identify changes in quality of and access to epilepsy care in the VA between FY08-FY14 (Objective 1). Qualitative and survey methods will be used to identify changes in the structure and processes of epilepsy care and RC over the course of the study (Objective 2). We will then link data from the first two objectives to determine the extent to which quality of and access to epilepsy care is associated with RC using multivariable models (Objective 3).

Veterans meeting criteria for epilepsy using our previously validated epilepsy identification algorithm comprise the population for quality assessment (Objectives 1 and 3). Briefly, individuals with a diagnosis indicative of epilepsy (ICD-9-CM codes 345, 780.39, 649.4) who have concomitant use of an AED will be included. Using algorithms and techniques developed in previously funded VA studies, we will classify Veterans’ epilepsy status as being New-Onset, Chronic or New to VA [2]. We will also use administrative databases to classify epilepsy status as poorly controlled if they receive emergency or hospital care with a primary diagnosis of seizure/epilepsy, use two or more AEDs, switch AEDs, or have a designation of intractability in the epilepsy specific diagnosis.

Using the overall VA patient epilepsy population, we will first identify random samples of individuals from each geographic ECOE stratified by onset (new, chronic) and severity (controlled, poorly controlled) in FY08 (N = 1,600), FY12 (N = 1,600), and FY14 (N = 1,600) for comparison of pre- and post-implementation of ECOE. The QUIET-VA measure (Appendix) will be used to identify quality of VA epilepsy care through medical chart abstraction (Additional file 1) in the year prior to ECOE implementation (FY08) and the years post-implementation (FY12, FY14). Secondary analyses will use population-based administrative data to assess broad measures of access to neurology and epilepsy specialty care to determine if access to epilepsy specialty care improved over the study period.

For Objective 2, we will sample individuals involved in the care of Veterans with epilepsy in select VA facilities, sampling each hub and spoke, and conduct an in-depth study in each hub-spoke system.

For Objective 1, we will employ three datasets to obtain information about inpatient/outpatient medical encounters, medication, and detailed information about care processes (Tables 1 and 2).

Medical SAS Datasets: National administrative data for VA-provided healthcare are stored as Medical SAS Datasets in the VA national data repository in Austin, TX. These data repositories are updated daily with inpatient and outpatient encounter data from VA clinical information systems and include patient demographic information, date and type of care (clinic code for outpatient or bed section for inpatient care), up to 10 diagnoses, procedures, and provider type; inpatient files also include length of stay. Secondary analyses will examine access to epilepsy specialty care and epilepsy surgery.

Medical chart abstraction: To examine the quality of epilepsy care as defined by the QUIET VA measure, we will perform medical chart abstractions for individuals identified above (Additional file 1). Two chart abstractors will use this tool, independently reviewing 10 charts; disagreements will be resolved by consensus. After there is 90% agreement on abstracted items, chart abstractors will perform remaining abstractions using the standardized chart abstraction form. In addition to quality, we will identify individuals who have epilepsy surgery, extended video EEG monitoring, or other procedures outside the VA.

Data for Objective 2 will be derived from surveys of epilepsy care teams from each selected facility (Table 3) and semi-structured interviews conducted with staff at selected facilities (Table 4). Semi-structured interviews will ask participants to articulate changes in the care processes. Moreover, archival documents generated by the ECOE (e.g., annual reports, minutes) and observations of meetings conducted by various ECOE groups (e.g., Directors, Basic Research) will be analyzed to examine the emerging impact of changes in the ECOE system.

Quality of epilepsy care: We will first identify the completion of recommended care based on the QUIET-VA measure (Additional file 1) for each individual. Quality of epilepsy care will be quantified as the proportion of recommended processes of care that were provided for each patient. We will examine overall quality, using the same categories used in our prior study: new seizure, initial diagnosis and treatment, chronic epilepsy care, and care unique to women.

Access to VA neurology and epilepsy care: Secondary analyses will also examine access to epilepsy specialty care and neurology care first using National VA administrative data. The primary measure, Epilepsy Specialty Care, will identify individuals who receive any inpatient or outpatient Epilepsy Specialty care based on clinic codes, ICD-9A and CPT codes.

Patient characteristics: In addition to our primary outcomes, we will identify patient characteristics for use as covariates/case-mix adjustments for multivariable models, and distance from the hub/spoke as a covariate (Table 2).

Relational Coordination: The RC survey includes questions that examine seven dimensions (Table 3) of how individuals interrelate (Objective 2) [29]. We will also include an examination of communication modalities (e.g., face to face communication, email; Computerized Patient Record System [CPRS]) within the RC survey. Items are rated by participants on a 5-point scale indicating the frequency to which each dimension exists in their care setting (e.g., frequency: 1 = Never, 5 = Constantly) [29].

RC scores are first calculated for each individual by summing the scores of all roles (e.g., neurologist, epileptologist, registered nurse, etc.) for each dimension (e.g., frequent communication) and then dividing by the number of responses. The overall RC score for each participant is derived by calculating the mean of the seven individual scores (range 1–5) [29].

RC scores at the facility level are calculated for each functional group (e.g., neurologist, epileptologist, nurse practitioner) by calculating the mean of each dimension for all members of the functional group, and then a facility RC mean and an overall ECOE RC (hub and spokes) mean. The primary analyses will use the regional ECOE RC mean score, and secondary analyses will examine variation in RC scores among functional groups and hubs vs. spokes [29].

Semi-structured interviews: Semi-structured interviews will be used to obtain richer details regarding the organization of care and specific innovative approaches used by epilepsy care teams in each ECOE to improve the quality of and access to care (Table 4). We will use data collected for the previous objectives to complete Objective 3 using a mixed methods approach for studying the ECOE initiative’s impact on access to and quality of epilepsy care. Each data type will first be analyzed independently to detect key patterns and associations among the variables in both the qualitative and quantitative data. Quantitative data will be analyzed using statistical methods to better understand the impact of the ECOE initiative on access and quality. Qualitative data will be analyzed using constant comparative analysis and open coding methods to generate new understandings of how the contextual nuances in the hubs and spoke facilities during and after the ECOE implementation are associated with access and quality.

Once we have generated the key findings from each data type, we will turn our attention to integrating the findings from these diverse data using a triangulation protocol [37]. We define triangulation as a process of studying a phenomenon using different methods to gain a more complete picture. The goal of this integration is to generate knowledge that goes beyond what is suggested from independent analysis of the qualitative and quantitative data [38]. To do this, we will list the findings from each data analysis effort together and consider where the findings from each method agree, where they provide complementary insights on the same topic, or where they contradict each other, developing meta-themes that go beyond simply reporting the findings from the administrative data, surveys and interviews. Final interpretations that result from this process will arise from the patterns in the data itself as well as the application of our primary theoretical framework, Relational Coordination.

Recruitment for survey participation began on March 7, 2013. Survey data was collected from March 7, 2013, and concluded April 30, 2013. Round 1 interviews commenced March 5, 2013, and are ongoing through March 5, 2014. Round 2 data collection is slated to be conducted from September, 2014 through August, 2015.

Our study applies state-of-the-art quantitative and qualitative approaches to a topic of high interest inside and outside the VA. It is the first of its kind to measure the quality of epilepsy care in a national integrated healthcare system at three points in time, while concomitantly collecting data to describe the specific changes in structure and processes of epilepsy care in the VA that result from the ECOE initiative. The inclusion of the RC measure further strengthens the usefulness of this study by also contributing to implementation science and allowing us to learn what works. Our methods are rigorous. We not only triangulate data sources by using national databases and interview data from both clinicians and administrators, but also feedback our observations to individuals from whom the data is collected to maximize the validity of our analysis.

While cause and effect cannot be ascertained in this quasi-experimental study, we will have longitudinal measures that can be examined, in light of other changes in the field, to better rule out a cohort effect. As such, our study will provide evidence that can inform development of interventions and subsequent research to implement those interventions to improve quality and access to epilepsy care.

Although these data will provide important insights into VA care processes, the findings from this integrated healthcare system may not be generalizable to other settings. These findings can, however, be used to develop future research in other settings and further our understanding of relational coordination in healthcare systems of national scope.

This study has been approved by VA IRB Central (study number IIR 11–067).