Measuring the impact of a health information exchange intervention on provider-based notifiable disease reporting using mixed methods: a study protocol

The intervention will be implemented within one of the largest and oldest HIE infrastructures in the U.S., the Indiana Network for Patient Care (INPC). The INPC is anchored by the Regenstrief Medical Record System which collects data from a variety of sources, including hospitals, clinics, pharmacies, and laboratories[19]. Lab results are routinely delivered to ordering physicians using the Regenstrief DOCS4DOCS® software and analyzed as ELR transactions by the Notifiable Condition Detector (NCD) which identifies mandated reportable diseases and notifies local and state public health departments of these laboratory results[20].

The study will include outpatient INPC primary care practices operating in urban and rural settings that are part of the INPC, representing multiple health systems and clinics. Clinics that do not use the DOCS4DOCS® software will be excluded.

Two technical interventions will be deployed over a staggered schedule at participating clinics: 1) “standard” pre-populated forms and 2) “enhanced” pre-populated forms. The “standard” forms intervention will use EHR (patient demographics and clinic information) and ELR (notifiable disease test results) data available in the HIE to pre-populate and deliver an electronic version of the official state notifiable disease reporting form to the provider. Providers will be able to review the pre-populated form, add any additional information, and fax completed forms to their local health department. The “enhanced” forms intervention will pre-populate an alternative reporting form with an expanded set of data available in the HIE. For example, the “enhanced” form will not only include test results data for a case of hepatitis B but also corollary results on the patient’s liver enzymes; this is information the health department typically requests from the provider in a follow-up phone call when investigating the reported case of hepatitis. Providers will still be able to review the pre-populated “enhanced” form, add any additional information, and fax completed forms to their local health department. Since deployment is staggered, at any point in time the non-intervention sites can act as natural controls for the intervention sites without the selection bias that is generally present in non-randomized experiments. Therefore, the study protocol is theoretically equivalent in its ability to generate causal evidence to a traditional randomized controlled experiment.

Mixed methods studies require that research questions be linked to and drive the data collection and analysis methods, as well as inform the study design, sample size, sampling, instruments developed and administered, and data analysis techniques[21]. Our primary research questions are:

Using a concurrent mixed methods design, data collection will be conducted during the three project phases: baseline or pre-implementation; post-implementation of the standard form; and post-implementation of the enhanced form. In each phase, qualitative and quantitative data are collected in tandem as coordinated but independent studies. This design will allow us to triangulate the quantitative results from surveys, time-series, and data quality measures with qualitative interview and open-ended survey results to understand experiences with public health reporting before and after each form implementation. Table 1 summarizes the categories of data collected.

The following data will be collected at baseline (retrospective to 12 months prior to introduction of the intervention) and at 6-, 9-, and 12-months after implementation of both form interventions: reporting rates (the number of reports for individual diseases and in aggregate submitted to public health daily); report data completeness (completeness of fields) and accuracy (errors); reporting timeliness (length of time between the laboratory test date and treatment); treatment timeliness (length of time between the laboratory test date and treatment); and reporting burden defined as communication volume (number of phone calls or FAX communications between public health and clinics/providers or laboratories) and duration (total number of minutes) measured at the level of phone call.

Semi-structured interviews will be conducted with representative clinical and public health workers will collect qualitative data regarding provider and public health perceptions of completeness, accuracy, timeliness and burden associated with notifiable disease reporting prior to and after each form intervention. In addition, perceptions regarding data quality, benefits, utility, adoption, utilization and impact on workflow of reporting prior to and after the intervention will be collected during baseline and at 12-months after implementation of each form intervention. Interviews will be digitally audio-taped and transcribed. Public health practitioner input regarding enhancements and supplemental data preferences for the enhanced report form will be captured by convening focus groups to establish consensus on desirable data elements.

Data analysis methods will be more thoroughly reported in the methods sections of future publications presenting the findings of specific analyses.

Quantitative analysis will provide measurable evidence of the impacts of the intervention and enable us to establish likely cause and effect. Longitudinal effects of the interventions will be evaluated in aggregate and across covariates on reporting rates, reporting timeliness and treatment timeliness using an interrupted time-series design to test for the intervention effect and the time trend post-intervention. This approach has been used in other decision support and time-series evaluations[22] and has been recommended for multiple baseline time-series designs that involve two or more sites that are repeatedly assessed while an intervention is introduced into one site at a time[23]. Since the effects of the interventions may vary across clinics, stratified regression analyses will also be performed. Following comparative assessment of reporting completeness, accuracy and communication burden, these data will be added to the regression model. Assessment of data completeness, timeliness, and accuracy will involve pre-post comparison of data quality metrics[15].

Qualitative analysis will provide in-depth context regarding facilitators and barriers to implementation, adoption, benefits and use of the intervention; identify and describe their impacts on HIE individual and organizational processes; and look at the broad range of interconnected processes or causes at play regarding data quality. Analyses will be conducted using qualitative software by experienced coders using the constant comparative method of analysis[24] and utilizing standard approaches to ensure credibility, consistency and robustness of the findings. Transcribed interviews and open-ended survey items will undergo a series of well-established steps to identify emerging themes and trends and, ultimately, build a model to describe the intervention phenomenon in a conceptual form[25]. The process will begin with developing a coding scheme which will be developed from combining concepts derived a priori from the conceptual frameworks driving the study and inductively as the analysis proceeds. Content will be grouped into nodes, a codebook will be built, and codes or code combinations will be summarized and stratified by contextual factors such as demographics, respondent role, etc. These summaries will be entered into appropriate data displays that specify interactions between the intervention, its context and its effects as preparation for triangulation.

Data will be triangulated to find convergence or agreement by cross-validating results. The first step will be exploratory to determine most appropriate “transformation” of the data: conversion of quantitative data into narrative data (“qualitized”) or conversion of qualitative data into numerical codes (“quantitized”) that can be represented statistically. There are several approaches to quantifying qualitative data, including enumerating the frequency of themes within a sample, the percentage of themes associated with a given category of respondent, or the percentage of people selecting specific themes. In these approaches the quantified data can be statistically compared to quantitative data collected concurrently but separately. Another strategy for quantifying qualitative data enumerates whether or not qualitative responses included certain codes—i.e., rather than seeking to understand how many times a certain code was provided by each participant or the frequency with which they appeared, the presence or absence of each code for each participant is quantified into dichotomous variables 0 or 1 based on absence or presence of each coded response. We will determine the most appropriate approach after reviewing the descriptive analysis. Depending on which approach is used, this mixed data will be correlated (quantitative data correlated with qualitized data or vice versa)[26]. Guided by the nature of the data collected, quantitative and qualitative data may be collated to create new or consolidated variables or data sets in order to further compare and integrate data.

Once transformed, we will calculate simple correlations, stratify codes by provider type, demographics, geographic location or organization attributes or look at similarity among respondents. This process will allow us to identify areas in which findings agree (convergence), contradict (discrepant or dissonant) or deepen understanding on the same issue (complementarity)[27]. We anticipate that the results will allow us to identify, analyze and explain social, behavioral and environmental similarities and differences between HIE settings, provider types and perceptions across pre- and post-implementation time units that will describe identified barriers and facilitators to the intervention.

There are several limitations to our proposed work. First, some clinical sites may be more open to recruitment and enrollment in the study and thus introduce a bias in our sample. For example, some sites may serve patient populations that are more likely to require notifiable disease reporting (for example, a women’s clinic with a high Chlamydia reporting history) and thus be more incentivized to participate. It is possible given use of an interrupted time-series design, that confounding may occur due to covariates that may change over time. The INPC is a growing HIE so it is possible that policy, governance, legal mandates or information technology changes could occur during the course of this study that may impact data collection or introduce additional confounding issues. This HIE includes an academic affiliation and medical training program so providers may rotate through training sites at different stages of the intervention which may introduce confounding. Introduction and adoption of the intervention may be more rapid in some settings (small ambulatory clinic) than others (large hospital) as the workplace may accommodate or adjust to the intervention more easily or require administrative protocols to support the intervention. It is also possible that the staggered implementation of the intervention may complicate quality control of data collection. Also, given data collection covers only one baseline year and a maximum of two years post-intervention, depending on site, this short time frame may limit ability for analysis to account for seasonal trends in reporting. Our findings may also be limited by the context in which this study is conducted. The INPC is one of the most advanced HIEs in the country; therefore, our results may have limited generalizability. However, we believe our emphasis on context and by clearly documenting the characteristics of the sites in which the intervention, our findings could inform implementation other HIE settings. Given current mandates to build systems and infrastructures for ELR and HIE in communities where it is absent, expand efforts in communities where ELR is already occurring, and requirement for eligible hospitals to routinely transmit reportable laboratory results to a public health agency, our findings may provide insights and inform roadmaps for more nascent HIEs to move forward towards better notifiable disease surveillance.

The project received approval by the Institutional Review Board of Indiana University with a concurrent Institutional Review Board deferral from the University of Washington to Indiana University. Informed consent will be obtained from all participants and confidentiality will be ensured. All data will be stored according to the rules of the research ethics committee. Because this study does not meet ICMJE guidelines it has not been registered in a publicly accessible registry.