Background
Incorporating new health information technology (HIT) into medical practice involves workflow changes and potential impacts, such as increased physician workload and compromised communication patterns between providers and patients. Although emerging HIT applications are directed to improving the quality of care and economies of service [
1], some investigators have warned that the use of HIT may threaten the very nature of the patient-provider relationship by undermining trust, inhibiting disclosure of relevant concerns, and by hampering meaningful discussions of patient preferences that impact treatment decisions [
2‐
5]. Reassuringly, HIT applications that incorporate point-of-care use of health-related quality of life (HRQL) assessments have been found to promote patient-centered interactions between seriously ill patients and physicians in specialty outpatient settings [
6‐
9]. Although prior work has demonstrated the effectiveness of HRQL assessments in specialty clinics it is unclear what factors may contribute to the successful implementation of HRQL-HIT applications in primary care settings. For example, one prior feasibility study examining the use of an HRQL-HIT application in 14 German general practices found substantial variation between practice sites and identified time constraints as a substantial barrier to both collecting and effectively using the HRQL information [
10]. Clearly, technological feasibility alone does not result in consistent use within real world practices.
To further explore factors affecting HRQL information use, we conducted a pilot implementation study [
11] of an HRQL-HIT application in three primary care practices. We were particularly interested in understanding how physicians assessed the value of an HRQL application for their chronically ill geriatric patients. For these providers, geriatric patients are only a subset of their patient population; therefore, the HRQL application had to be embedded within the practices' existing electronic health record (EHR) systems in a way that it became available only for relevant patients.
To evaluate the implementation of the HRQL module, we used a comparative case study design. Our initial thinking about the implementation process was informed by the perceived attributes of innovations identified by Rogers [
12]. However, our data collection was guided more specifically by the Technology Acceptance Model (TAM), which posits that a user's perceptions about the ease of use and usefulness of a technology influence the intent to use the technology, and ultimately actual use of the technology [
13]. The specific aim of the study was to explore factors that providers consider when assessing the value added of a new HRQL information technology application. Our overall goal was to better understand how the practice environment and preferences of individual providers impact decisions about whether to use an HRQL EHR module.
Methods
Description of HIT Intervention
The Geriatric Enhancement Module (GEM) was developed by a team of university-based health services researchers in collaboration with a private software vendor. It is comprised of a 7-item questionnaire that gathers patient-reported health-related quality of life (HRQL) data about physical health, emotional health, physical functioning and limitations in activities of daily living, and level of social support. The goal of the GEM is improve the quality of care discussions among staff, providers and patients. The care settings in this study were the first to use the GEM.
The GEM software was designed so that items would be prompted to appear within the EHR during the intake portion of the medical encounter (i.e., when vital signs and chief complaints are recorded by clinical staff) for all patients 50 years and older. Patients viewed and entered answers to the GEM questions directly into the medical record system with the help of a research assistant (RA) and/or clinical staff person. During the patient encounter, GEM items were displayed in the EHR for the provider. Patients also could raise issues related to their answers to the GEM questions.
Study design
In evaluating the value of using GEM, we were particularly interested in: (1) the practices' level of engagement in the GEM implementation; (2) the users' (i.e., providers and staff) opinions about their EHR system and expectations for the GEM; (3) the level of use of the GEM information during the patient encounter, and; (4) factors affecting the level of use and decisions about continuing to use the software. To structure our data collection and analysis, we used a multiple, embedded case study design with the embedded units being providers, nursing/administrative staff, and patients nested within the larger case (i.e., the physician practice). This design enables within and between case analyses [
14]. Data collection for the study was staggered, beginning with one practice and expanding to others. This approach allowed for data collection procedures and/or sampling strategies to be modified as needed to account for issues that emerged [
15].
Selection and recruitment of study sites and patients
The study involved three primary care practices. Two were small (i.e., 1 or 2 providers), independently owned family practices located in small towns. The other was a general internal medicine practice that was housed within a large academic medical center. Given the focus of the study - HRQL modules within EHR systems - we needed to utilize practices that had operational EHRs that could incorporate the GEM. The two independently owned practices were recruited with the help of a medical software vendor that had a prior proprietary relationship with the practices and was knowledgeable about their IT capabilities. Based on initial findings from these first two sites, we sought variation in terms of ownership and size for the third site to explore the impact of these practice characteristics on the GEM implementation and assessment. Therefore, we recruited the practice from an academic health center with no current relationship with the vendor.
For all practices, the PI and other members of the project team visited the site to provide details about the study and obtained informed consent from members of the practice. After informed consent was obtained, a co-investigator and/or an (RA) with office nursing experience provided training to physicians and practice staff about the GEM. In addition to this training session, the RA was available for physicians and staff on designated data collection days to provide support and answer questions. Participating sites received a one-time $2500 incentive to reimburse them for staff time related to the study.
Clinical staff in the practices, with support from the RA, recruited a sample of 60 patient subjects for the study, approximately 20 from each practice site. No re-enrollment was permitted. Patients who met the following criteria were eligible for the study: (1) age 50 years of age or older; (2) self-reported diagnosis of heart disease, lung disease, stroke, or cancer, and; (3) capable of speaking and reading English language. Specific exclusion criteria for the study included: (1) severe memory loss or impaired orientation, and; (2) acutely ill appearing. Participating patients received a $10 gift card. The study was approved by the Institutional Review Board of the University of North Carolina at Chapel Hill prior to its initiation.
Data collection and analyses
We used multiple sources of data to explore: (1) the practices' level of engagement in the GEM implementation; (2) the users' (i.e., providers and staff) opinions about their EHR system and expectations for the GEM; (3) the level of use of the GEM information during the patient encounter, and; (4) factors affecting the level of use and decisions about continuing to use the software. Our guiding framework for data collection about EHR opinions and GEM expectations, level of GEM use, and factors affecting GEM use was the Technology Acceptance Model (TAM) [
13,
16]. According to the TAM, two factors influence users' acceptance of new technologies: perceived usefulness and perceived ease of use. Perceived usefulness refers to the extent that a person believes that a particular system would enhance his or her job performance. Perceived ease of use refers to the extent that a person believes that using a particular system would be free of effort [
13,
16] (See Table
1.).
Table 1
Key Concepts and Data Sources
Baseline Survey
| X | X | | |
Direct Observation of Patient and Staff GEM Use
| X | | | |
RIAS Coding of Provider-Patient Interaction
| | | X | |
Semi-structured Interviews with Providers and Staff
| | | | X |
Prior to launching GEM in each practice, we administered a brief questionnaire to the physicians and staff who were likely users of GEM, or whose work potentially would be affected by use of the GEM in the practice. This questionnaire consisted of five-point Likert-type scales to gauge: (1) the perceived values related to HIT and patient care within the practice; (2) the perceived ease of use and usefulness of the practice's existing EHR system; (3) the awareness of the GEM intervention, and; (4) the expected ease of use and usefulness of the GEM. Patient recruitment began after the GEM was installed into each practice's EHR system. After informed consent was obtained, patients responded to the GEM items during the routine intake collection and recording of vital signs and chief complaint into the electronic health record. The RA was available to assist the clinical support staff and/or patient in GEM administration. After the intake collection was completed, the subsequent patient-provider encounter was recorded using a digital audio recorder. Immediately after the visit, the RA collected the audiotape and administered a post-encounter survey to patients, which asked about their use and satisfaction with GEM. We debriefed the RA about observations of the patients' and clinical support staff's engagement with the GEM.
Approximately three months after GEM patient data collection was completed, we conducted individual semi-structured interviews with providers and clinical/administrative staff from each site. A total of 16 interviews were conducted. The interviews explored the facilitators and barriers that users experienced in implementing GEM; their perceptions about the ease of use and usefulness of the GEM; the perceptions about the degree of alignment between the GEM and the users' values, and; recommendations for improving the GEM and promoting its sustainability. The semi-structured interviews were audio-recorded and professionally transcribed. In addition to these interviews, study staff recorded field notes about their general observations of the practice setting, as well as the practices' use of the EHR system.
Our approach to analyzing GEM's value-added included examining the level of engagement in the implementation, reviewing stakeholder opinions of the existing EHR and expectations for the GEM, measuring the level of use of the GEM, and identifying factors that influenced perceptions and use of the GEM. We assessed engagement in the implementation process both quantitatively and qualitatively via baseline surveys and direct observations. To measure provider usage quantitatively, we coded the patient-provider audiotapes using the Roter Interaction Analysis System (RIAS), a widely recognized method of coding doctor-patient interactions during the medical visit [
17]. This analysis allowed us to gauge the level of GEM use at both the provider and practice levels. Specifically, RIAS measured provider references during the encounter to either GEM prompts in the EHR or to the GEM questionnaire administration and patient references to HRQL-related topics and the GEM administration. For provider measures, encounters were categorized by the number of references: 0 = no references, 1 = one reference, and 2 = two or more references. To gain a richer understanding of factors contributing to this variation, we analyzed the interview data, considering also the level of engagement and GEM expectations of each practice.
To assess each practice's satisfaction with their current EHR and expectations regarding GEM implementation, we analyzed data from the baseline surveys of providers, clinical staff, and administrative staff. These baseline questionnaires consisted of five-point Likert-type scales informed primarily by the TAM concepts of usefulness and ease of use. To identify factors affecting post-implementation perceptions and use of the GEM, we analyzed semi-structured interview data from providers and clinical/administrative staff collected approximately 3 months after GEM rollout. An investigator with expertise in health care innovation adoption and implementation (CS) initially coded the qualitative data from the interviews using the TAM framework [
13,
16] as a guide for developing the codebook. The analytic process involved memoing and identifying emergent codes [
18]. It became clear during the case studies, that the providers were sole decision-makers about whether or not to continue using the GEM after the study period. We therefore focused our attention to the providers assessments of value added, which were based on perceived usefulness. To identify themes that provided insight into the factors considered by providers when assessing the value of the GEM and to help ensure internal validity [
19], the entire research team reviewed the texts selected to illustrate the themes. The team, however, did not directly assess coding reliability for the entire interview transcripts.
Conclusion
Implementing a new EHR module that focuses on patient-entered HRQL information within a primary care practice has implications for providers, staff, and patients. From our data, providers' perceptions about the module appear to have the strongest influence on decisions about whether the module should be used in the practice. In addition, our exploratory findings study shed light on the factors providers consider when determining the value-added of an HRQL EHR module.
One framework for understanding the value of a product or service, such as HIT, is to calculate its "quality" divided by "cost" (Value = Quality/Cost) [
20]. Implicitly, the practices' stakeholders (i.e., providers and staff) in this study made a similar assessment about the value added by the GEM. Their primary concern (i.e., the relative advantage of using the GEM vs. the status quo EHR) relates to the GEM's impact on the quality of the service provided during the encounter. For providers, criteria for assessing this impact were the extent to which the GEM information was duplicative of information gathered elsewhere, specific enough to be acted upon, and applicable for enough patients. This was a different assumption by our research study staff since we hypothesized that a perceived benefit of the GEM would be supportive documentation for enhanced billing and coding. However, none of the practices saw this as a benefit, either because there was not a perceived need for it or the GEM was not sufficiently integrated into the billing system. The opportunity for the GEM clearly was in the realm of improving the quality of patient care.
The secondary factors that emerged (i.e., EHR integration, nursing workflow, and patient reluctance) relate to the cost of using the GEM once implemented. For integration, the cost is lost time during an encounter due to inadequate integration of the module into the EHR (or the cost of achieving better integration, if possible). Successful integration of the module is made even more difficult when providers do not perceive the original functions of the EHR system (e.g., history of present illness, medications list, etc.) to be well integrated. This lack of integration leads to fragmentation of the EHR and, subsequently, to "cutting and pasting" of information and/or toggling between screens. In such cases, adding the HRQL module runs the risk of causing further EHR system fragmentation.
For staff and patients, such a module can affect workflow and communication patterns during the visit. If nursing staff must spend more time asking the HRQL questions or assisting patients with data entry of their responses, there could be a negative impact on patient throughput and, ultimately, on staff job stress and satisfaction. Furthermore, there is some evidence that staff satisfaction is positively correlated with patient satisfaction [
21]. Clearly, a perception that the GEM could result in increased staff turnover costs or lost business for the practice would be a significant barrier to implementation. In summary, the cost concerns illustrate the need for any HRQL module to align with the current EHR system design, nursing staff capabilities and workflows, and patient preferences.
The findings from this study inform future research on EHR implementation in a several ways. First, the Technology Acceptance Model concept "usefulness" is multi-dimensional and context dependent. For example, a technology may be useful because it increases an individual's effectiveness or efficiency, or both, depending on the purpose of the technology, the user's role, and the setting [
22]. The criteria providers used to determine the impact of the GEM on the quality of care delivered (i.e., information duplication, specificity, and applicability to enough patients) could assist researchers with operationalizing "usefulness" in the context of HRQL EHR modules. Second, the providers' cost concerns illuminate a range of costs that must be mitigated in order for an HRQL module to be accepted. These findings are consistent with those of Rogausch and colleagues [
10], as the costs all relate to time constraints. Clearly, the HRQL module must be well integrated into both the EHR and into the workflow. From a research perspective, these are distinct but related concerns and both should be measured. Third, this study illustrates that EHR module implementation is influenced by both organizational (i.e., practice-level) and individual (i.e., provider-level) factors [
23]. Even practices that are similar across important characteristics (e.g., size and years experience with EHRs) will have substantial variation in priorities and workflows. Providers within the same practice may have different views about their EHR system and the need for an HRQL module. Future research should include both the practice and provider levels of analysis. Furthermore, HRQL interventions may need to be customizable to meet the needs of different practices and providers.
This study had a few limitations. First, the project team was responsible for developing the GEM module, recruiting participating practices, assisting with implementation, and assessing the module (e.g., uptake and provider satisfaction with it). This involvement potentially could have resulted in a positive response bias, but this does not appear to have occurred in this case. However, assisting with implementation and data entry into the GEM may have resulted in an easier implementation process for the study participants than for practices that might implement the GEM without such support. Second, not having the nursing staff in Practice C available to participate in the study was a data limitation for assessing the impact of the GEM on workflow in that practice. However, the rationale provided by the practice for not including the nursing staff was useful data in itself for assessing the practice's culture and level of engagement in the GEM implementation. Third, while we were able to code audio data of provider-patient communication to assess the level of GEM usage, we were not able to observe how the provider accessed the GEM in the EHR. Differences in how providers incorporated the GEM into their workflow may have affected their assessment of its ease of use and the value of its information. Fourth, the providers were not provided with information about patients' perceptions of the value of the HRQL module, which generally were positive. Providers' considerations of value added might have been different if they were aware of these patient perceptions. Finally, since the study evaluated the feasibility of implementing the GEM in only three practice sites, the findings may not reflect all important factors considered when assessing the value added of an HRQL EHR module in primary care practices. However, the study provides a foundation for larger studies aiming for generalizability. Our study also identifies issues to consider for developers of HRQL modules [
24].
In summary, our findings suggest that any EHR enhancement must have perceived value that justifies the investment. Plausibly, a benefit of an HRQL module is improved quality of patient care; however, in busy practice settings, we found that providers and staff are skeptical of adding another activity to complete during the patient visit. In addition, some providers may not believe that including structured HRQL information in the EHR is the most effective method for improving patient centeredness. Therefore, the benefit of new EHR modules must not only be present, it must be prominent for providers and align with their priorities and workflows. For some practices, this prominence perhaps could be achieved by framing the HRQL module as a method for obtaining external incentives (e.g., Meaningful Use of EHRs). However, achieving alignment with priorities and workflows is still a complex, context-dependent process.
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
CS participated in designing the study, led analysis of qualitative interview data, and drafted the manuscript. JH participated in designing the study, debriefed the research assistant to capture observational data, validated themes identified in the qualitative analysis, and assisted with editing the manuscript. DR provided analysis of quantitative data and assisted with editing the manuscript. TD participated in designing the study, validated themes identified in the qualitative analysis, and assisted with editing the manuscript. All authors read and approved the final manuscript.