An importance-performance analysis of hospital information system attributes: A nurses' perspective

https://doi.org/10.1016/j.ijmedinf.2015.10.010Get rights and content

Highlights

  • Importance-performance analysis is demonstrated to be useful in HIS evaluation.

  • HIS attributes requiring managerial intervention are revealed.

  • Data quality attributes were found most important to productivity of nurse users.

  • System quality was found most important to the satisfaction of nurse users.

  • User support is important to the productivity and satisfaction of nurse users.

Abstract

Purpose

Health workers have numerous concerns about hospital IS (HIS) usage. Addressing these concerns requires understanding the system attributes most important to their satisfaction and productivity. Following a recent HIS implementation, our objective was to identify priorities for managerial intervention based on user evaluations of the performance of the HIS attributes as well as the relative importance of these attributes to user satisfaction and productivity outcomes.

Procedures

We collected data along a set of attributes representing system quality, data quality, information quality, and service quality from 154 nurse users. Their quantitative responses were analysed using the partial least squares approach followed by an importance-performance analysis. Qualitative responses were analysed using thematic analysis to triangulate and supplement the quantitative findings.

Main findings

Two system quality attributes (responsiveness and ease of learning), one information quality attribute (detail), one service quality attribute (sufficient support), and three data quality attributes (records complete, accurate and never missing) were identified as high priorities for intervention.

Conclusions

Our application of importance-performance analysis is unique in HIS evaluation and we have illustrated its utility for identifying those system attributes for which underperformance is not acceptable to users and therefore should be high priorities for intervention.

Introduction

Information systems (IS) have the potential to address numerous problems in healthcare delivery, patient safety, and clinical practice. This may be especially so if IS can be successfully implemented at the point of care [11], [17]. Unfortunately, health workers have numerous concerns about IS usage and its implications for their work [5], [6], [7], [37], [50], [59], [58]. Problems with health workers' acceptance and satisfaction are now regarded among the most significant barriers to the diffusion of IS within health settings [15], [34], [53]. Tracking the needs of health workers and understanding the attributes of an IS that are most important to their satisfaction and productivity have thus become important components within health IS evaluation and research [4].

An IS user's evaluation of system attributes can influence their affective satisfaction and in turn their usage behaviours [19]. More satisfied users have been associated with deeper levels of engagement with a system's functionality [3], [31], [35], which is important to achieving higher-order benefits from IS implementations [20]. In the hospital context, nurses comprise the largest group of workers and, as generators and users of health information, need to interact frequently with hospital IS systems [61]. Therefore, improving nurses' satisfaction with hospital IS (HIS) and ensuring that HIS embed attributes necessary for nurse productivity are critical issues for hospital administrators and HIS providers. However, there are few studies in the hospital context that examine how multiple IS system attributes correlate with both user satisfaction and productivity outcomes for nurses (see [12], [29]). Past studies have also not adequately examined the relative importance of different HIS attributes to nurse-user outcomes in a manner that distinguishes between those attributes for which underperformance is acceptable to nurse users, those attributes where high performance must be maintained, and those attributes that must be prioritized for intervention. One recent attempt to prioritise HIS attributes from the nursing perspective used a fuzzy analytic hierarchy approach [36]. However, this approach is based on a very small sample size and does not identify priorities with reference to a criterion variable. Without understanding the system attributes most important to users' satisfaction and productivity, efforts to improve HIS performance cannot be effectively prioritized. Consequently, there is risk that the intended impacts of HIS on nursing outcomes will be compromised.

The purpose of this study is therefore to identify priorities for managerial intervention by determining which attributes of an integrated hospital information system (HIS) are most important to the satisfaction and productivity of nurses who use the system in day-to-day clinical practice. Our study is undertaken within an importance-performance analysis (IPA) framework [42]. We utilize both quantitative and qualitative data collected from a public hospital in South Africa to understand nurses' evaluations of the performance of the hospital information system along the identified attributes, and the relative importance of these attributes to nurses' satisfaction with the system and its impacts on their productivity. Our results identify those system attributes requiring the most immediate attention, which can help health care administrators to direct scarce resources in a manner that reduces the risks of system failure, user frustrations and resentment.

Section snippets

Conceptual background

The attribute-level performance of an IS has long been recognized as fundamental to user satisfaction and IS success outcomes (e.g. [9], [19], [33], [62]. The DeLone and McLean model of IS success provides a particularly useful organizing framework for understanding system attributes relevant to users. According to this model, attributes along dimensions of technical system quality, stored data quality, information output quality, and support service quality are important to a user's evaluation

Study context

The empirical setting for our study is a public hospital in South Africa. The hospital has regional hospital status and had approximately 570 beds at the time of data collection with a staff complement of about 2200. The hospital services include general surgery, orthopaedics, ENT, ophthalmology, general medicine, radiotherapy and oncology, coronary care, high care, obstetrics/maternity gynaecology, paediatrics, as well as a number of support services including inter-alia radiology, laboratory,

Sample profile

Ninety percent of the respondents were female, which is generally reflective of the nursing population in South Africa. Over 50% of the nurses have more than 10 years of experience in nursing. About 40% have spent more than 10 years at the hospital under study, with 90% having spent at least four or more years at the hospital. The nurses surveyed cover a large area of hospital activity. Of the responses, 34.4% were from general medicine wards, 11.7% from surgical wards, 8.4% from intensive and

Discussion and conclusion

Health workers have numerous concerns about IS usage. Addressing these concerns requires understanding of the system attributes most important to their satisfaction and productivity. Our study examined nurses' evaluations of the performance of an integrated hospital information system along a set of attributes representing system quality, data quality, information quality, and service quality. We also considered the relative importance of these attributes to nurses' satisfaction with using the

Conflicts of interest

No conflicts to declare.

Authors' contributions

Authors contributed equally to the conceptualization of the study. Cohen wrote the paper and was predominantly responsible for carrying out the quantitative analysis. Coleman wrote the paper and was predominantly responsible for carrying out the qualitative analysis. Kangethe was responsible for data collection and this work formed part of his Master's research studies. Cohen and Coleman also finalized the paper for submission.

Summary table

What was already known on the topic?

  • Health workers have

Acknowledgement

The study was supported in part by funding provided to the first author from the National Research Foundation, South Africa.

References (67)

  • T.T. Lee et al.

    Two-stage evaluation of the impact of a nursing information system in Taiwan

    Int. J. Med. Inform.

    (2008)
  • F.Y. Pai et al.

    Applying the technology acceptance model to the introduction of healthcare information systems

    Technol. Forecasting Social Change

    (2011)
  • E.S. Patterson et al.

    Comparison of extent of use, information accuracy, and functions for manual and electronic patient status boards

    Int. J. Med. Inform.

    (2010)
  • L. Poissant et al.

    The impact of electronic health records on time efficiency of physicians and nurses: a systematic review

    J. Am. Med. Inform. Assoc.

    (2005)
  • G. Torkzadeh et al.

    The development of a tool for measuring the perceived impact of information technology on work

    Omega

    (1999)
  • V.P. Aggelidis et al.

    Methods for evaluating hospital information systems: a literature review

    EuroMed J. Bus.

    (2008)
  • H. Ayatollahi et al.

    Paper-based versus computer-based records in the emergency department: staff preferences, expectations, and concerns

    Health Inform. J.

    (2009)
  • H. Ayatollahi et al.

    Factors influencing the use of IT in the emergency department: a qualitative study

    Health Inform. J.

    (2010)
  • D.R. Bacon

    A comparison of approaches to importance-performance analysis

    Int. J. Market Res.

    (2003)
  • J.E. Bailey et al.

    Development of a tool for measuring and analyzing computer user satisfaction

    Manage. Sci.

    (1983)
  • A. Bhattacherjee

    Understanding information systems continuance: an expectation-confirmation model

    MIS Q.

    (2001)
  • D. Blumenthal et al.

    Information technology comes to medicine

    N. Eng. J. Med.

    (2007)
  • R.G. Booth

    Examining the functionality of the DeLone and McLean information system success model as a framework for synthesis in nursing information and communication technology research

    Comput. Inform.Nurs.

    (2012)
  • S.A. Brown et al.

    Do I really have to? User acceptance of mandated technology

    Eur. J. Inf. Syst.

    (2002)
  • B. Chaudhry et al.

    Systematic review: impact of health information technology on quality, efficiency, and costs of medical care

    Ann. Intern. Med.

    (2006)
  • S.K. Chow et al.

    Nurses' perceptions and attitudes towards computerisation in a private hospital

    J. Clin. Nurs.

    (2012)
  • W.H. DeLone et al.

    Information systems success: the quest for the dependent variable

    Inf. Syst. Res.

    (1992)
  • W.H. DeLone et al.

    The DeLone and McLean model of information systems success: a ten-year update

    J. Manage. Inf. Syst.

    (2003)
  • G.G. Gable et al.

    Re-conceptualizing information system success: the IS-impact measurement model

    J. Assoc. Inf. Syst.

    (2008)
  • N. Ghahramani et al.

    User satisfaction with computerized order entry system and its effect on workplace level of stress

    J. Med. Syst.

    (2009)
  • D. Gruber et al.

    Factors influencing outcomes of clinical information systems implementation: a systematic review

    Comput. Inform. Nurs.

    (2009)
  • A. Gustafsson et al.

    Determining attribute importance in a service satisfaction model

    J. Serv. Res.

    (2004)
  • J.F. Hair et al.

    PLS-SEM: indeed a silver bullet

    J. Mark. Theory Pract.

    (2011)
  • Cited by (64)

    • Identifying the value of a clinical information system during the COVID-19 pandemic

      2023, Technovation
      Citation Excerpt :

      This study adopted a pragmatic approach using a sequential mixed-methods design that comprised a qualitative study to inform questions for a quantitative study. Studies on health sciences have previously used mixed research methods to achieve an accurate and comprehensive interpretation from empirical research (Cohen et al., 2016; Campos et al., 2017). This exploration is conducted in three phases.

    • University librarians' perspectives on an importance-performance analysis of authentication system attributes and their attitudes towards authentication log visualization

      2022, Journal of Academic Librarianship
      Citation Excerpt :

      In addition, 17% of them had worked in the library for less than 5 years, 19% for 6–10 years, 12% for 11–15 years, 22% for 16–20 years, and 30% for more than 20 years. The scales used in this study consisted of three sections, which were adapted from previous studies, including the scales developed by Cohen et al. (2016) and Wang and Wang (2009). The first section involves university librarians' demographic data.

    • Importance – Performance Analysis (IPA) of bus service attributes: A case study in a developing country

      2020, Transportation Research Part A: Policy and Practice
      Citation Excerpt :

      Allocation of further budget and considerable effort is fundamental to improving these attributes. The application of Importance – Performance Analysis (IPA) for deriving managerial actions has been used in various fields including tourism (Boley et al., 2017; Lai and Hitchcock, 2016; Pan, 2015; Randall and Rollins, 2009; Ziegler et al., 2012), healthcare (Abalo et al., 2007; Cohen et al., 2016; Hawes and Rao, 1985; Mohebifar et al., 2016), education (Alberty and Mihalik, 1989; Nale et al., 2000; O’Neill and Palmer, 2004; Phadermrod et al., 2019), public administration (Hua and Chen, 2019; Keith and Boley, 2019), banking (Joseph et al., 2005; Matzler et al., 2003), e-business, technology and IT (Chen and Ann, 2016; Magal and Levenburg, 2005; Skok et al., 2001), food services (Tontini and Silveira, 2007), the automotive industry (Matzler et al., 2004), the building industry (Lee, 2019) and sport centres (Rial et al., 2008). Validity and reliability issues associated with IPA have also been reviewed in previous researches (Azzopardi and Nash, 2013; Bacon, 2003; Griffin and Hauser, 1993; Gustafsson and Johnson, 2004; Lai and Hitchcock, 2015; Oh, 2001; Sever, 2015).

    View all citing articles on Scopus
    View full text