Background
There is an increasing use of electronic patient record (EPR) systems in healthcare. The UK government has a target for a comprehensive electronic health system in England by 2020 [1].
Clinicians use EPR systems to make treatment decisions. It is essential that the information presented in these electronic visualisations is usable—efficient, effective and satisfying. If clinical data are not displayed in a usable format, this can compromise care, for example, by leading to delays in diagnosis or treatment.
Anzeige
A wide variety of visual techniques are used to display clinical data. Usability tools are used for their assessment. How and why individual tools have been used has not been studied. Given the heterogeneity of systems, it is unlikely that one assessment tool would allow accurate assessment or comparison of them all.
A systematic review published in 2014 looked at the use of visualisation techniques and evaluated innovative approaches to information visualisation of electronic health record data [2]. Whilst this review acknowledged the need for a usable design (one of the themes identified), it did not look at the tools and techniques to assess the usability of the different information visualisation techniques.
Aim
This systematic review aims to identify and systematically review the methods used to assess the usability of information visualisations of patient data for clinical use. The review will for the first time summarise the methods used to assess usability of information visualisations of individual patient data for clinical use. The strengths and weaknesses found for different techniques in different situations will be discussed. This will allow those developing such software to undertake development in light of the assessment methods that they should use to ensure quality software. It will also allow customers to ensure products they require are suitably assessed and meet their needs.
Methods/design
This protocol adheres to the requirements of Preferred Reporting Items for Systematic Review and Meta-analysis Protocols (PRISMA-P) [3] which is included as Additional file 1.
Anzeige
Eligibility criteria
Studies satisfying all the criteria below will be included:
-
Describes a method of electronic information visualisation used to display patient data AND includes an assessment of the information visualisation
-
Is related to electronic visualisations either prototype or used in clinical practice
-
Is related to methods for evaluating usability of the visualisation technique used to display the medical data
-
Displays individual patient data, not just patient cohort data
-
Is for clinician’s use only and not for patient access
-
Is not related to dentistry
Exclusions
Systematic reviews will not be included in the review, but appropriate studies referenced in reviews will be included. Correspondence and short communications will be excluded. Doctoral research will be included. As our systematic review aims to identify which methods have been used and where these were informative, we will not include protocols without usage data.
Study design
The study design is a systematic search of the medical literature followed by a narrative synthesis of the results.
Setting
Individual patient data display is implemented in any healthcare settings where it is used for clinical use.
Time frame
There will be no restriction placed on the time frame of the studies.
Years considered
Studies published from 1996 onwards will be considered.
Language
No language restrictions will be applied.
Information sources
Literature search strategies will be developed using Medical Subject Headings (MeSH) and text words related to the usability of information visualisation methods for displaying patient medical data for clinical use.
Anzeige
The following databases will be searched: MEDLINE (Ovid), EMBASE (Ovid), CINAHL (HDAS), OpenGrey and the Cochrane Database of Systematic Reviews (Wiley).
Reference lists of eligible studies and relevant reviews will be explored to identify further eligible studies.
Search strategy
A draft of the search strategy was developed by three of the authors (AB, LM and TP, a medical librarian). The proposed search strategy is shown in Additional file 2.
Study records
Data management
The citations and full text of papers identified from the search will be stored using Mendeley. A data extraction form will be developed in Excel and piloted.
Anzeige
Literature search results will be uploaded to Covidence, an online software program designed to improve the production of systematic reviews.
Selection process
Two assessors will independently screen title and abstracts of papers returned by the search against the inclusion criteria. If there is uncertainty from the abstract, the full text will be reviewed. Papers selected on title and abstract will be full-text screened for eligibility. Eligible papers will be included in the study.
Disagreements about eligibility will be resolved by a third party. We will record the reasons for excluding the studies.
The study selection process is illustrated by the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram in Additional file 3.
Anzeige
Data collection process
Data extraction forms will be created using Excel and piloted prior to use. Two reviewers will independently extract data from the full text of eligible papers. Any uncertainties regarding data extraction will be resolved by discussion amongst the authors.
Data items extracted
We will extract the following data items from each publication:
Study characteristics:Period of data collectionPatient demographicsStudy settingOutcome measures
-
Date of study
-
Type of study design
-
Clinician/users of system demographics
-
Type of task visualised
-
Country of study
Display data:
Information contained in the display, including for example:
-
Static: patient demographics, admission history, past medical history
-
Semi static: ward/bed number
-
Dynamic: laboratory data, vital signs, neurological status, respiratory status, cardiovascular status, assessment/warning scores, drug charts, fluid charts and medical notes of various clinician groups
Visualisation techniques/display categories as described by Starren et al. [4]:TableIcons (atomic icon/iconic language)
-
List (simple/nested)
-
Graph (simple chart/configural chart/graph notation)
-
Generated text
The following display design features will be identified and recorded:
-
Interaction level: display, input, alerting and messaging capabilities
Some publications may refer to more than one category of visualisation technique or type of information. Multiple data points will be captured where relevant. The pilot phase of the data extraction form may identify other relevant information which will also be collected.
Usability data
We will extract data on how usability was assessed in the following categories:Usability assessmentInter-rater performance of toolAny assessment of the tool’s user-friendliness
-
Usability assessment technique/tool type
-
Test/retest performance of tool
-
Outcome scale generated
-
Specific data required by the tool
We will also extract data, where available, on whether the tool records the following categories suggested by Kopanitsa et al. [5]:
-
Efficiency: time to complete tasks (% of tasks fully completed, % of tasks half completed); comparing task completion quality using software/new software compared to without software/previous software
-
Effectiveness: % of errors; surveys on % of participants’ responses to task completion; comparing task completion quality using software/new software compared to without software/previous software
-
Satisfaction: % of participants who make positive/negative comments about the system
This list of performance metrics is not exhaustive, and other measures identified in the literature will also be used.
Outcomes and prioritisation
The primary outcome will be to document methods employed to assess the usability of electronic information visualisations of patient data for clinical use.
Secondary outcomes will include a narrative comparison of the identified methods.
Risk of bias of individual studies
The methodological quality of the studies will be assessed using a modified Downs and Black (D&B) checklist [6]. This checklist was designed to provide an evaluation of the quality of both randomised and non-randomised studies of healthcare interventions on the same scale.
The D&B checklist will be modified to create a 20-question checklist by omitting questions 5, 9, 12, 14, 17, 25 and 26. These questions are deemed not appropriate for assessing the methodological quality of the studies which will be evaluated.
Data synthesis
We anticipate that the data extracted will be most appropriate for qualitative synthesis. This will be presented as a narrative synthesis. Themes will be identified in the text, and tables will be used to summarise and explain the characteristics/findings of the included studies.
Discussion
Currently, there are a variety of information visualisation methods used to present electronic medical data for clinical use [2]. However, there is minimal evidence in the literature regarding assessments of the usability of these systems, despite a clear need to be able to do this effectively.
To date, we are unaware of any systematic review exploring the methods used to assess the usability of electronic information visualisations of patient data for clinical use. This review will summarise the similarities and differences between methods used to assess the usability of electronic information visualisations of patient data for clinical use. We will present the differences between the methods found. Our results will inform best practice when developing new user interfaces to display electronic patient data for clinical use.
Limitations
The systemic review will be limited by the quality of the data available.
Funding
This systematic review protocol is funded by the Department of Health and Wellcome Trust through the Health Innovation Challenge Fund.
This publication presents independent research commissioned by the Health Innovation Challenge Fund (HICF-R9-524; WT-103703/Z/14/Z), a parallel funding partnership between the Department of Health and Wellcome Trust. The views expressed in this publication are those of the author(s) and not necessarily those of the Department of Health or Wellcome Trust.
Availability of data and materials
The datasets supporting the conclusions of the systematic review article will be included within the article and in the additional files.
Consent for publication
Consent for publication is not applicable as no individual person’s data is contained in the paper.
Authors’ information
AB has a Masters in Ergonomics (Human Factors). LM has a PhD in Ergonomics (Human Factors).
Ethics approval and consent to participate
Name of ethics committee: Oxfordshire Research Ethics Committee C (REC); Confidentiality Advisory Group (CAG)
Reference number: REC: 16/SC/0264; CAG: 16/CAG/0066.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.