Background
• The DSS is integrated with Vision EHR system (www.inps4.co.uk/vision) and is triggered by the GP entering the reason for encounter (RfE). |
• Upon being triggered, the DSS presents a list of possible diagnoses, based on the RfE and information in the patient’s record (age, sex and risk factors). The intention is for the list to appear as early as possible, before GPs start gathering any further information. |
• The interface allows the GP to easily code the patient’s signs, symptoms and examinations using a context-sensitive search box, while the order of diagnoses on the list is updated according to user input. |
• For each suggested diagnosis, the GP can view a checklist of associated symptoms and signs, and indicate their presence or absence. |
• At the end of the consultation, all the information is transferred automatically to the patient’s electronic health record, with the correct codes and structure. |
Method
Design and procedure
-
“Do you have any comments about today’s session? Feel free to comment on anything you want”.
-
“Do you think that a computerised diagnostic support tool, integrated with the patient EHR, would have helped with any of the patients today? Which patients?”
-
“Do you think that the diagnostic support tool helped with any of the patients today? Which patients?”
Analysis
Results
Perceptions of GPs
Theme 1. Perceived usefulness of the DSS
“[The tool helped] in all of them to an extent. The prompts were useful, organising your own mind. In general, it widened up more things to consider.” (GP 24)“Yes, the tool helps in elaborating the symptoms more thoroughly, you analyse the different symptoms.” (GP 22)“I think in straightforward cases it doesn’t really help, like in the UTI… Only when uncertainty increases [it helps]. In the last case [colorectal cancer] it helped. I used it as a checklist, could it be cancer?” (GP3)
“I personally don’t think tools help me too much, interfere with my process of thought, would have done better without it, it was unnatural.” (GP 5)“It hindered me, it slowed me down. I already had my working diagnosis in mind. The system had 20 items, I had about 5 in my head.” (GP 15)“[The DSS] didn’t help but did not distract me from the way I thought.” (GP 14)
“Reminded me to ask questions. Served as a prompt - reminded me to ask about blood in stool which I would have forgotten.” (GP6)“You start to think about the unusual things more, compared to without the system. You ask more questions. You start wide and then focus on it, sometimes you need to open up again.” (GP3)“Helpful to refresh your memory about things that are rare…we mainly stick to the phrase: ‘common things are common’.” (GP7)“I think it widens your diagnosis, like the guy that had COPD and then ended up with AS. I would get to the diagnosis, but it would be longer without the tool.” (GP16)Four participants (12%) claimed that they felt more uncertain and reflective with the tool:“I am more reflective with this: have I missed something here?” (GP8)“You ask more questions, more uncertainty even if you were certain.” (GP3)
-
Integration with Vision – nine GPs (26%) mentioned the advantage of integrating the DSS with Vision, indicating that it is well integrated and/or could be better integrated (see suggestions for improvements).“Useful, good to have it in Vision, you can access it quickly.” (GP 10)“It integrates very well with Vision.” (GP6)“[The tool] should be integrated better with Vision, should be a part of Vision like an added box [part of Vision main screen].” (GP 35)
-
Initial list of suggested diagnoses – four GPs indicated that the initial list was useful.“The first bit when you put the reason for encounter and receive the list is a good idea.” (GP14)“I did get used to it and it had benefits, especially the list in the beginning, makes you think.” (GP25)
-
Symptoms associated with each diagnosis – eight GPs (23.5%) found useful the option to click on a suggested diagnosis, view its associated symptoms and signs and indicate their presence or absence.“There were a lot of questions I wrote in free-text but the system didn’t get it. I then ticked the checkboxes under the diagnosis - which was very useful.” (GP33)“It was really easy and efficient clicking in the list and asking the relevant symptoms.” (GP7)“It was helpful in one of the cases actually, a symptom I forgot to ask. In the last case also – ‘being comfortable when lying flat’ - even if it was negative.” (GP 17)Most GPs (31/34, 91%) clicked on a suggested diagnosis to view the associated symptoms and signs at least once in the DSS session.
PSSUQ question | Mean (SD) |
---|---|
Q1 Overall, I am satisfied with how easy it is to use this system. | 4.85 (1.42) |
Q2 It was simple to use this system. | 4.88 (1.63) |
Q3 I could effectively complete the tasks and scenarios using this system. | 4.47 (1.62) |
Q4 I was able to complete the tasks and scenarios quickly using this system. | 3.82 (1.31) |
Q5 I was able to efficiently complete the tasks and scenarios using this system. | 4.41 (1.42) |
Q6 I felt comfortable using this system. | 4.53 (1.58) |
Q7 It was easy to learn to use this system. | 5.47 (1.38) |
Q8 I believe I could become productive quickly using this system. | 4.27 (1.68) |
Q9 The system gave error messages that clearly told me how to fix problems. | 3.26 (1.73) |
Q10 Whenever I made a mistake using the system, I could recover easily and quickly. | 4.39 (1.67) |
Q11 The information (such as on-line help, on-screenmessages and other documentation) provided with this system was clear. | 4.95 (1.59) |
Q12 It was easy to find the information I needed. | 4.69 (1.55) |
Q13 The information provided for the system was easy to understand. | 5.22 (1.34) |
Q14 The information was effective in helping me complete the tasks and scenarios. | 4.53 (1.34) |
Q15 The organization of information on the system screens was clear. | 4.88 (1.62) |
Q16 The interface of this system was pleasant. | 5.21 (1.39) |
Q17 I liked using the interface of this system. | 4.88 (1.49) |
Q18 This system has all the functions and capabilities I expect it to have. | 4.09 (1.59) |
Q19 Overall, I am satisfied with this system. | 4.18 (1.55) |
Theme 2: Impact of the DSS on the consultation
“It’s completely different to how I normally work, unsettling and confusing, I’m not quite sure where I am, to what extent I’m following a template and how I take history.” (GP8)“You need to get used to it…I do my consultations in a different way, but it works quite quick.” (GP12)“It felt different, I definitely used different questions. From a doctor’s point of view it’s good, from the patient’s point of view it’s weird…I would have asked more open questions relative to closed ones.” (GP16)“It throws my standard history examination style, stop me from doing unnecessary investigations.” (GP 9)
“I normally chat and look at the patients, it throws my normal thing.” (GP9)“I usually don’t code during the consultation, less contact with patient, my style is to listen for a long time.” (GP3)
“Using the tool in the present format is time consuming, need to speed it up.” (GP7)“It will be hard to use it in a 10-min. consultation.” (GP22)“It was too time consuming selecting all the symptoms, you need to minimise interaction during consultation, I would group relevant symptoms together.” (GP13)
“Obviously familiarity over time will improve.” (GP19)“I can see its use. It’s a new thing, with any new system the initial use will affect the customer relationship. As it was going on, it was getting better.” (GP26)“I did get used to it and it had benefits.” (GP25)
Theme 3: Suggestions for improving the DSS
“One comment box, write everything you want, using a smart technology, the system could read everything you wrote and come up with a diagnosis on its own.” (GP 15)“Why can’t the system analyse free text?” (GP 6).
“Advising what tests one should do; if for example there is blood in sputum - suggest chest x-ray - possible investigations.” (GP 11)“If the patient came for the same reason in the last 6 months - indicate it in the system.” (GP 6)
“After you select a diagnosis, for example, asthma the system should ask have you done this, a table at the end, for example, saturation, BP, Pulse and what is missing. What legally I have to do in Asthma.” (GP 9)“[The DSS] should integrate with QoF data, for example, COPD performance management domain instead of entering again [the data] to the QoFsystem.” (GP 13)
“Group examinations, for example respiratory symptoms, same for abdominal. abdo pain – yes/no, if yes then mass/guarding/PR examination and then the diagnoses on the right will change and display.” (GP 33)“Add prevalence, probabilities. The tool didn’t help in that, not enough to flash - it can be cancer, but add probabilities, it could be cancer above 30%. Move them up the list and make you do something.” (GP 29)
Patients satisfaction with the consultation
“There were times when the doctor was talking to me, not needing to check anything in the computer, but still looking at it.” (SP2)“The technology was in the way of his communication with me.” (SP1)