Principal findings
According to the perspectives 44% of participants in this study, the efficiency and effectiveness of the surgery information system were acceptable. Also, there was no significant difference between the efficiency and effectiveness of the surgery information system in the two companies. About 30% of users reported that they can learn how to use the system in a short period of time, though they need some training to use it successfully. Moreover, more than 50% of them claimed that using this system can improve the confidentiality and accuracy of patient’s information as well as the completion of documents. The findings of the present study showed that the users irrespective of their age, sex, education, experience of working with computers, computer skills, and experience of working with surgery information systems have the same viewpoint about the efficiency and effectiveness of the system.
In our study, 44.7% of the participants rated the system’s effectiveness as high and 48.5%of them rated its efficiency as high, as well as 44% of participants believed that the system effectiveness and efficiency were acceptable. In this regard, Meulendijk [
19] evaluated a decision support system and showed that users performed their tasks better using the system compared to traditional method, but they sometimes made mistakes when working with the system. Furthermore, due to the users’ unfamiliarity with the user interface, it took more time to get tasks done by the system, leading to low efficiency. Hence, participants indicated that the system was marginally acceptable. Findings of a review study [
20] that evaluated three aspects (effectiveness, efficiency, and satisfaction) of the clinical decision making support system showed a positive effect of this system on efficiency and effectiveness. Also, Treadwell study [
21] assessed the usability of the personal digital assistant system for clinical examinations. According to the results, the effectiveness of this system was not different in the effectiveness of the paper-based system, but its efficiency was higher. Farrahi in a study [
22] measuring the relationship between user interface problems of an admission, discharge, and transfer (ADT) module with usability attributes, found that the efficiency and effectiveness of that system were unacceptable. This was due to the user interface issues such as unclear function of some keys, lack of “help” option, and the existence of hidden menus in the system. These issues prevented a successful interaction of the users with system wasted their time. Findings of Sheehan study [
23] also indicated that quality of a system interface can influence the duration of work and consequently the system performance. Moreover Guo study [
24] showed that system interface problems reduce the efficiency because users have to devote more time to fulfill a task.
The results of the study showed a significant relationship between the efficiency and effectiveness of surgery information systems. The findings of the present study are in line with the results of the study conducted by Erik [
25] concerning the relationship between the three usability attributes (efficiency, effectiveness, and satisfaction). Findings of his study also showed a meaningful correlation between the efficiency and effectiveness attributes. Ramadan [
26], in a study evaluating the usability of geographic information system (GIS), showed that although the effectiveness score was higher than the efficiency score, consistent with our results, there was a significant relationship between these two attributes. In a study conducted by Georgsson [
2] participants who did their tasks successfully were also quicker in performing them.
In our study, 51% of the participants evaluated high rate of the use of the surgery information system to increasing the precision of recording patients’ information. Increased accuracy in data recording reduces errors and re-work, which can decrease the time required to perform tasks and improve the system efficiency. Consistent with this finding, Maat study findings [
5] showed that after performing the computerized physician order entry (CPOE) in the intensive care unit (ICU), the glucose calculation accuracy increased. Use of this system, in comparison with the manual computation, reduced the computation time.
In the present study, about 48% of the users indicated that the physicians’ errors reduced highly when using the system. Also, 34% of the participants believed that this system improved patient safety up to a medium level This finding is consistent with the results of a study by Hsieh [
27] in which the use of mobile electronic medication administration record (ME-MAR) system reduced medication errors by nurses and consequently increased patient safety. About 37% of our study participants mentioned that this system could improve patient satisfaction at a moderate level. This finding is congruent with the results of Yilmaz’s study that investigated the use of an architectural system for the development of smart information systems [
28].
Based on the results, about 35% of the participants rated the system’s ease of use as high. Since system ease of use saves the time users spend on the learning and using a system, it can affect the efficiency of that system. Therefore, the design of such systems should be simplified in their development phase to improve their efficiency. In this regard, a study conducted by Lee [
29] regarding the efficiency of the nursing care planning system showed that ease of use of the system improved efficiency and satisfaction of users. Furthermore, the study by Khajouei [
30] also showed that physicians were satisfied with CPOE ease of use and its effect on medication safety, workflow, and efficiency. Likewise, nurses also had a positive attitude towards it. In the present study, the 45%of the participants mentioned their need for training to use the system. This may not be the case for other systems. For example, Alnasser [
31] in a study which investigated the usability of Twazon Arabic Weight Loss App showed that a large number of users did not need to learn the app before using it. The majority of participants in this study believed that the surgery information system improves communication among specialists (40%) and clinical wards (55%). Since the aim of this communication is to provide care for the patients, the facilitation of communication can improve patient care [
32]. Sturzlinger [
6], in a study concerning the efficiency and effectiveness of CPOE with integrated decision support systems, highlighted the improvement of communication and patient care after implementation of this CPOE. Georgiou [
33] in a study revealed that the use of CPOE and clinical decision support system (CDSS) reduced the turnaround time for laboratory test results and improved patient care as a result.
Fifty-six percent of the participants in this study noted that surgery information systems increase the confidentiality of information. This finding is in line with the results of Dale’s study [
34] on the investigation of the privacy, confidentiality, and security of information in electronic information systems. Participants in our study asserted that surgery information systems increase the speed of completing documents. This finding is consistent with the results of previous studies [
9,
35].
Limitations
This study had three limitations. First, the efficiency and effectiveness of the system were evaluated in three university hospitals. More accurate results can be obtained by choosing a bigger sample size form a broader environment. However, this research was carried out on two surgery information systems that are currently used in 303 university hospitals in Iran. Second, since the study was conducted on an already implemented system, it was not possible to compare the effectiveness and efficiency of the surgery information system with the traditional system. Yet, the findings of this study can be used by subsequent studies that plan to compare efficiency and effectiveness of the systems before and after implementation. Third, this study examined the efficiency and effectiveness of two surgery information systems (Tirazheh and Peyvand Dadeh). Although the surgery information systems developed by different vendors may have different efficiency and effectiveness based on each questionnaire component, due to the following reasons this may not have a major impact on our findings: a) the efficiency and effectiveness of these two systems were not significantly different, b) since these two systems have the same purpose and are designed for the same goal, they are similar in terms of interface features and capabilities and c) this study was conducted in three teaching hospitals affiliated with the same university, hence, they have almost similar staff composition, and the technical knowledge of their system users is similar.