Download PDF
CC BY-NC-ND 4.0 · Yearb Med Inform 2023; 32(01): 115-126
DOI: 10.1055/s-0043-1768733
Section 3: Clinical Information Systems
Survey

Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings

Farah Magrabi
Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
,
David Lyell
Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
,
Enrico Coiera
Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
› Author Affiliations Funding This research is supported by the Australian National Health and Medical Research Council Centre for Research Excellence in Digital Health (APP1134919) and Macquarie University. The funding sources did not play any role in study design, in the collection, analysis, and interpretation of data, in the writing of the report, or in the decision to submit the article for publication.
› Further Information
   Permissions and Reprints

Summary

Aims and objectives: To examine the nature and use of automation in contemporary clinical information systems by reviewing studies reporting the implementation and evaluation of artificial intelligence (AI) technologies in healthcare settings.

Method: PubMed/MEDLINE, Web of Science, EMBASE, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies reporting evaluation of AI in clinical settings from January 2021 to December 2022. We documented the clinical application areas and tasks supported, and the level of system autonomy. Reported effects on user experience, decision-making, care delivery and outcomes were summarised.

Results: AI technologies are being applied in a wide variety of clinical areas. Most contemporary systems utilise deep learning, use routinely collected data, support diagnosis and triage, are assistive (requiring users to confirm or approve AI provided information or decisions), and are used by doctors in acute care settings in high-income nations. AI systems are integrated and used within existing clinical information systems including electronic medical records. There is limited support for One Health goals. Evaluation is largely based on quantitative methods measuring effects on decision-making.

Conclusion: AI systems are being implemented and evaluated in many clinical areas. There remain many opportunities to understand patterns of routine use and evaluate effects on decision-making, care delivery and patient outcomes using mixed-methods. Support for One Health including integrating data about environmental factors and social determinants needs further exploration.

Keywords

Clinical information systems - artificial intelligence - automation - implementation - evaluation

Competing Interests

The authors have no competing interests to declare.


Contributions

FM conceptualised the study, undertook the literature search, performed data analysis and drafted the article. All authors participated in writing and revising the article. All aspects of the study (including design; collection, analysis, and interpretation of data; writing of the report; and decision to publish) were led by the authors.


Supplementary Material



Publication History

Article published online:
26 December 2023

© 2023. IMIA and Thieme. This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Coiera E. Guide to health informatics, third edition. CRC Press; 2015.
    PubMed
  • 2 Coiera E. The fate of medicine in the time of AI. Lancet. 2018 Dec 1;392(10162):2331-2332. PubMed PMID: 30318263. Epub 2018/10/16. doi: 10.1016/s0140-6736(18)31925-1.
    PubMed
  • 3 Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019 Jan;25(1):44-56. PubMed PMID: 30617339. Epub 2019/01/09. doi: 10.1038/s41591-018-0300-7.
    PubMed
  • 4 One Health. Available from: https://www.who.int/news-room/questions-and-answers/item/one-health (last accessed: 10 Jan 2023).
    PubMed
  • 5 Prata JC, Ribeiro AI, Rocha-Santos T. Chapter 1 - An introduction to the concept of One Health. In: Prata JC, Ribeiro AI, Rocha-Santos T, editors. One Health: Academic Press; 2022. p. 1-31. doi: 10.1016/B978-0-12-822794-7.00004-6
    PubMed
  • 6 Dalton KR, Rock C, Carroll KC, Davis MF. One Health in hospitals: how understanding the dynamics of people, animals, and the hospital built-environment can be used to better inform interventions for antimicrobial-resistant gram-positive infections. Antimicrob Resist Infect Control. 2020 Jun 1;9(1):78. PubMed PMID: 32487220. Epub 2020/06/04. doi: 10.1186/s13756-020-00737-2.
    PubMed
  • 7 Rahimi-Ardabili H, Magrabi F, Coiera E. Digital health for climate change mitigation and response: a scoping review. J Am Med Inform Assoc. 2022 Nov 14;29(12):2140-2152. PubMed PMID: 35960171. Epub 2022/08/13. doi: 10.1093/jamia/ocac134.
    PubMed
  • 8 Domingos P. The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World: Penguin Books Limited; 2015.
    PubMed
  • 9 Parasuraman R, Sheridan TB, Wickens CD. A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern A Syst Hum. 2000 May;30(3):286-97. PubMed PMID: 11760769. Epub 2002/01/05. doi: 10.1109/3468.844354.
    PubMed
  • 10 Coiera E, Liu S. Evidence synthesis, digital scribes, and translational challenges for artificial intelligence in healthcare. Cell Rep Med. 2022 Dec 20;3(12):100860. PubMed PMID: 36513071. Epub 2022/12/14. doi: 10.1016/j.xcrm.2022.100860.
    PubMed
  • 11 Coiera E. The Last Mile: Where Artificial Intelligence Meets Reality. J Med Internet Res. 2019 Nov 8;21(11):e16323. PubMed PMID: 31702559. Epub 2019/11/09. doi: 10.2196/16323.
    PubMed
  • 12 Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K. Artificial intelligence, bias and clinical safety. BMJ Qual Saf. 2019 Mar;28(3):231-237. PubMed PMID: 30636200. Epub 2019/01/14. doi: 10.1136/bmjqs-2018-008370.
    PubMed
  • 13 Lyell D, Wang Y, Coiera E, Magrabi F. More than algorithms: an analysis of safety events involving ML-enabled medical devices reported to the FDA. J Am Med Inform Assoc. 2023 June 20;30(7):1227-1236. PubMed PMID: 37071804. Epub 2023/04/18. doi: 10.1093/jamia/ocad065.
    PubMed
  • 14 United Nations. Country Classification: Data Sources, Country Classifications and Aggregation Methodology. New York (NY): United Nations; 2014: 143-150. Available from: https://www.un.org (last accessed: 10 Jan 2023).
    PubMed
  • 15 Lyell D, Coiera E, Chen J, Shah P, Magrabi F. How machine learning is embedded to support clinician decision making: an analysis of FDA-approved medical devices. BMJ Health Care Inform. 2021 Apr;28(1):e100301. doi: 10.1136/bmjhci-2020-100301.
    PubMed
  • 16 Coiera E. Assessing Technology Success and Failure Using Information Value Chain Theory. Stud Health Technol Inform. 2019 Jul 30;263:35-48. PubMed PMID: 31411151. Epub 2019/08/15. doi: 10.3233/shti190109.
    PubMed
  • 17 Mohammed NI, Jarde A, Mackenzie G, D'Alessandro U, Jeffries D. Deploying Machine Learning Models Using Progressive Web Applications: Implementation Using a Neural Network Prediction Model for Pneumonia Related Child Mortality in The Gambia. Front Public Health. 2022;9:772620. PubMed PMID: 35252109. Epub 2022/03/08. doi: 10.3389/fpubh.2021.772620.
    PubMed
  • 18 Seol HY, Shrestha P, Muth JF, Wi CI, Sohn S, Ryu E, et al. Artificial intelligence-assisted clinical decision support for childhood asthma management: A randomized clinical trial. PLoS One. 2021;16(8):e0255261. PubMed PMID: 34339438. Epub 2021/08/03. doi: 10.1371/journal.pone.0255261.
    PubMed
  • 19 Wu L, He X, Liu M, Xie H, An P, Zhang J, et al. Evaluation of the effects of an artificial intelligence system on endoscopy quality and preliminary testing of its performance in detecting early gastric cancer: a randomized controlled trial. Endoscopy. 2021 Dec;53(12):1199-1207. PubMed PMID: 33429441. Epub 2021/01/12. doi: 10.1055/a-1350-5583.
    PubMed
  • 20 Peng S, Liu Y, Lv W, Liu L, Zhou Q, Yang H, et al. Deep learning-based artificial intelligence model to assist thyroid nodule diagnosis and management: a multicentre diagnostic study. Lancet Digit Health. 2021 Apr;3(4):e250-e259. PubMed PMID: 33766289. Epub 2021/03/27. doi: 10.1016/s2589-7500(21)00041-8.
    PubMed
  • 21 Calisto FM, Santiago C, Nunes N, Nascimento JC. BreastScreening-AI: Evaluating medical intelligent agents for human-AI interactions. Artif Intell Med. 2022 May;127:102285. PubMed PMID: 35430044. Epub 2022/04/18. doi: 10.1016/j.artmed.2022.102285.
    PubMed
  • 22 Martins Jarnalo CO, Linsen PVM, Blazís SP, van der Valk PHM, Dickerscheid DBM. Clinical evaluation of a deep-learning-based computer-aided detection system for the detection of pulmonary nodules in a large teaching hospital. Clin Radiol. 2021 Nov;76(11):838-845. PubMed PMID: 34404517. Epub 2021/08/19. doi: 10.1016/j.crad.2021.07.012.
    PubMed
  • 23 Quan SY, Wei MT, Lee J, Mohi-Ud-Din R, Mostaghim R, Sachdev R, et al. Clinical evaluation of a real-time artificial intelligence-based polyp detection system: a US multi-center pilot study. Sci Rep. 2022 Apr 21;12(1):6598. PubMed PMID: 35449442. Epub 2022/04/23. doi: 10.1038/s41598-022-10597-y.
    PubMed
  • 24 Ou YC, Tsao TY, Chang MC, Lin YS, Yang WL, Hang JF, et al. Evaluation of an artificial intelligence algorithm for assisting the Paris System in reporting urinary cytology: A pilot study. Cancer Cytopathol. 2022 Nov;130(11):872-880. PubMed PMID: 35727052. Epub 2022/06/22. doi: 10.1002/cncy.22615.
    PubMed
  • 25 Nasir-Moin M, Suriawinata AA, Ren B, Liu X, Robertson DJ, Bagchi S, et al. Evaluation of an Artificial Intelligence-Augmented Digital System for Histologic Classification of Colorectal Polyps. JAMA Netw Open. 2021 Nov 1;4(11):e2135271. PubMed PMID: 34792588. Epub 2021/11/19. doi: 10.1001/jamanetworkopen.2021.35271.
    PubMed
  • 26 Duan X, Su D, Yu H, Xin W, Wang Y, Zhao Z. Adoption of Artificial Intelligence (AI)-Based Computerized Tomography (CT) Evaluation of Comprehensive Nursing in the Operation Room in Laparoscopy-Guided Radical Surgery of Colon Cancer. Comput Intell Neurosci. 2022;2022:2180788. PubMed PMID: 35300396. Epub 2022/03/19. doi: 10.1155/2022/2180788.
    PubMed
  • 27 Giavina-Bianchi M, de Sousa RM, Paciello VZA, Vitor WG, Okita AL, Prôa R, et al. Implementation of artificial intelligence algorithms for melanoma screening in a primary care setting. PLoS One. 2021;16(9):e0257006. PubMed PMID: 34550970. Epub 2021/09/23. doi: 10.1371/journal.pone.0257006.
    PubMed
  • 28 Rabinovich D, Mosquera C, Torrens P, Aineseder M, Benitez S. User Satisfaction with an AI System for Chest X-Ray Analysis Implemented in a Hospital's Emergency Setting. Stud Health Technol Inform. 2022 May 25;294:8-12. PubMed PMID: 35612006. Epub 2022/05/26. doi: 10.3233/shti220386.
    PubMed
  • 29 Chonde DB, Pourvaziri A, Williams J, McGowan J, Moskos M, Alvarez C, et al. RadTranslate: An Artificial Intelligence-Powered Intervention for Urgent Imaging to Enhance Care Equity for Patients With Limited English Proficiency During the COVID-19 Pandemic. J Am Coll Radiol. 2021 Jul;18(7):1000-1008. PubMed PMID: 33609456. Epub 2021/02/21. doi: 10.1016/j.jacr.2021.01.013.
    PubMed
  • 30 Duron L, Ducarouge A, Gillibert A, Lainé J, Allouche C, Cherel N, et al. Assessment of an AI Aid in Detection of Adult Appendicular Skeletal Fractures by Emergency Physicians and Radiologists: A Multicenter Cross-sectional Diagnostic Study. Radiology. 2021 Jul;300(1):120-129. PubMed PMID: 33944629. Epub 2021/05/05. doi: 10.1148/radiol.2021203886.
    PubMed
  • 31 Schmuelling L, Franzeck FC, Nickel CH, Mansella G, Bingisser R, Schmidt N, et al. Deep learning-based automated detection of pulmonary embolism on CT pulmonary angiograms: No significant effects on report communication times and patient turnaround in the emergency department nine months after technical implementation. Eur J Radiol. 2021 Aug;141:109816. PubMed PMID: 34157638. Epub 2021/06/23. doi: 10.1016/j.ejrad.2021.109816.
    PubMed
  • 32 Seah JCY, Tang CHM, Buchlak QD, Holt XG, Wardman JB, Aimoldin A, et al. Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study. Lancet Digit Health. 2021 Aug;3(8):e496-e506. PubMed PMID: 34219054. Epub 2021/07/06. doi: 10.1016/s2589-7500(21)00106-0.
    PubMed
  • 33 Zhang B, Jia C, Wu R, Lv B, Li B, Li F, et al. Improving rib fracture detection accuracy and reading efficiency with deep learning-based detection software: a clinical evaluation. Br J Radiol. 2021 Feb 1;94(1118):20200870. PubMed PMID: 33332979. Epub 2020/12/18. doi: 10.1259/bjr.20200870.
    PubMed
  • 34 Liu X, Wu D, Xie H, Xu Y, Liu L, Tao X, et al. Clinical evaluation of AI software for rib fracture detection and its impact on junior radiologist performance. Acta Radiol. 2022 Nov;63(11):1535-1545. PubMed PMID: 34617809. Epub 2021/10/08. doi: 10.1177/02841851211043839.
    PubMed
  • 35 Lee S, Shin HJ, Kim S, Kim EK. Successful Implementation of an Artificial Intelligence-Based Computer-Aided Detection System for Chest Radiography in Daily Clinical Practice. Korean J Radiol. 2022 Sep;23(9):847-852. PubMed PMID: 35762186. Epub 2022/06/29. doi: 10.3348/kjr.2022.0193.
    PubMed
  • 36 Shiang T, Garwood E, Debenedectis CM. Artificial intelligence-based decision support system (AI-DSS) implementation in radiology residency: Introducing residents to AI in the clinical setting. Clin Imaging. 2022 Dec;92:32-7. PubMed PMID: 36183619. Epub 2022/10/03. doi: 10.1016/j.clinimag.2022.09.003.
    PubMed
  • 37 Hinson JS, Klein E, Smith A, Toerper M, Dungarani T, Hager D, et al. Multisite implementation of a workflow-integrated machine learning system to optimize COVID-19 hospital admission decisions. NPJ Digit Med. 2022 Jul 16;5(1):94. PubMed PMID: 35842519. Epub 2022/07/17. doi: 10.1038/s41746-022-00646-1.
    PubMed
  • 38 Soltan AAS, Yang J, Pattanshetty R, Novak A, Yang Y, Rohanian O, et al. Real-world evaluation of rapid and laboratory-free COVID-19 triage for emergency care: external validation and pilot deployment of artificial intelligence driven screening. Lancet Digit Health. 2022 Apr;4(4):e266-e278. PubMed PMID: 35279399. Epub 2022/03/14. doi: 10.1016/s2589-7500(21)00272-7.
    PubMed
  • 39 Wang YC, Chen KW, Tsai BY, Wu MY, Hsieh PH, Wei JT, et al. Implementation of an All-Day Artificial Intelligence-Based Triage System to Accelerate Door-to-Balloon Times. Mayo Clin Proc. 2022 Dec;97(12):2291-303. PubMed PMID: 36336511. Epub 2022/11/07. doi: 10.1016/j.mayocp.2022.05.014.
    PubMed
  • 40 Kim JH, Han SG, Cho A, Shin HJ, Baek S-E. Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study. BMC Med Inform Decis Mak. 2021 Nov 8;21(1):311. doi: 10.1186/s12911-021-01679-4.
    PubMed
  • 41 Ivanov O, Wolf L, Brecher D, Lewis E, Masek K, Montgomery K, et al. Improving ED Emergency Severity Index Acuity Assignment Using Machine Learning and Clinical Natural Language Processing. J Emerg Nurs. 2021 Mar;47(2):265-278.e7. PubMed PMID: 33358394. Epub 2020/12/29. doi: 10.1016/j.jen.2020.11.001.
    PubMed
  • 42 Jordan M, Hauser J, Cota S, Li H, Wolf L. The Impact of Cultural Embeddedness on the Implementation of an Artificial Intelligence Program at Triage: A Qualitative Study. J Transcult Nurs. 2023 Jan;34(1):32-39. PubMed PMID: 36214065. Epub 2022/10/11. doi: 10.1177/10436596221129226.
    PubMed
  • 43 Wong J, Huang V, Wells D, Giambattista J, Giambattista J, Kolbeck C, et al. Implementation of deep learning-based auto-segmentation for radiotherapy planning structures: a workflow study at two cancer centers. Radiat Oncol. 2021 Jun 8;16(1):101. PubMed PMID: 34103062. Epub 2021/06/10. doi: 10.1186/s13014-021-01831-4.
    PubMed
  • 44 Byun HK, Chang JS, Choi MS, Chun J, Jung J, Jeong C, et al. Evaluation of deep learning-based autosegmentation in breast cancer radiotherapy. Radiat Oncol. 2021 Oct 14;16(1):203. PubMed PMID: 34649569. Epub 2021/10/16. doi: 10.1186/s13014-021-01923-1.
    PubMed
  • 45 Cha E, Elguindi S, Onochie I, Gorovets D, Deasy JO, Zelefsky M, et al. Clinical implementation of deep learning contour autosegmentation for prostate radiotherapy. Radiother Oncol. 2021 Jun;159:1-7. PubMed PMID: 33667591. Epub 2021/03/06. doi: 10.1016/j.radonc.2021.02.040.
    PubMed
  • 46 Kneepkens E, Bakx N, van der Sangen M, Theuws J, van der Toorn PP, Rijkaart D, et al. Clinical evaluation of two AI models for automated breast cancer plan generation. Radiat Oncol. 2022 Feb 5;17(1):25. PubMed PMID: 35123517. Epub 2022/02/07. doi: 10.1186/s13014-022-01993-9.
    PubMed
  • 47 Hong JC, Eclov NCW, Stephens SJ, Mowery YM, Palta M. Implementation of machine learning in the clinic: challenges and lessons in prospective deployment from the System for High Intensity EvaLuation During Radiation Therapy (SHIELD-RT) randomized controlled study. BMC Bioinformatics. 2022 Sep 30;23(Suppl 12):408. PubMed PMID: 36180836. Epub 2022/10/01. doi: 10.1186/s12859-022-04940-3.
    PubMed
  • 48 Benrimoh D, Tanguay-Sela M, Perlman K, Israel S, Mehltretter J, Armstrong C, et al. Using a simulation centre to evaluate preliminary acceptability and impact of an artificial intelligence-powered clinical decision support system for depression treatment on the physician-patient interaction. BJPsych Open. 2021 Jan 6;7(1):e22. PubMed PMID: 33403948. Epub 20210106. doi: 10.1192/bjo.2020.127.
    PubMed
  • 49 Popescu C, Golden G, Benrimoh D, Tanguay-Sela M, Slowey D, Lundrigan E, et al. Evaluating the Clinical Feasibility of an Artificial Intelligence-Powered, Web-Based Clinical Decision Support System for the Treatment of Depression in Adults: Longitudinal Feasibility Study. JMIR Form Res. 2021 Oct 25;5(10):e31862. PubMed PMID: 34694234. Epub 20211025. doi: 10.2196/31862.
    PubMed
  • 50 Tanguay-Sela M, Benrimoh D, Popescu C, Perez T, Rollins C, Snook E, et al. Evaluating the perceived utility of an artificial intelligence-powered clinical decision support system for depression treatment using a simulation center. Psychiatry Res. 2022 Feb;308:114336. PubMed PMID: 34953204. Epub 2021/12/26. doi: 10.1016/j.psychres.2021.114336.
    PubMed
  • 51 Wilimitis D, Turer RW, Ripperger M, McCoy AB, Sperry SH, Fielstein EM, et al. Integration of Face-to-Face Screening With Real-time Machine Learning to Predict Risk of Suicide Among Adults. JAMA Netw Open. 2022 May 2;5(5):e2212095. PubMed PMID: 35560048. Epub 2022/05/14. doi: 10.1001/jamanetworkopen.2022.12095.
    PubMed
  • 52 Yao X, Rushlow DR, Inselman JW, McCoy RG, Thacher TD, Behnken EM, et al. Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial. Nat Med. 2021 May;27(5):815-9. PubMed PMID: 33958795. Epub 20210506. doi: 10.1038/s41591-021-01335-4.
    PubMed
  • 53 Edalati M, Zheng Y, Watkins MP, Chen J, Liu L, Zhang S, et al. Implementation and prospective clinical validation of AI-based planning and shimming techniques in cardiac MRI. Med Phys. 2022 Jan;49(1):129-143. PubMed PMID: 34748660. Epub 2021/11/09. doi: 10.1002/mp.15327.
    PubMed
  • 54 Chen J, Gao Y. The Role of Deep Learning-Based Echocardiography in the Diagnosis and Evaluation of the Effects of Routine Anti-Heart-Failure Western Medicines in Elderly Patients with Acute Left Heart Failure. J Healthc Eng. 2021;2021:4845792. PubMed PMID: 34422243. Epub 2021/08/24. doi: 10.1155/2021/4845792.
    PubMed
  • 55 Pangti R, Mathur J, Chouhan V, Kumar S, Rajput L, Shah S, et al. A machine learning-based, decision support, mobile phone application for diagnosis of common dermatological diseases. J Eur Acad Dermatol Venereol. 2021 Feb;35(2):536-45. PubMed PMID: 32991767. Epub 2020/09/30. doi: 10.1111/jdv.16967.
    PubMed
  • 56 Jain A, Way D, Gupta V, Gao Y, de Oliveira Marinho G, Hartford J, et al. Development and Assessment of an Artificial Intelligence-Based Tool for Skin Condition Diagnosis by Primary Care Physicians and Nurse Practitioners in Teledermatology Practices. JAMA Netw Open. 2021 Apr 1;4(4):e217249. PubMed PMID: 33909055. Epub 2021/04/29. doi: 10.1001/jamanetworkopen.2021.7249.
    PubMed
  • 57 Yoelin S, Green JB, Dhawan SS, Hasan F, Mahbod B, Khan B, et al. The Use of a Novel Artificial Intelligence Platform for the Evaluation of Rhytids. Aesthet Surg J. 2022 Oct 13;42(11):NP688-NP694. PubMed PMID: 35869540. Epub 2022/07/23. doi: 10.1093/asj/sjac200.
    PubMed
  • 58 Ipp E, Liljenquist D, Bode B, Shah VN, Silverstein S, Regillo CD, et al. Pivotal Evaluation of an Artificial Intelligence System for Autonomous Detection of Referrable and Vision-Threatening Diabetic Retinopathy. JAMA Netw Open. 2021 Nov 1;4(11):e2134254. PubMed PMID: 34779843. Epub 2021/11/16. doi: 10.1001/jamanetworkopen.2021.34254.
    PubMed
  • 59 Hao S, Liu C, Li N, Wu Y, Li D, Gao Q, et al. Clinical evaluation of AI-assisted screening for diabetic retinopathy in rural areas of midwest China. PLoS One. 2022;17(10):e0275983. PubMed PMID: 36227905. Epub 2022/10/14. doi: 10.1371/journal.pone.0275983.
    PubMed
  • 60 Ming S, Xie K, Lei X, Yang Y, Zhao Z, Li S, et al. Evaluation of a novel artificial intelligence-based screening system for diabetic retinopathy in community of China: a real-world study. Int Ophthalmol. 2021 Apr;41(4):1291-1299. PubMed PMID: 33389425. Epub 2021/01/04. doi: 10.1007/s10792-020-01685-x.
    PubMed
  • 61 Martinez VA, Betts RK, Scruth EA, Buckley JD, Cadiz VR, Bertrand LD, et al. The Kaiser Permanente Northern California Advance Alert Monitor Program: An Automated Early Warning System for Adults at Risk for In-Hospital Clinical Deterioration. Jt Comm J Qual Patient Saf. 2022 Aug;48(8):370-375. PubMed PMID: 35902140. Epub 2022/07/29. doi: 10.1016/j.jcjq.2022.05.005.
    PubMed
  • 62 Winslow CJ, Edelson DP, Churpek MM, Taneja M, Shah NS, Datta A, et al. The Impact of a Machine Learning Early Warning Score on Hospital Mortality: A Multicenter Clinical Intervention Trial. Crit Care Med. 2022 Sep 1;50(9):1339-1347. PubMed PMID: 35452010. Epub 2022/04/23. doi: 10.1097/ccm.0000000000005492.
    PubMed
  • 63 Schwartz JM, George M, Rossetti SC, Dykes PC, Minshall SR, Lucas E, et al. Factors Influencing Clinician Trust in Predictive Clinical Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive Study. JMIR Hum Factors. 2022 May 12;9(2):e33960. PubMed PMID: 35550304. Epub 20220512. doi: 10.2196/33960.
    PubMed
  • 64 Gunda B, Neuhaus A, Sipos I, Stang R, Böjti PP, Takács T, et al. Improved Stroke Care in a Primary Stroke Centre Using AI-Decision Support. Cerebrovasc Dis Extra. 2022;12(1):28-32. PubMed PMID: 35134802. Epub 20220208. doi: 10.1159/000522423.
    PubMed
  • 65 Yahav-Dovrat A, Saban M, Merhav G, Lankri I, Abergel E, Eran A, et al. Evaluation of Artificial Intelligence-Powered Identification of Large-Vessel Occlusions in a Comprehensive Stroke Center. AJNR Am J Neuroradiol. 2021 Jan;42(2):247-254. PubMed PMID: 33384294. Epub 2021/01/02. doi: 10.3174/ajnr.A6923.
    PubMed
  • 66 Adams R, Henry KE, Sridharan A, Soleimani H, Zhan A, Rawat N, et al. Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis. Nat Med. 2022 Jul;28(7):1455-1460. PubMed PMID: 35864252. Epub 2022/07/22. doi: 10.1038/s41591-022-01894-0.
    PubMed
  • 67 Zhou S, Ma X, Jiang S, Huang X, You Y, Shang H, et al. A retrospective study on the effectiveness of Artificial Intelligence-based Clinical Decision Support System (AI-CDSS) to improve the incidence of hospital-related venous thromboembolism (VTE). Ann Transl Med. 2021 Mar;9(6):491. PubMed PMID: 33850888. Epub 2021/04/15. doi: 10.21037/atm-21-1093.
    PubMed
  • 68 Herter WE, Khuc J, Cinà G, Knottnerus BJ, Numans ME, Wiewel MA, et al. Impact of a Machine Learning-Based Decision Support System for Urinary Tract Infections: Prospective Observational Study in 36 Primary Care Practices. JMIR Med Inform. 2022 May 4;10(5):e27795. PubMed PMID: 35507396. Epub 2022/05/05. doi: 10.2196/27795.
    PubMed
  • 69 Li YY, Wang JJ, Huang SH, Kuo CL, Chen JY, Liu CF, et al. Implementation of a machine learning application in preoperative risk assessment for hip repair surgery. BMC Anesthesiol. 2022 Apr 23;22(1):116. PubMed PMID: 35459103. Epub 2022/04/24. doi: 10.1186/s12871-022-01648-y.
    PubMed
  • 70 Glick A, Clayton M, Angelov N, Chang J. Impact of explainable artificial intelligence assistance on clinical decision-making of novice dental clinicians. JAMIA Open. 2022 Jul;5(2):ooac031. PubMed PMID: 35651525. Epub 2022/06/03. doi: 10.1093/jamiaopen/ooac031.
    PubMed
  • 71 Kurstjens S, de Bel T, van der Horst A, Kusters R, Krabbe J, van Balveren J. Automated prediction of low ferritin concentrations using a machine learning algorithm. Clin Chem Lab Med. 2022 Nov 25;60(12):1921-1928. PubMed PMID: 35258239. Epub 2022/03/09. doi: 10.1515/cclm-2021-1194.
    PubMed
  • 72 Alrajhi AA, Alswailem OA, Wali G, Alnafee K, AlGhamdi S, Alarifi J, et al. Data-Driven Prediction for COVID-19 Severity in Hospitalized Patients. Int J Environ Res Public Health. 2022 Mar 3;19(5):2958. PubMed PMID: 35270653. Epub 2022/03/11. doi: 10.3390/ijerph19052958.
    PubMed
  • 73 Dong Y, Wang Z, Zhang Z, Niu B, Chen P, Zhang P, et al. Artificial Intelligence Algorithm-Based Computed Tomography Images in the Evaluation of the Curative Effect of Enteral Nutrition after Neonatal High Intestinal Obstruction Operation. J Healthc Eng. 2021;2021:7096286. PubMed PMID: 34824765. Epub 2021/11/27. doi: 10.1155/2021/7096286.
    PubMed
  • 74 Howell RS, Liu HH, Khan AA, Woods JS, Lin LJ, Saxena M, et al. Development of a Method for Clinical Evaluation of Artificial Intelligence-Based Digital Wound Assessment Tools. JAMA Netw Open. 2021 May 3;4(5):e217234. PubMed PMID: 34009348. Epub 2021/05/20. doi: 10.1001/jamanetworkopen.2021.7234.
    PubMed
  • 75 Maeda Y, Kudo SE, Ogata N, Misawa M, Iacucci M, Homma M, et al. Evaluation in real-time use of artificial intelligence during colonoscopy to predict relapse of ulcerative colitis: a prospective study. Gastrointest Endosc. 2022 Apr;95(4):747-756.e2. PubMed PMID: 34695422. Epub 2021/10/26. doi: 10.1016/j.gie.2021.10.019.
    PubMed
  • 76 Chen X, Huang X, Yin M. Implementation of Hospital-to-Home Model for Nutritional Nursing Management of Patients with Chronic Kidney Disease Using Artificial Intelligence Algorithm Combined with CT Internet. Contrast Media Mol Imaging. 2022;2022:1183988. PubMed PMID: 35414801. Epub 2022/04/14. doi: 10.1155/2022/1183988.
    PubMed
  • 77 Hwang J, Lee T, Lee H, Byun S. A Clinical Decision Support System for Sleep Staging Tasks With Explanations From Artificial Intelligence: User-Centered Design and Evaluation Study. J Med Internet Res. 2022 Jan 19;24(1):e28659. PubMed PMID: 35044311. Epub 2022/01/20. doi: 10.2196/28659.
    PubMed
  • 78 Sujan M, Furniss D, Grundy K, Grundy H, Nelson D, Elliott M, et al. Human factors challenges for the safe use of artificial intelligence in patient care. BMJ Health Care Inform. 2019 Nov;26(1):e100081. PubMed PMID: 31780459. Epub 2019/11/30. doi: 10.1136/bmjhci-2019-100081.
    PubMed
  • 79 Sujan M, Pool R, Salmon P. Eight human factors and ergonomics principles for healthcare artificial intelligence. BMJ Health Care Inform. 2022 Feb;29(1):e100516. PubMed PMID: 35121617. Epub 2022/02/06. doi: 10.1136/bmjhci-2021-100516.
    PubMed
  • 80 Kim MO, Coiera E, Magrabi F. Problems with health information technology and their effects on care delivery and patient outcomes: a systematic review. J Am Med Inform Assoc. 2017 Mar 01;24(2):246-250. PubMed PMID: 28011595. Epub 2016/12/25. doi: 10.1093/jamia/ocw154.
    PubMed
  • 81 Lyell D, Coiera E. Automation bias and verification complexity: a systematic review. J Am Med Inform Assoc. 2017 Mar 1;24(2):423-431. PubMed PMID: 27516495. Epub 2016/08/16. doi: 10.1093/jamia/ocw105.
    PubMed
  • 82 Hu M, Chen N, Zhou X, Wu Y, Ma C. Deep Learning-Based Computed Tomography Perfusion Imaging to Evaluate the Effectiveness and Safety of Thrombolytic Therapy for Cerebral Infarct with Unknown Time of Onset. Contrast Media Mol Imaging. 2022;2022:9684584. PubMed PMID: 35615733. Epub 2022/05/27. doi: 10.1155/2022/9684584.
    PubMed
  • 83 Magrabi F, Ammenwerth E, McNair JB, De Keizer NF, Hypponen H, Nykanen P, et al. Artificial Intelligence in Clinical Decision Support: Challenges for Evaluating AI and Practical Implications. Yearb Med Inform. 2019 Aug;28(1):128-134. PubMed PMID: 31022752. Epub 2019/04/26. doi: 10.1055/s-0039-1677903.
    PubMed
  • 84 Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al. Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ. 2022 May 18;377:e070904. PubMed PMID: 35584845. Epub 2022/05/19. doi: 10.1136/bmj-2022-070904.
    PubMed
  • 85 Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care. 2010 Oct;19 Suppl 3(Suppl 3):i68-74. PubMed PMID: 20959322. Epub 2010/10/27. doi: 10.1136/qshc.2010.042085.
    PubMed
  • 86 Lyell D, Lustig A, Denyer K, Vedantam S, Magrabi F. Using clinical simulation to evaluate AI-enabled decision support. In proceedings of Medinfo - the19th World Congress on Medical and Health Informatics (8 -12 July, Sydney) 2023. (to appear).
    PubMed
  • 87 Benis A, Tamburis O, Chronaki C, Moen A. One Digital Health: A Unified Framework for Future Health Ecosystems. J Med Internet Res. 2021 Feb 5;23(2):e22189. PubMed PMID: 33492240. Epub 2021/01/26. doi: 10.2196/22189.
    PubMed
  • 88 Celi LA, Cellini J, Charpignon M-L, Dee EC, Dernoncourt F, Eber R, et al. Sources of bias in artificial intelligence that perpetuate healthcare disparities—A global review. PLOS Digit Health. 2022;1(3):e0000022. doi: 10.1371/journal.pdig.0000022.
    PubMed