nach oben

Erschienen in:

24.01.2024 | Original Article

Applicability of Online Chat-Based Artificial Intelligence Models to Colorectal Cancer Screening

verfasst von: Joseph Atarere, Haider Naqvi, Christopher Haas, Comfort Adewunmi, Sumanth Bandaru, Rakesh Allamneni, Onyinye Ugonabo, Olachi Egbo, Mfoniso Umoren, Priyanka Kanth

Erschienen in: Digestive Diseases and Sciences | Ausgabe 3/2024

Einloggen, um Zugang zu erhalten

Abstract

Background

Over the past year, studies have shown potential in the applicability of ChatGPT in various medical specialties including cardiology and oncology. However, the application of ChatGPT and other online chat-based AI models to patient education and patient-physician communication on colorectal cancer screening has not been critically evaluated which is what we aimed to do in this study.

Methods

We posed 15 questions on important colorectal cancer screening concepts and 5 common questions asked by patients to the 3 most commonly used freely available artificial intelligence (AI) models. The responses provided by the AI models were graded for appropriateness and reliability using American College of Gastroenterology guidelines. The responses to each question provided by an AI model were graded as reliably appropriate (RA), reliably inappropriate (RI) and unreliable. Grader assessments were validated by the joint probability of agreement for two raters.

Results

ChatGPT and YouChat™ provided RA responses to the questions posed more often than BingChat. There were two questions that > 1 AI model provided unreliable responses to. ChatGPT did not provide references. BingChat misinterpreted some of the information it referenced. The age of CRC screening provided by YouChat™ was not consistently up-to-date. Inter-rater reliability for 2 raters was 89.2%.

Conclusion

Most responses provided by AI models on CRC screening were appropriate. Some limitations exist in their ability to correctly interpret medical literature and provide updated information in answering queries. Patients should consult their physicians for context on the recommendations made by these AI models.

Nur mit Berechtigung zugänglich

Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA: A Cancer Journal for Clinicians 2021;71:209–249. https://doi.org/10.3322/caac.21660.CrossRefPubMed

Reif de Paula T, Haas EM, Keller DS. Colorectal cancer in the 45-to-50 age group in the United States: a National Cancer Database (NCDB) analysis. Surgical Endoscopy 2022;36:6629–6637.CrossRefPubMed

American Cancer Society. (2023). Key Statistics for Colorectal Cancer. https://www.cancer.org/cancer/colon-rectal-cancer/about/key-statistics.html

Davidson KW, Barry MJ, Mangione CM, et al. Screening for colorectal cancer: US Preventive Services Task Force recommendation statement. Jama 2021;325:1965–1977.CrossRef

Amlani B, Radaelli F, Bhandari P. A survey on colonoscopy shows poor understanding of its protective value and widespread misconceptions across Europe. PLOS ONE 2020;15:e0233490. https://doi.org/10.1371/journal.pone.0233490.CrossRefPubMedPubMedCentral

Brandt HM, Dolinger HR, Sharpe PA, Hardin JW, Berger FG. Relationship of colorectal cancer awareness and knowledge with colorectal cancer screening. Colorectal Cancer 2012;1:383–396. https://doi.org/10.2217/crc.12.45.CrossRefPubMedPubMedCentral

Underhill ML, Kiviniemi MT. The Association of Perceived Provider-Patient Communication and Relationship Quality With Colorectal Cancer Screening. Health Education & Behavior 2012;39:555–563. https://doi.org/10.1177/1090198111421800.CrossRef

Arora NK. Interacting with cancer patients: the significance of physicians’ communication behavior. Social Science & Medicine 2003;57:791–806. https://doi.org/10.1016/S0277-9536(02)00449-5.CrossRef

Pew Research Center. (2014, March 11). World Wide Web Timeline. Pew Research Center. https://www.pewresearch.org/internet/2014/03/11/world-wide-web-timeline/

10.

Hesse BW, Nelson DE, Kreps GL, et al. Trust and Sources of Health Information. Archives of Internal Medicine 2005;165:2618. https://doi.org/10.1001/archinte.165.22.2618.CrossRefPubMed

11.

Murray E, Lo B, Pollack L, et al. The Impact of Health Information on the Internet on the Physician-Patient Relationship. Archives of Internal Medicine 2003;163:1727. https://doi.org/10.1001/archinte.163.14.1727.CrossRefPubMed

12.

Berland GK, Elliott MN, Morales LS, et al. Health Information on the Internet. JAMA 2001;285:2612. https://doi.org/10.1001/jama.285.20.2612.CrossRefPubMedPubMedCentral

13.

Daraz L, Morrow AS, Ponce OJ, et al. Can Patients Trust Online Health Information? A Meta-narrative Systematic Review Addressing the Quality of Health Information on the Internet. Journal of General Internal Medicine 2019;34:1884–1891. https://doi.org/10.1007/s11606-019-05109-0.CrossRefPubMedPubMedCentral

14.

Bundorf MK, Wagner TH, Singer SJ, Baker LC. Who Searches the Internet for Health Information? Health Services Research 2006;41:819–836. https://doi.org/10.1111/j.1475-6773.2006.00510.x.CrossRefPubMedPubMedCentral

15.

Kumar Y, Koul A, Singla R, Ijaz MF. Artificial intelligence in disease diagnosis: a systematic literature review, synthesizing framework and future research agenda. Journal of Ambient Intelligence and Humanized Computing, 2021;1–28. https://doi.org/10.1007/s12652-021-03612-z

16.

Yin Z, Yao C, Zhang L, Qi S. Application of artificial intelligence in diagnosis and treatment of colorectal cancer: A novel Prospect. Frontiers in Medicine 2023;10:1128084. https://doi.org/10.3389/fmed.2023.1128084.CrossRefPubMedPubMedCentral

17.

OpenAI. (2022, November). ChatGPT: optimizing language models for dialogue. https://openai.com/blog/chatgpt

18.

Ortiz, S. (2023). The best AI chatbots: ChatGPT and other interesting alternatives to try. ZDNET. https://www.zdnet.com/article/best-ai-chatbot/

19.

Cascella M, Montomoli J, Bellini V, Bignami E. Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios. Journal of Medical Systems 2023;47:33. https://doi.org/10.1007/s10916-023-01925-4.CrossRefPubMedPubMedCentral

20.

Sarraju A, Bruemmer D, van Iterson E, Cho L, Rodriguez F, Laffin L. Appropriateness of Cardiovascular Disease Prevention Recommendations Obtained From a Popular Online Chat-Based Artificial Intelligence Model. JAMA 2023;329:842. https://doi.org/10.1001/jama.2023.1044.CrossRefPubMedPubMedCentral

21.

Johnson SB, King AJ, Warner EL, et al. Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information. JNCI Cancer Spectrum. 2023. https://doi.org/10.1093/jncics/pkad015.CrossRefPubMedPubMedCentral

22.

American Cancer Society. (2023, April). About Colorectal Cancer. American Cancer Society. https://www.cancer.org/cancer/colon-rectal-cancer/about.html

23.

Kassab J, el Dahdah J, Chedid El Helou M, et al. Assessing the Accuracy of an Online Chat-Based Artificial Intelligence Model in Providing Recommendations on Hypertension Management in Accordance With the 2017 American College of Cardiology/American Heart Association and 2018 European Society of Cardiology/European Society of Hypertension Guidelines. Hypertension 2023;80(7). https://doi.org/10.1161/HYPERTENSIONAHA.123.21183

24.

Shaukat A, Kahi CJ, Burke CA, Rabeneck L, Sauer BG, Rex DK. ACG Clinical Guidelines: Colorectal Cancer Screening 2021. American Journal of Gastroenterology 2021;116:458–479. https://doi.org/10.14309/ajg.0000000000001122

25.

Wikipedia. (2023, January 24). Inter-rater reliability. Wikipedia. https://en.wikipedia.org/wiki/Inter-rater_reliability

26.

Glen, S. (n.d.). Inter-rater Reliability IRR: Definition, Calculation. Elementary Statistics for the Rest of Us! Retrieved May 15, 2023, from https://www.statisticshowto.com/inter-rater-reliability/

27.

Li W, Zhang Y, Chen F. ChatGPT in Colorectal Surgery: A Promising Tool or a Passing Fad? Annals of Biomedical Engineering. 2023. https://doi.org/10.1007/s10439-023-03232-y.CrossRefPubMedPubMedCentral

28.

Temsah O, Khan SA, Chaiah Y, et al. Overview of Early ChatGPT’s Presence in Medical Literature: Insights From a Hybrid Literature Review by ChatGPT and Human Experts. Cureus. 2023. https://doi.org/10.7759/cureus.37281.CrossRefPubMedPubMedCentral

29.

Taveira F, Areia M, Elvas L, et al. A 3-day low-fibre diet does not improve colonoscopy preparation results compared to a 1-day diet: A randomized, single-blind, controlled trial. United European Gastroenterology Journal 2019;7:1321–1329. https://doi.org/10.1177/2050640619883176.CrossRefPubMedPubMedCentral

30.

Nov O, Singh N, Mann DM. Putting ChatGPT’s Medical Advice to the (Turing) Test. MedRxiv 2023. https://doi.org/10.1101/2023.01.23.23284735

31.

Uebersax JS. Diversity of decision-making models and the measurement of interrater agreement. Psychological Bulletin 1987;101:140–146. https://doi.org/10.1037/0033-2909.101.1.140.CrossRef

Titel: Applicability of Online Chat-Based Artificial Intelligence Models to Colorectal Cancer Screening
verfasst von: Joseph Atarere
Haider Naqvi
Christopher Haas
Comfort Adewunmi
Sumanth Bandaru
Rakesh Allamneni
Onyinye Ugonabo
Olachi Egbo
Mfoniso Umoren
Priyanka Kanth
Publikationsdatum: 24.01.2024
Verlag: Springer US
Erschienen in: Digestive Diseases and Sciences / Ausgabe 3/2024
Print ISSN: 0163-2116
Elektronische ISSN: 1573-2568
DOI: https://doi.org/10.1007/s10620-024-08274-3

Leitlinien kompakt für die Innere Medizin

Mit medbee Pocketcards sicher entscheiden.

^{Seit 2022 gehört die medbee GmbH zum Springer Medizin Verlag}

Kostenlos registrieren

Neu im Fachgebiet Innere Medizin

CKD bei Diabetes: Neuheiten und Zukunftsaussichten

16.05.2024 DDG-Jahrestagung 2024 Kongressbericht

Jeder Mensch mit Diabetes muss auf eine chronische Nierenerkrankung gescreent werden – diese neue Empfehlung spricht die KDIGO aus. Die Therapie erfolgt individuell und je nach Szenario mit verschiedenen Substanzklassen. Künftig kommt wahrscheinlich, neben RAS-Hemmung, SGLT2-Inhibition und nsMRA, eine vierte Therapiesäule hinzu.

Riesenzellarteriitis: 15% der Patienten sind von okkulter Form betroffen

16.05.2024 Riesenzellarteriitis Nachrichten

In einer retrospektiven Untersuchung haben Forschende aus Belgien und den Niederlanden die okkulte Form der Riesenzellarteriitis genauer unter die Lupe genommen. In puncto Therapie und Rezidivraten stellten sie keinen sehr großen Unterschied zu Erkrankten mit kranialen Symptomen fest.

SGLT2-Inhibitoren und GLP-1-Rezeptoragonisten im Schlagabtausch

16.05.2024 DDG-Jahrestagung 2024 Kongressbericht

Wer hat die Nase vorn – SGLT2-Inhibitoren oder GLP-1-Rezeptoragonisten? Diese Frage diskutierten zwei Experten in einer Session auf dem diesjährigen Diabetes-Kongress.

Betalaktam-Allergie: praxisnahes Vorgehen beim Delabeling

16.05.2024 Pädiatrische Allergologie Nachrichten

Die große Mehrheit der vermeintlichen Penicillinallergien sind keine. Da das „Etikett“ Betalaktam-Allergie oft schon in der Kindheit erworben wird, kann ein frühzeitiges Delabeling lebenslange Vorteile bringen. Ein Team von Pädiaterinnen und Pädiatern aus Kanada stellt vor, wie sie dabei vorgehen.

Update Innere Medizin

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.

Newsletter bestellen

Springer Medizin

Abstract

Background

Methods

Results

Conclusion

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 3/2024

Spatio-Temporal Feature Transformation Based Polyp Recognition for Automatic Detection: Higher Accuracy than Novice Endoscopists in Colorectal Polyp Detection and Diagnosis

Hsa_circ_0014606 Derived from Exosomes Promotes Gastric Carcinoma Tumorigenesis and Proliferation by Sponging miR-514b-3p to Upregulate HNRNPC

A Novel Technique for the Management of Persistent Biliary Fistulas Developing After Liver Hydatid Cyst Surgeries: Drain Clamping Technique

Prevalence and Appropriateness of Polypharmacy in Older Adults with Inflammatory Bowel Diseases

Correction: Population Based Time Trends in the Epidemiology and Mortality of Gastroesophageal Junction and Esophageal Adenocarcinoma