nach oben

memo - Magazine of European Medical Oncology

26.01.2024 | original report

Can artificial intelligence provide accurate and reliable answers to cancer patients’ questions about cancer pain? Comparison of chatbots based on ESMO cancer pain guideline

verfasst von: Kadriye Bir Yücel, MD, Osman Sutcuoglu, MD, Ozan Yazıcı, MD, Ahmet Ozet, MD, Nuriye Ozdemir, MD

Erschienen in: memo - Magazine of European Medical Oncology

Einloggen, um Zugang zu erhalten

Summary

Purpose

The purpose of this study was to assess the accuracy, readability, and stability of the answers given by chatbots to patients’ questions about cancer pain management.

Materials and methods

Twenty questions about cancer pain were constructed based on the European Society of Medical Oncology (ESMO) cancer pain patient guide. These questions were asked to five chatbots: Google Bard (Google AI, USA), ChatGPT‑4 (Chat Generative Pre-trained Transforme, OpenAI, USA) (premium version), ChatGPT‑3.5 (free version), Perplexity (Perplexity AI, USA), and Chatsonic (WriteSonic, USA). Three medical oncologists with at least 10 years of experience evaluated the chatbots’ responses for accuracy, readability, and stability.

Results

ChatGPT-4 had the highest accuracy rate of 96%. Perplexity had the highest readability but the lowest accuracy (86%). Google Bard and ChatGPT‑4 were the most stable (100%) chatbots. Both versions of ChatGPT appeared to provide extensive information, but the answers only included information before September 2021.

Conclusion

All chatbots are insufficient to obtain accurate information for cancer patients, and the resources are quite inadequate in acquiring accurate information for cancer patients and their families, and they need further development.

Nur mit Berechtigung zugänglich

Bahar-Ozdemir Y, Ozsoy-Unubol T, Akyuz G. Is YouTube a high-quality source of information on cancer rehabilitation? J Cancer Surviv. 2022;16(5):1016–22. https://doi.org/10.1007/s11764-021-01093-9.CrossRefPubMed

Sütcüoğlu O, Özay Zİ, Özet A, Yazıcı O, Özdemir N. Evaluation of scientific reliability and quality of YouTube videos on cancer and nutrition. Nutrition. 2023;108:111933. https://doi.org/10.1016/j.nut.2022.111933.CrossRefPubMed

Ali SR, Dobbs TD, Hutchings HA, Whitaker IS. Using ChatGPT to write patient clinic letters. Lancet Digit Health. 2023;5(4):e179–e81. https://doi.org/10.1016/S2589-7500(23)00048-1.CrossRefPubMed

Xue VW, Lei P, Cho WC. The potential impact of ChatGPT in clinical and translational medicine. Clin Transl Med. 2023;13(3):e1216. https://doi.org/10.1002/ctm2.1216.CrossRefPubMedPubMedCentral

Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9:e45312. https://doi.org/10.2196/45312.CrossRefPubMedPubMedCentral

Arif TB, Munaf U, Ul-Haque I. The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online. 2023;28(1):2181052. https://doi.org/10.1080/10872981.2023.2181052.CrossRefPubMedPubMedCentral

Homolak J. Opportunities and risks of ChatGPT in medicine, science, and academic publishing: a modern Promethean dilemma. Croat Med J. 2023;64(1):1–3. https://doi.org/10.3325/cmj.2023.64.1.CrossRefPubMedPubMedCentral

Baumgartner C. The potential impact of ChatGPT in clinical and translational medicine. Clin Transl Med. 2023;13(3):e1206. https://doi.org/10.1002/ctm2.1206.CrossRefPubMedPubMedCentral

Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr. 2023;7(2):pkad10. https://doi.org/10.1093/jncics/pkad010.CrossRefPubMedPubMedCentral

10.

Si L, Callan J. A statistical model for scientific readability. In proceedings of the tenth international conference on Information and knowledge management (CIKM ’01). New York, NY, USA: Association for Computing Machinery; 2001. pp. 574–6. https://doi.org/10.1145/502585.502695.CrossRef

11.

Haver HL, Ambinder EB, Bahl M, Oluyemi ET, Jeudy J, Yi PH. Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT. Radiology. 2023;307(4):e230424. https://doi.org/10.1148/radiol.230424.CrossRefPubMed

12.

Sarraju A, Bruemmer D, Van Iterson E, Cho L, Rodriguez F, Laffin L. Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model. JAMA. 2023;329(10):842–4. https://doi.org/10.1001/jama.2023.1044.CrossRefPubMedPubMedCentral

13.

Di Bello F, Collà Ruvolo C, Cilio S, La Rocca R, Capece M, Creta M, et al. Testicular cancer and YouTube: what do you expect from a social media platform? Int J Urol. 2022;29(7):685–91. https://doi.org/10.1111/iju.14871.CrossRefPubMed

14.

Johnson SB, Parsons M, Dorff T, Moran MS, Ward JH, Cohen SA, et al. Cancer misinformation and harmful information on facebook and other social media: a brief report. J Natl Cancer Inst. 2022;114(7):1036–9. https://doi.org/10.1093/jnci/djab141.CrossRefPubMed

15.

Johnson SB, King AJ, Warner EL, Aneja S, Kann BH, Bylund CL. Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information. JNCI Cancer Spectr. 2023;7(2):pkad15. https://doi.org/10.1093/jncics/pkad015.CrossRefPubMedPubMedCentral

16.

Yang K, Ji S, Zhang T, Xie Q, Ananiadou S. On the evaluations of ChatGPT and emotion-enhanced prompting for mental health analysis. 2023. https://doi.org/10.48550/arXiv.2304.03347.CrossRef

17.

Singh OP. Artificial intelligence in the era of ChatGPT-Opportunities and challenges in mental health care. Indian J Psychiatry. 2023;65(3):297–8. https://doi.org/10.4103/indianjpsychiatry.indianjpsychiatry_112_23.CrossRefPubMedPubMedCentral

18.

Uludag K. Testing creativity of ChatGPT in psychology: Interview with ChatGPT. 2023. https://doi.org/10.2139/ssrn.4390872. SSRN 4390872.CrossRef

Titel: Can artificial intelligence provide accurate and reliable answers to cancer patients’ questions about cancer pain? Comparison of chatbots based on ESMO cancer pain guideline
verfasst von: Kadriye Bir Yücel, MD
Osman Sutcuoglu, MD
Ozan Yazıcı, MD
Ahmet Ozet, MD
Nuriye Ozdemir, MD
Publikationsdatum: 26.01.2024
Verlag: Springer Vienna
Erschienen in: memo - Magazine of European Medical Oncology
Print ISSN: 1865-5041
Elektronische ISSN: 1865-5076
DOI: https://doi.org/10.1007/s12254-023-00954-6

Springer Medizin

Can artificial intelligence provide accurate and reliable answers to cancer patients’ questions about cancer pain? Comparison of chatbots based on ESMO cancer pain guideline

Summary

Purpose

Materials and methods

Results

Conclusion

Neu im Fachgebiet Onkologie

Adjuvante Immuntherapie verlängert Leben bei RCC

Alectinib verbessert krankheitsfreies Überleben bei ALK-positivem NSCLC

Bei Senioren mit Prostatakarzinom auf Anämie achten!

ICI-Therapie in der Schwangerschaft wird gut toleriert

Update Onkologie

Springer Medizin

Summary

Purpose

Materials and methods

Results

Conclusion

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Neu im Fachgebiet Onkologie

Adjuvante Immuntherapie verlängert Leben bei RCC

Alectinib verbessert krankheitsfreies Überleben bei ALK-positivem NSCLC

Bei Senioren mit Prostatakarzinom auf Anämie achten!

ICI-Therapie in der Schwangerschaft wird gut toleriert

Update Onkologie