nach oben

memo - Magazine of European Medical Oncology

Erschienen in:

26.01.2024 | original report

Can artificial intelligence provide accurate and reliable answers to cancer patients’ questions about cancer pain? Comparison of chatbots based on ESMO cancer pain guideline

verfasst von: Kadriye Bir Yücel, MD, Osman Sutcuoglu, MD, Ozan Yazıcı, MD, Ahmet Ozet, MD, Nuriye Ozdemir, MD

Erschienen in: memo - Magazine of European Medical Oncology | Ausgabe 4/2024

Einloggen, um Zugang zu erhalten

Summary

Purpose

The purpose of this study was to assess the accuracy, readability, and stability of the answers given by chatbots to patients’ questions about cancer pain management.

Materials and methods

Twenty questions about cancer pain were constructed based on the European Society of Medical Oncology (ESMO) cancer pain patient guide. These questions were asked to five chatbots: Google Bard (Google AI, USA), ChatGPT‑4 (Chat Generative Pre-trained Transforme, OpenAI, USA) (premium version), ChatGPT‑3.5 (free version), Perplexity (Perplexity AI, USA), and Chatsonic (WriteSonic, USA). Three medical oncologists with at least 10 years of experience evaluated the chatbots’ responses for accuracy, readability, and stability.

Results

ChatGPT-4 had the highest accuracy rate of 96%. Perplexity had the highest readability but the lowest accuracy (86%). Google Bard and ChatGPT‑4 were the most stable (100%) chatbots. Both versions of ChatGPT appeared to provide extensive information, but the answers only included information before September 2021.

Conclusion

All chatbots are insufficient to obtain accurate information for cancer patients, and the resources are quite inadequate in acquiring accurate information for cancer patients and their families, and they need further development.

Nur mit Berechtigung zugänglich

Bahar-Ozdemir Y, Ozsoy-Unubol T, Akyuz G. Is YouTube a high-quality source of information on cancer rehabilitation? J Cancer Surviv. 2022;16(5):1016–22. https://doi.org/10.1007/s11764-021-01093-9.CrossRefPubMed

Sütcüoğlu O, Özay Zİ, Özet A, Yazıcı O, Özdemir N. Evaluation of scientific reliability and quality of YouTube videos on cancer and nutrition. Nutrition. 2023;108:111933. https://doi.org/10.1016/j.nut.2022.111933.CrossRefPubMed

Ali SR, Dobbs TD, Hutchings HA, Whitaker IS. Using ChatGPT to write patient clinic letters. Lancet Digit Health. 2023;5(4):e179–e81. https://doi.org/10.1016/S2589-7500(23)00048-1.CrossRefPubMed

Xue VW, Lei P, Cho WC. The potential impact of ChatGPT in clinical and translational medicine. Clin Transl Med. 2023;13(3):e1216. https://doi.org/10.1002/ctm2.1216.CrossRefPubMedPubMedCentral

Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9:e45312. https://doi.org/10.2196/45312.CrossRefPubMedPubMedCentral

Arif TB, Munaf U, Ul-Haque I. The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online. 2023;28(1):2181052. https://doi.org/10.1080/10872981.2023.2181052.CrossRefPubMedPubMedCentral

Homolak J. Opportunities and risks of ChatGPT in medicine, science, and academic publishing: a modern Promethean dilemma. Croat Med J. 2023;64(1):1–3. https://doi.org/10.3325/cmj.2023.64.1.CrossRefPubMedPubMedCentral

Baumgartner C. The potential impact of ChatGPT in clinical and translational medicine. Clin Transl Med. 2023;13(3):e1206. https://doi.org/10.1002/ctm2.1206.CrossRefPubMedPubMedCentral

Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr. 2023;7(2):pkad10. https://doi.org/10.1093/jncics/pkad010.CrossRefPubMedPubMedCentral

10.

Si L, Callan J. A statistical model for scientific readability. In proceedings of the tenth international conference on Information and knowledge management (CIKM ’01). New York, NY, USA: Association for Computing Machinery; 2001. pp. 574–6. https://doi.org/10.1145/502585.502695.CrossRef

11.

Haver HL, Ambinder EB, Bahl M, Oluyemi ET, Jeudy J, Yi PH. Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT. Radiology. 2023;307(4):e230424. https://doi.org/10.1148/radiol.230424.CrossRefPubMed

12.

Sarraju A, Bruemmer D, Van Iterson E, Cho L, Rodriguez F, Laffin L. Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model. JAMA. 2023;329(10):842–4. https://doi.org/10.1001/jama.2023.1044.CrossRefPubMedPubMedCentral

13.

Di Bello F, Collà Ruvolo C, Cilio S, La Rocca R, Capece M, Creta M, et al. Testicular cancer and YouTube: what do you expect from a social media platform? Int J Urol. 2022;29(7):685–91. https://doi.org/10.1111/iju.14871.CrossRefPubMed

14.

Johnson SB, Parsons M, Dorff T, Moran MS, Ward JH, Cohen SA, et al. Cancer misinformation and harmful information on facebook and other social media: a brief report. J Natl Cancer Inst. 2022;114(7):1036–9. https://doi.org/10.1093/jnci/djab141.CrossRefPubMed

15.

Johnson SB, King AJ, Warner EL, Aneja S, Kann BH, Bylund CL. Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information. JNCI Cancer Spectr. 2023;7(2):pkad15. https://doi.org/10.1093/jncics/pkad015.CrossRefPubMedPubMedCentral

16.

Yang K, Ji S, Zhang T, Xie Q, Ananiadou S. On the evaluations of ChatGPT and emotion-enhanced prompting for mental health analysis. 2023. https://doi.org/10.48550/arXiv.2304.03347.CrossRef

17.

Singh OP. Artificial intelligence in the era of ChatGPT-Opportunities and challenges in mental health care. Indian J Psychiatry. 2023;65(3):297–8. https://doi.org/10.4103/indianjpsychiatry.indianjpsychiatry_112_23.CrossRefPubMedPubMedCentral

18.

Uludag K. Testing creativity of ChatGPT in psychology: Interview with ChatGPT. 2023. https://doi.org/10.2139/ssrn.4390872. SSRN 4390872.CrossRef

Titel: Can artificial intelligence provide accurate and reliable answers to cancer patients’ questions about cancer pain? Comparison of chatbots based on ESMO cancer pain guideline
verfasst von: Kadriye Bir Yücel, MD
Osman Sutcuoglu, MD
Ozan Yazıcı, MD
Ahmet Ozet, MD
Nuriye Ozdemir, MD
Publikationsdatum: 26.01.2024
Verlag: Springer Vienna
Erschienen in: memo - Magazine of European Medical Oncology / Ausgabe 4/2024
Print ISSN: 1865-5041
Elektronische ISSN: 1865-5076
DOI: https://doi.org/10.1007/s12254-023-00954-6

Next generation of drugs in breast cancer

Open Access
short review

Unraveling (K)RAS in pancreatic ductal adenocarcinoma

Open Access
short review

CAR-T cell therapy for relapsed or refractory multiple myeloma in 2024—clinically available treatment options in Austria

short review

NEC versus NET G3—is there a grey zone? Case report of pancreatic NET G3 with rapid disease progression

Open Access
case report

New therapy options for myelodysplastic syndrome—all for all or targeted?

Open Access
short review

Treatment sequencing in gastroenteropancreatic neuroendocrine tumors

Open Access
short review

Neu im Fachgebiet Onkologie

Ringen um den richtigen Umgang mit Zufallsbefunden

24.04.2025
DGP 2025
Kongressbericht

Wenn 2026 in Deutschland das Lungenkrebsscreening mittels Low-Dose-Computertomografie (LDCT) eingeführt wird, wird es auch viele Zufallsbefunde ans Licht bringen. Das birgt Chancen und Risiken.

Bald 5% der Krebserkrankungen durch CT verursacht

23.04.2025
Computertomografie
Nachrichten

Die jährlich rund 93 Millionen CTs in den USA könnten künftig zu über 100.000 zusätzlichen Krebserkrankungen führen, geht aus einer Modellrechnung hervor. Damit würde eine von 20 Krebserkrankungen auf die ionisierende Strahlung bei CT-Untersuchungen zurückgehen.

Weniger aggressive Prostatakrebs-Therapie, gleiche Prognose

23.04.2025
Prostatakarzinom
Nachrichten

Erhalten ältere Prostatakrebs-Patienten vor Therapiebeginn ein geriatrisches Assessment, führt dies häufig zu einer Deeskalation der Behandlung – ohne dass ihre Prognose dadurch schlechter würde. Darauf deuten Erfahrungen aus Frankreich hin.

Nach kardiovaskulärer Erkrankung steigt das Krebsrisiko

22.04.2025
Prävention und Rehabilitation in der Kardiologie
Nachrichten

Wer an einer kardiovaskulären Erkrankung (CVD) leidet, hat offenbar ein erhöhtes Risiko, in der Folge auch noch eine Krebsdiagnose zu erhalten. Der Zusammenhang scheint für diverse CVD und Krebsentitäten zu gelten.

Update Onkologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.

Newsletter bestellen

Springer Medizin

Can artificial intelligence provide accurate and reliable answers to cancer patients’ questions about cancer pain? Comparison of chatbots based on ESMO cancer pain guideline