Skip to main content
Erschienen in:

24.02.2022 | Original Paper

Text Comprehension Analyses to Improve Assessment Accuracy: Demonstration Using Gambling Disorder Screening

verfasst von: Samuel C. Peter, James P. Whelan, Rory A. Pfund

Erschienen in: Journal of Gambling Studies | Ausgabe 4/2022

Einloggen, um Zugang zu erhalten

Abstract

Many individuals diagnosed with an addictive disorder are members of disadvantaged groups and obtain a high school education or less, yet self-report questionnaires widely used to identify symptoms of addictive disorders do not use best practices to ensure item clarity and comprehension. In the present study, we explore how advanced text-analysis technology can be used to guide the development of a diagnostic questionnaire with an emphasis on maximizing its readability and then test the accuracy of this questionnaire. In Study 1, a self-report questionnaire for symptoms of gambling disorder was created using best practices for item clarity and comprehension. In study 2 an experimental design was used to test whether the measure with enhanced readability, compared to a commonly used screening instrument, improved diagnostic symptom accuracy among samples of high school and college educated individuals. Subsequent analyses revealed that education was positively related to item comprehension, and participants who completed the maximized readability questionnaire correctly identified more symptoms of gambling disorder than participants who completed the comparison questionnaire, regardless of educational attainment. These studies indicate that the rate at which individuals accurately identify symptoms of psychopathology is strongly related to their educational attainment and the readability of the questionnaire items themselves.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
Zurück zum Zitat Aikens, N. L., & Barbarin, O. (2008). Socioeconomic differences in reading trajectories: The contribution of family, neighborhood, and school contexts. Journal of Educational Psychology, 100, 235–251.CrossRef Aikens, N. L., & Barbarin, O. (2008). Socioeconomic differences in reading trajectories: The contribution of family, neighborhood, and school contexts. Journal of Educational Psychology, 100, 235–251.CrossRef
Zurück zum Zitat Gerstein, D., Murphy, S., Toce, M., Hoffman, J., Palmer, A., Johnson, R., & Sinclair, S. (1999). Gambling impact and behavior study: Report to the national gambling impact study commission. National Opinion Research Center. Gerstein, D., Murphy, S., Toce, M., Hoffman, J., Palmer, A., Johnson, R., & Sinclair, S. (1999). Gambling impact and behavior study: Report to the national gambling impact study commission. National Opinion Research Center.
Zurück zum Zitat Graesser, A. C., Cai, Z., Louwerse, M. M., & Daniel, F. (2006). Question Understanding Aid (QUAID) A web facility that tests question comprehensibility. Public Opinion Quarterly, 70, 3–22.CrossRef Graesser, A. C., Cai, Z., Louwerse, M. M., & Daniel, F. (2006). Question Understanding Aid (QUAID) A web facility that tests question comprehensibility. Public Opinion Quarterly, 70, 3–22.CrossRef
Zurück zum Zitat Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix providing multilevel analyses of text characteristics. Educational Researcher, 40, 223–234.CrossRef Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix providing multilevel analyses of text characteristics. Educational Researcher, 40, 223–234.CrossRef
Zurück zum Zitat Grove, W. M., Andreasen, N. C., McDonald-Scott, P., Keller, M. B., & Shapiro, R. W. (1981). Reliability studies of psychiatric diagnosis: Theory and practice. Achieves in General Psychiatry, 38(4), 408–413.CrossRef Grove, W. M., Andreasen, N. C., McDonald-Scott, P., Keller, M. B., & Shapiro, R. W. (1981). Reliability studies of psychiatric diagnosis: Theory and practice. Achieves in General Psychiatry, 38(4), 408–413.CrossRef
Zurück zum Zitat Hart, S. A., Soden, B., Johnson, W., Schatschneider, C., & Taylor, J. (2013). Expanding the environment: Gene× school-level SES interaction on reading comprehension. Journal of Child Psychology and Psychiatry, 54, 1047–1055.CrossRefPubMed Hart, S. A., Soden, B., Johnson, W., Schatschneider, C., & Taylor, J. (2013). Expanding the environment: Gene× school-level SES interaction on reading comprehension. Journal of Child Psychology and Psychiatry, 54, 1047–1055.CrossRefPubMed
Zurück zum Zitat Hodgins, D. C. (2004). Using the NORC DSM Screen for gambling problems as an outcome measure for pathological gambling: Psychometric evaluation. Addictive Behaviors, 29(8), 1685–1690.CrossRefPubMed Hodgins, D. C. (2004). Using the NORC DSM Screen for gambling problems as an outcome measure for pathological gambling: Psychometric evaluation. Addictive Behaviors, 29(8), 1685–1690.CrossRefPubMed
Zurück zum Zitat Holbrook, A., Cho, Y. I., & Johnson, T. (2006). The impact of question and respondent characteristics on comprehension and mapping difficulties. Public Opinion Quarterly, 70, 565–595.CrossRef Holbrook, A., Cho, Y. I., & Johnson, T. (2006). The impact of question and respondent characteristics on comprehension and mapping difficulties. Public Opinion Quarterly, 70, 565–595.CrossRef
Zurück zum Zitat Kincaid, J. P., Fishburne Jr, R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel (No. RBR-8–75). Naval Technical Training Command Millington TN Research Branch. Kincaid, J. P., Fishburne Jr, R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel (No. RBR-8–75). Naval Technical Training Command Millington TN Research Branch.
Zurück zum Zitat Kirsch, I. S. (1993). Adult literacy in America: A first look at the results of the National Adult Literacy Survey. US Government Printing Office, Superintendent of Documents, Washington, DC 20402 (Stock No. 065–000–00588–3). Kirsch, I. S. (1993). Adult literacy in America: A first look at the results of the National Adult Literacy Survey. US Government Printing Office, Superintendent of Documents, Washington, DC 20402 (Stock No. 065–000–00588–3).
Zurück zum Zitat Ledgerwood, D. M., & Arfken, C. L. (2017). Assessing the need for higher levels of care among problem gambling outpatients. Journal of Gambling Studies, 33, 1–13.CrossRef Ledgerwood, D. M., & Arfken, C. L. (2017). Assessing the need for higher levels of care among problem gambling outpatients. Journal of Gambling Studies, 33, 1–13.CrossRef
Zurück zum Zitat Lenzner, T. (2012). Effects of survey question comprehensibility on response quality. Field Methods, 24, 409–428.CrossRef Lenzner, T. (2012). Effects of survey question comprehensibility on response quality. Field Methods, 24, 409–428.CrossRef
Zurück zum Zitat Lenzner, T. (2014). Are readability formulas valid tools for assessing survey question difficulty? Sociological Methods & Research, 43(4), 677–698.CrossRef Lenzner, T. (2014). Are readability formulas valid tools for assessing survey question difficulty? Sociological Methods & Research, 43(4), 677–698.CrossRef
Zurück zum Zitat McHugh, R. K., Sugarman, D. E., Kaufman, J. S., Park, S., Weiss, R. D., & Greenfield, S. F. (2014). Readability of self-report alcohol misuse measures. Journal of Studies on Alcohol and Drugs, 75(2), 328–334.CrossRefPubMedPubMedCentral McHugh, R. K., Sugarman, D. E., Kaufman, J. S., Park, S., Weiss, R. D., & Greenfield, S. F. (2014). Readability of self-report alcohol misuse measures. Journal of Studies on Alcohol and Drugs, 75(2), 328–334.CrossRefPubMedPubMedCentral
Zurück zum Zitat McLaughlin, G. H. (1969). SMOG grading-a new readability formula. Journal of Reading, 12(8), 639–646. McLaughlin, G. H. (1969). SMOG grading-a new readability formula. Journal of Reading, 12(8), 639–646.
Zurück zum Zitat McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press.CrossRef McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press.CrossRef
Zurück zum Zitat National Research Council. (1999). Pathological gambling: A critical review. National Academies Press. National Research Council. (1999). Pathological gambling: A critical review. National Academies Press.
Zurück zum Zitat Nehlin, C., Nyberg, F., & Jess, K. (2016). Brief Intervention within primary care for at-risk gambling: A pilot study. Journal of Gambling Studies, 32(4), 1327–1335.CrossRefPubMed Nehlin, C., Nyberg, F., & Jess, K. (2016). Brief Intervention within primary care for at-risk gambling: A pilot study. Journal of Gambling Studies, 32(4), 1327–1335.CrossRefPubMed
Zurück zum Zitat Otto, J. L., Smolenski, D. J., Garvey, A. L., Wilson, D. P., Evatt, M. S., Campbell, E. H., Beech, D. E., Workman, R. L., Morgan, K. O. ’G., & Belsher, B. E. (2020). A systematic review evaluating screening instruments for gambling disorder finds lack of adequate evidence. Journal of Clinical Epidemiology, 120, 86–93.CrossRefPubMed Otto, J. L., Smolenski, D. J., Garvey, A. L., Wilson, D. P., Evatt, M. S., Campbell, E. H., Beech, D. E., Workman, R. L., Morgan, K. O. ’G., & Belsher, B. E. (2020). A systematic review evaluating screening instruments for gambling disorder finds lack of adequate evidence. Journal of Clinical Epidemiology, 120, 86–93.CrossRefPubMed
Zurück zum Zitat Peter, S. C., Whelan, J. P., Pfund, R. A., & Meyers, A. W. (2018). A text comprehension approach to questionnaire readability: An example using gambling disorder measures. Psychological Assessment, 30, 1567–1580.CrossRefPubMed Peter, S. C., Whelan, J. P., Pfund, R. A., & Meyers, A. W. (2018). A text comprehension approach to questionnaire readability: An example using gambling disorder measures. Psychological Assessment, 30, 1567–1580.CrossRefPubMed
Zurück zum Zitat Rikard, R. V., Thompson, M. S., McKinney, J., & Beauchamp, A. (2016). Examining health literacy disparities in the United States: A third look at the National Assessment of Adult Literacy (NAAL). BMC Public Health, 16(1), 975.CrossRefPubMedPubMedCentral Rikard, R. V., Thompson, M. S., McKinney, J., & Beauchamp, A. (2016). Examining health literacy disparities in the United States: A third look at the National Assessment of Adult Literacy (NAAL). BMC Public Health, 16(1), 975.CrossRefPubMedPubMedCentral
Zurück zum Zitat Rogier, G., Picci, G., & Velotti, P. (2019). Struggling with happiness: A pathway leading depression to gambling disorder. Journal of Gambling Studies, 35, 293–305.CrossRefPubMed Rogier, G., Picci, G., & Velotti, P. (2019). Struggling with happiness: A pathway leading depression to gambling disorder. Journal of Gambling Studies, 35, 293–305.CrossRefPubMed
Zurück zum Zitat Rumpf, H.-J., Brandt, D., Demetrovics, Z., Billieux, J., Carragher, N., Brand, M., Bowden-Jones, H., Rahimi-Movaghar, A., Assanangkornchai, S., Glavak-Tkalic, R., Borges, G., Lee, H.-K., Rehbein, F., Fineberg, N. A., Mann, K., Potenza, M. N., Stein, D. J., Higuchi, S., King, D., … Poznyak, V. (2019). Epidemiological challenges in the study of behavioral addictions: A call for high standard methodologies. Current Addictions Reports, 6, 331–337.CrossRef Rumpf, H.-J., Brandt, D., Demetrovics, Z., Billieux, J., Carragher, N., Brand, M., Bowden-Jones, H., Rahimi-Movaghar, A., Assanangkornchai, S., Glavak-Tkalic, R., Borges, G., Lee, H.-K., Rehbein, F., Fineberg, N. A., Mann, K., Potenza, M. N., Stein, D. J., Higuchi, S., King, D., … Poznyak, V. (2019). Epidemiological challenges in the study of behavioral addictions: A call for high standard methodologies. Current Addictions Reports, 6, 331–337.CrossRef
Zurück zum Zitat Tabachnick, B. L., & Fidell, L. S. (2007). Using multivariate statistics. Pearson Education Inc. Tabachnick, B. L., & Fidell, L. S. (2007). Using multivariate statistics. Pearson Education Inc.
Zurück zum Zitat Volberg, R. A., McNamara, L. M., & Carris, K. L. (2018). Risk factors for problem gambling in California: Demographics, comorbidities and gambling participation. Journal of Gambling Studies, 34(2), 361–377.CrossRefPubMed Volberg, R. A., McNamara, L. M., & Carris, K. L. (2018). Risk factors for problem gambling in California: Demographics, comorbidities and gambling participation. Journal of Gambling Studies, 34(2), 361–377.CrossRefPubMed
Zurück zum Zitat Weinstock, J., April, L. M., & Kallmi, S. (2017). Is subclinical gambling really subclinical? Addictive Behaviors, 73, 185.CrossRefPubMed Weinstock, J., April, L. M., & Kallmi, S. (2017). Is subclinical gambling really subclinical? Addictive Behaviors, 73, 185.CrossRefPubMed
Zurück zum Zitat Welte, J. W., Barnes, G. M., Tidwell, M. C. O., Hoffman, J. H., & Wieczorek, W. F. (2015). Gambling and problem gambling in the United States: Changes between 1999 and 2013. Journal of Gambling Studies, 31, 695–715.CrossRefPubMedPubMedCentral Welte, J. W., Barnes, G. M., Tidwell, M. C. O., Hoffman, J. H., & Wieczorek, W. F. (2015). Gambling and problem gambling in the United States: Changes between 1999 and 2013. Journal of Gambling Studies, 31, 695–715.CrossRefPubMedPubMedCentral
Zurück zum Zitat Wickwire, E. M., Jr., Burke, R. S., Brown, S. A., Parker, J. D., & May, R. K. (2008). Psychometric evaluation of the national opinion research center DSM-IV screen for gambling problems (NODS). American Journal on Addictions, 17(5), 392–395.CrossRefPubMed Wickwire, E. M., Jr., Burke, R. S., Brown, S. A., Parker, J. D., & May, R. K. (2008). Psychometric evaluation of the national opinion research center DSM-IV screen for gambling problems (NODS). American Journal on Addictions, 17(5), 392–395.CrossRefPubMed
Metadaten
Titel
Text Comprehension Analyses to Improve Assessment Accuracy: Demonstration Using Gambling Disorder Screening
verfasst von
Samuel C. Peter
James P. Whelan
Rory A. Pfund
Publikationsdatum
24.02.2022
Verlag
Springer US
Erschienen in
Journal of Gambling Studies / Ausgabe 4/2022
Elektronische ISSN: 1573-3602
DOI
https://doi.org/10.1007/s10899-022-10110-0

Neu im Fachgebiet Psychiatrie

MCI-Screening per Handy und Smartwatch? Könnte klappen

Handys und Smartwatches könnten niedrigschwellige Tools sein, um eine beginnende Demenz zu erfassen. In einer US-Studie ließ sich damit eine leichte kognitive Beeinträchtigung (MCI) gut nachweisen. Auch waren gerade die älteren Teilnehmenden sehr motiviert.

Demenz-Leitlinie aktualisiert

  • 13.03.2025
  • Demenz
  • Nachrichten

Empfohlen wird jetzt auch eine Musiktherapie sowie bei unklarem diagnostischem Befund ein Tau-PET, und Depressive mit leichten kognitiven Störungen sollten eine Verhaltenstherapie erhalten – das sind einige wichtige Änderungen der aktuellen S3-Leitlinie Demenzen.

Positive Signale für Semaglutid bei Alkoholabhängigkeit

Daten einer kleinen Phase-II-Studie sprechen dafür, GLP-1-Rezeptoragonisten in der Behandlung von Alkoholabhängigkeit weiter zu untersuchen.

Wer hat Angst vorm elektromagnetischen Feld?

Ist elektromagnetische Strahlung auch unterhalb von gesetzlichen Grenzwerten gesundheitsschädlich? Ein kleiner Teil der Ärzteschaft hält das für möglich, wobei eher an psychosomatische Effekte gedacht wird, wie eine deutsche Studie nahelegt. Das selbst eingeschätzte Wissen zum Thema ist indes gering.