Skip to main content
Erschienen in: Psychiatric Quarterly 4/2018

01.05.2018 | Original Paper

Automated Risk Assessment for School Violence: a Pilot Study

verfasst von: Drew Barzman, Yizhao Ni, Marcus Griffey, Alycia Bachtel, Kenneth Lin, Hannah Jackson, Michael Sorter, Melissa DelBello

Erschienen in: Psychiatric Quarterly | Ausgabe 4/2018

Einloggen, um Zugang zu erhalten

Abstract

School violence has increased over the past ten years. This study evaluated students using a more standard and sensitive method to help identify students who are at high risk for school violence. 103 participants were recruited through Cincinnati Children’s Hospital Medical Center (CCHMC) from psychiatry outpatient clinics, the inpatient units, and the emergency department. Participants (ages 12–18) were active students in 74 traditional schools (i.e. non-online education). Collateral information was gathered from guardians before participants were evaluated. School risk evaluations were performed with each participant, and audio recordings from the evaluations were later transcribed and manually annotated. The BRACHA (School Version) and the School Safety Scale (SSS), both 14-item scales, were used. A template of open-ended questions was also used. This analysis included 103 participants who were recruited from 74 different schools. Of the 103 students evaluated, 55 were found to be moderate to high risk and 48 were found to be low risk based on the paper risk assessments including the BRACHA and SSS. Both the BRACHA and the SSS were highly correlated with risk of violence to others (Pearson correlations>0.82). There were significant differences in BRACHA and SSS total scores between low risk and high risk to others groups (p-values <0.001 under unpaired t-test). In particular, there were significant differences in individual SSS items between the two groups (p-value <0.001). Of these items, Previous Violent Behavior (Pearson Correlation = 0.80), Impulsivity (0.69), School Problems (0.64), and Negative Attitudes (0.61) were positively correlated with risk to others. The novel machine learning algorithm achieved an AUC of 91.02% when using the interview content to predict risk of school violence, and the AUC increased to 91.45% when demographic and socioeconomic data were added. Our study indicates that the BRACHA and SSS are clinically useful for assessing risk for school violence. The machine learning algorithm was highly accurate in assessing school violence risk.
Literatur
1.
Zurück zum Zitat Musu-Gillette, L., Zhang, A., Wang, K., Zhang, J., and Oudekerk, B.A. (2017). Indicators of school crime and safety: 2016. National Center for Education Statistics, U.S. Department of Education, and Bureau of Justice Statistics, Office of Justice Programs, U.S. Department of Justice. Washington, DC. Musu-Gillette, L., Zhang, A., Wang, K., Zhang, J., and Oudekerk, B.A. (2017). Indicators of school crime and safety: 2016. National Center for Education Statistics, U.S. Department of Education, and Bureau of Justice Statistics, Office of Justice Programs, U.S. Department of Justice. Washington, DC.
2.
Zurück zum Zitat Centers for Disease Control and Prevention (CDC), 1992–2014 School-Associated Violent Death Surveillance System (SAVD-SS), retrieved July 2016 from http://www.cdc.gov/injury/wisqars/index.html; and Federal Bureau of Investigation and Bureau of Justice Statistics, Supplementary Homicide Reports (SHR), preliminary data (August 2016). Centers for Disease Control and Prevention (CDC), 1992–2014 School-Associated Violent Death Surveillance System (SAVD-SS), retrieved July 2016 from http://​www.​cdc.​gov/​injury/​wisqars/​index.​html; and Federal Bureau of Investigation and Bureau of Justice Statistics, Supplementary Homicide Reports (SHR), preliminary data (August 2016).
7.
Zurück zum Zitat Gottfredson GD, Cook PJ, NA C. Schools and prevention. In: Welsh BC, Farrington DP, editors. Crime and prevention. Oxford, United Kingdom: Oxford University Press; 2000. p. 269–87. Gottfredson GD, Cook PJ, NA C. Schools and prevention. In: Welsh BC, Farrington DP, editors. Crime and prevention. Oxford, United Kingdom: Oxford University Press; 2000. p. 269–87.
8.
Zurück zum Zitat Tanner-Smith EE, Wilson SJ, Lipsey MW. Risk factors and crime. In: Maguire M, Morgan R, Reiner R, editors. The Oxford handbook of criminology. 5th edn. Oxford: Oxford University Press; 2012. p. 89–111. Tanner-Smith EE, Wilson SJ, Lipsey MW. Risk factors and crime. In: Maguire M, Morgan R, Reiner R, editors. The Oxford handbook of criminology. 5th edn. Oxford: Oxford University Press; 2012. p. 89–111.
9.
Zurück zum Zitat Mytton J, DiGuiseppi C, Gough D, Taylor R, Logan S. School-based secondary prevention programmes for preventing violence. Cochrane Database Syst Rev. 2006 Jul 19;3 CD004606 Mytton J, DiGuiseppi C, Gough D, Taylor R, Logan S. School-based secondary prevention programmes for preventing violence. Cochrane Database Syst Rev. 2006 Jul 19;3 CD004606
11.
Zurück zum Zitat Borum R, Cornell DG, Modzeleski W, Jimerson SR. What can be done about school shootings? A review of the evidence. Educ Res. 2010;39(1):27–37.CrossRef Borum R, Cornell DG, Modzeleski W, Jimerson SR. What can be done about school shootings? A review of the evidence. Educ Res. 2010;39(1):27–37.CrossRef
12.
Zurück zum Zitat Nekvasil EK, Cornell DG. Student reports of peer threats of violence: prevalence and outcomes. J Sch Violence. 2012;11(4):357–75.CrossRef Nekvasil EK, Cornell DG. Student reports of peer threats of violence: prevalence and outcomes. J Sch Violence. 2012;11(4):357–75.CrossRef
13.
Zurück zum Zitat Bernes KB, Bardick AD. Conducting adolescent violence risk assessments: a framework for school counselors. Prof Sch Couns. 2007;10(4):419–27.CrossRef Bernes KB, Bardick AD. Conducting adolescent violence risk assessments: a framework for school counselors. Prof Sch Couns. 2007;10(4):419–27.CrossRef
14.
Zurück zum Zitat McGowan MR, Horn RA, Mellott RN. The predictive validity of the structured assessment of violence risk in youth in secondary educational settings. Psychol Assess. 2011;23(2):478–86.CrossRefPubMed McGowan MR, Horn RA, Mellott RN. The predictive validity of the structured assessment of violence risk in youth in secondary educational settings. Psychol Assess. 2011;23(2):478–86.CrossRefPubMed
16.
Zurück zum Zitat Barzman D, Brackenbury L, Sonnier L, Schnell B, Cassedy A, Salisbury S, et al. Brief rating of aggression by children and adolescents (BRACHA): development of a tool to assess risk of inpatients’ aggressive behavior. J Am Acad Psychiatry Law. 2011;39(2):170–9.PubMed Barzman D, Brackenbury L, Sonnier L, Schnell B, Cassedy A, Salisbury S, et al. Brief rating of aggression by children and adolescents (BRACHA): development of a tool to assess risk of inpatients’ aggressive behavior. J Am Acad Psychiatry Law. 2011;39(2):170–9.PubMed
17.
Zurück zum Zitat Xia F, Yetisgen-Yildiz: Clinical corpus annotation: challenges and strategies. Proc. Of Third Workshop on Building and Evaluating Resources for Biomedical Text Mining of the International Conference on Language Resources and Evaluation, 2012. Xia F, Yetisgen-Yildiz: Clinical corpus annotation: challenges and strategies. Proc. Of Third Workshop on Building and Evaluating Resources for Biomedical Text Mining of the International Conference on Language Resources and Evaluation, 2012.
19.
Zurück zum Zitat Wilbur WJ, Rzhetsky A, Shatkay H. New directions in biomedical text annotation: definitions, guidelines and corpus construction. BMC Bioinformatics. 2006 Jul 25;7:356.CrossRefPubMedCentralPubMed Wilbur WJ, Rzhetsky A, Shatkay H. New directions in biomedical text annotation: definitions, guidelines and corpus construction. BMC Bioinformatics. 2006 Jul 25;7:356.CrossRefPubMedCentralPubMed
20.
Zurück zum Zitat Barzman D, Mossman D, Sonnier L, Sorter M. Brief rating of aggression by children and adolescents (BRACHA): a reliability study. J Am Acad Psychiatry Law. 2012;40:374–82.PubMed Barzman D, Mossman D, Sonnier L, Sorter M. Brief rating of aggression by children and adolescents (BRACHA): a reliability study. J Am Acad Psychiatry Law. 2012;40:374–82.PubMed
21.
Zurück zum Zitat Barzman DH, Ni Y, Griffey M, Patel B, Warren A, Latessa E, et al. A pilot study on developing a standardized and sensitive school violence risk assessment with manual annotation. Psychiatry Q. 2017;88(3):447–57.CrossRef Barzman DH, Ni Y, Griffey M, Patel B, Warren A, Latessa E, et al. A pilot study on developing a standardized and sensitive school violence risk assessment with manual annotation. Psychiatry Q. 2017;88(3):447–57.CrossRef
23.
Zurück zum Zitat Delgado SV, Barzman D, Gehle M, Caring M, Sorter MD, Kowatch R, et al. Characteristics of discharges against medical advice from acute inpatient psychiatric units for children and adolescents. Boston: Poster presented at the annual meeting of the American Academy of Child and Adolescent Psychiatry; 2007. Delgado SV, Barzman D, Gehle M, Caring M, Sorter MD, Kowatch R, et al. Characteristics of discharges against medical advice from acute inpatient psychiatric units for children and adolescents. Boston: Poster presented at the annual meeting of the American Academy of Child and Adolescent Psychiatry; 2007.
24.
Zurück zum Zitat Hilterman EL, Nicholls TL, van Nieuwenhuizen C: Predictive performance of risk assessments in juvenile offenders: comparing the SAVRY, PCL:YV, and YLS/CMI with unstructured clinical assessments. Assessment, 2014, 21, 324, 339. Hilterman EL, Nicholls TL, van Nieuwenhuizen C: Predictive performance of risk assessments in juvenile offenders: comparing the SAVRY, PCL:YV, and YLS/CMI with unstructured clinical assessments. Assessment, 2014, 21, 324, 339.
26.
Zurück zum Zitat Lingren T, Deleger L, Molnar K, Zhai H, Meinzen-Derr J, Kaiser M, et al. Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements. J Am Med Inform Assoc. 2014;2013 https://doi.org/10.1136/amiajnl-2013-001837.CrossRefPubMed Lingren T, Deleger L, Molnar K, Zhai H, Meinzen-Derr J, Kaiser M, et al. Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements. J Am Med Inform Assoc. 2014;2013 https://​doi.​org/​10.​1136/​amiajnl-2013-001837.CrossRefPubMed
27.
Zurück zum Zitat Deleger L, Molnar K, Savova G, Xia F, Lingren T, Li Q, et al. Large-scale evaluation of automated clinical note de-identification and its impact on information extraction. J Am Med Inform Assoc. 2012;20(1):84–94.CrossRefPubMed Deleger L, Molnar K, Savova G, Xia F, Lingren T, Li Q, et al. Large-scale evaluation of automated clinical note de-identification and its impact on information extraction. J Am Med Inform Assoc. 2012;20(1):84–94.CrossRefPubMed
28.
Zurück zum Zitat Ganzert S, Guttmann J, Kersting K, Kuhlen R, Putensen C, Sydow M, et al. Analysis of respiratory pressure-volume curves in intensive care medicine using inductive machine learning. Artif Intell Med. 2002;26(1):69–86.CrossRefPubMed Ganzert S, Guttmann J, Kersting K, Kuhlen R, Putensen C, Sydow M, et al. Analysis of respiratory pressure-volume curves in intensive care medicine using inductive machine learning. Artif Intell Med. 2002;26(1):69–86.CrossRefPubMed
29.
Zurück zum Zitat Zacharaki EI, Wang S, Chawla S, Soo Yoo D, Wolf R, Melhem ER, et al. Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magn Reson Med. 2009;62(6):1609–18.CrossRefPubMedCentralPubMed Zacharaki EI, Wang S, Chawla S, Soo Yoo D, Wolf R, Melhem ER, et al. Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magn Reson Med. 2009;62(6):1609–18.CrossRefPubMedCentralPubMed
30.
Zurück zum Zitat Zrimec, T., & Kononenko, I. (2004). Feasibility analysis of machine learning in medical diagnosis from aura images. In Proc. Int. Conf. KIRLIONICS-98 (Abstracts) (pp. 10–11). Zrimec, T., & Kononenko, I. (2004). Feasibility analysis of machine learning in medical diagnosis from aura images. In Proc. Int. Conf. KIRLIONICS-98 (Abstracts) (pp. 10–11).
31.
Zurück zum Zitat Sara, NB, Halland R, Igel C, Alstrup S. High-school dropout prediction using machine learning: a Danish large-scale study European symposium on artificial neural networks, computational intelligence and machine learning. Bruges (Belgium), 22–24 April 2015. Sara, NB, Halland R, Igel C, Alstrup S. High-school dropout prediction using machine learning: a Danish large-scale study European symposium on artificial neural networks, computational intelligence and machine learning. Bruges (Belgium), 22–24 April 2015.
32.
Zurück zum Zitat Welsh JL, Schmidt F, McKinnon L, Chattha HK, Meyers JR A comparative study of adolescent risk assessment instruments: predictive and incremental validity assessment. 2008 Mar;15(1):104–15. Welsh JL, Schmidt F, McKinnon L, Chattha HK, Meyers JR A comparative study of adolescent risk assessment instruments: predictive and incremental validity assessment. 2008 Mar;15(1):104–15.
34.
Zurück zum Zitat Reed MO, Jakubovski E, Johnson JA, Bloch MH. Predictor of long-term school-based behavioral outcomes in the multimodal treatment study of children with attention-deficit/hyperactivity disorder. J Child Adolesc Psychopharmacol. 27(4):296–309.CrossRefPubMedCentralPubMed Reed MO, Jakubovski E, Johnson JA, Bloch MH. Predictor of long-term school-based behavioral outcomes in the multimodal treatment study of children with attention-deficit/hyperactivity disorder. J Child Adolesc Psychopharmacol. 27(4):296–309.CrossRefPubMedCentralPubMed
35.
Zurück zum Zitat Singh JP, Grann M, Fazel S. A comparative study of violence risk assessment tools: a systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clin Psychol Rev. 2011;31:499–513.CrossRefPubMed Singh JP, Grann M, Fazel S. A comparative study of violence risk assessment tools: a systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clin Psychol Rev. 2011;31:499–513.CrossRefPubMed
36.
Zurück zum Zitat Mossman D. Assessing prediction of violence: being accurate about accuracy. J Consult Clin Psychol. 1994;62(4):783–92.CrossRefPubMed Mossman D. Assessing prediction of violence: being accurate about accuracy. J Consult Clin Psychol. 1994;62(4):783–92.CrossRefPubMed
37.
Zurück zum Zitat Janofsky JS, Spears S, Neubauer DN. Psychiatrists' accuracy in predicting violent behavior on an inpatient unit. Hosp Community Psychiatry. 1988;39:1090–4.PubMed Janofsky JS, Spears S, Neubauer DN. Psychiatrists' accuracy in predicting violent behavior on an inpatient unit. Hosp Community Psychiatry. 1988;39:1090–4.PubMed
38.
Zurück zum Zitat Neuman Y, Assaf D, Cohen Y, Knoll J. Profiling school shooters: automatic text-based analysis. Front Psych. 2015;6:1–5. Neuman Y, Assaf D, Cohen Y, Knoll J. Profiling school shooters: automatic text-based analysis. Front Psych. 2015;6:1–5.
39.
Zurück zum Zitat Shultz JM, Cohen AM, Muschert GW, Flores de Apodaca R. Fatal school shootings and the epidemiological context of firearm mortality in the United States. Disaster Health. 2013 Apr-Dec;1(2):84–101.CrossRefPubMedCentralPubMed Shultz JM, Cohen AM, Muschert GW, Flores de Apodaca R. Fatal school shootings and the epidemiological context of firearm mortality in the United States. Disaster Health. 2013 Apr-Dec;1(2):84–101.CrossRefPubMedCentralPubMed
40.
Zurück zum Zitat Flannery DJ, Modzeleski W, Kretschmar JM. Violence and school shootings. Curr Psychiatry Rep. 2013 Jan;15(1):331.CrossRefPubMed Flannery DJ, Modzeleski W, Kretschmar JM. Violence and school shootings. Curr Psychiatry Rep. 2013 Jan;15(1):331.CrossRefPubMed
Metadaten
Titel
Automated Risk Assessment for School Violence: a Pilot Study
verfasst von
Drew Barzman
Yizhao Ni
Marcus Griffey
Alycia Bachtel
Kenneth Lin
Hannah Jackson
Michael Sorter
Melissa DelBello
Publikationsdatum
01.05.2018
Verlag
Springer US
Erschienen in
Psychiatric Quarterly / Ausgabe 4/2018
Print ISSN: 0033-2720
Elektronische ISSN: 1573-6709
DOI
https://doi.org/10.1007/s11126-018-9581-8

Weitere Artikel der Ausgabe 4/2018

Psychiatric Quarterly 4/2018 Zur Ausgabe

Update Psychiatrie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.