Trends in concussions at Ontario schools prior to and subsequent to the introduction of a concussion policy - an analysis of the Canadian hospitals injury reporting and prevention program from 2009 to 2016

The Canadian Hospitals Injury Reporting and Prevention Program (CHIRPP) was selected as the data source for this study for several reasons. First, when examining data from the OSBIE, concussions appeared to be under-reported. Second CHIRPP is an injury and poisoning surveillance system that is based on data drawn from the emergency rooms of 11 pediatric hospitals and 6 general hospitals in Canada [24]. It began in 1990 and has since accumulated more than 2.8 million records nationally, with more than 80% of them being on individuals aged 19 and younger. The main objective of CHIRPP is to reduce the number and severity of injuries in Canada [26]. Any time an injured person presents to the emergency room of a participating hospital, he/she (or caregiver) is asked to complete a short one-page questionnaire that requests detailed information on the nature of their injury. These questions deal with what caused the injury, what the activity was at the time of injury, the time and location of the injury, as well as sex and age. The reverse side of the CHIRPP Form is completed by hospital staff, providing details on the nature of the injury, diagnosis, injured body part and treatment received [26]. The completed forms are entered into an electronic database by trained personnel who enter and code information on more than 40 variables and write a short overview on what transpired based on the account of the injured individual. Thus it is possible to include children who sustained an injury at school.

Data for this study was based on five children’s hospitals in the province of Ontario. These hospitals were the Children’s Hospital of Eastern Ontario in Ottawa, Hotel Dieu Hospital, including Children’s Outpatient Clinic in Kingston, Kingston General Hospital in Kingston, The Hospital for Sick Children in Toronto, and the Children’s Hospital at London Health Sciences Centre in London, ON. Children and youth aged 4 to 18 who were treated at these hospitals between 2009 and 2016 were included in this study. The years 2009–2010 were the most recent years that included complete data in the old CHIRPP system, before it switched to an electronic version in 2011; thus the 8 years represented the most recent complete database. Individuals that were diagnosed with either a minor closed head injury or a concussion were included. Minor closed head injuries were defined as ‘suspected concussions’, signifying that even though they were not diagnosed as concussion at the time, they were still a cause for concern. A secondary analysis was also conducted on injuries that occurred in school, because the PPM only required concussion prevention and education in school boards. Beyond the switch to the electronic version in 2011, there were no known changes to the population or in the ways CHIRPP hospitals diagnosed concussions. The authors could not identify any other contextual changes that coincided with the introduction of the PPM.

This study’s primary outcome measure was a suspected or confirmed concussion, which included both minor head injuries and diagnosed concussions. The exposure variables were age, sex, month of injury, location of injury (school), and where in the school the injury took place. Age groups were separated into three categories: 4–9, 10–14 and 15–18, to align with developmental stages. The percent of diagnosed versus suspected concussions was examined for all locations and those that took place at a school, including examining where in the school the injury occurred. Information related to the school’s location was not available in CHIRPP. For the purposes of this study, we assumed that the children attended the school boards within the hospitals’ catchment areas, but we had no way to validate this assumption.

Descriptive statistics were calculated using Statistical Analysis System (SAS) version 9.4 and Microsoft Excel. Data were only available until August 2016, making it an incomplete year. Fall 2016 was thus estimated for graphing purposes only using a linear approximation model. We compared the rates between diagnosed and suspected youth concussions, and for diagnosed concussions pre−/post- introduction of the PPM (March 2014), both generally and for school-incurred injuries only. Statistical significance was set at p < 0.05, and differences between means were based on a two-tailed t-test. In addition, a segmented linear regression analysis for an interrupted time series was performed to identify whether the introduction of the PPM #158 had an effect on the observed trends. The two independent variables were “Time” (measured in months) and “Policy” (the introduction of PPM #158). The outcome variable was the number of ED visits for diagnosed school-incurred concussions. Maximum likelihood estimation was used with an autoregressive parameters assumed given using PROC AUTOREG procedure in SAS (Additional file 1). Ethics approval was obtained from York University and the Public Health Agency of Canada.

In total, between January 2009 and August 2016, there were 21,094 children and youth treated in participating EDs for head injuries, including 12,159 suspected concussion and 8935 diagnosed concussions. In total, there were 164,766 ED injury-related visits recorded in CHIRPP during this time period, with diagnosed concussions accounting for 5.4% of injury-related visits and minor closed head injuries accounting for 7.4%. In 2009, there were 610 (31%) diagnosed concussions out of 1969 suspected concussions. This number rose to 974 diagnosed out of 1822 suspected concussions (53.45%) in 2016 (Table 1). The actual number of emergency department visits for suspected concussions started to decrease in January–February 2014, just before the PPM was introduced (March 2014) but the number of diagnosed concussions remained similar (Fig. 1). The monthly average number of suspected concussions before and including the March 2014 introduction of the PPM was 232 per month. The average number of suspected concussions after the introduction of the PPM was slightly lower at 229 per month. The average rate of diagnosed concussions before the March 2014 PPM was about 89 concussions/month. Subsequent to the introduction of the PPM this number increased significantly by more than 30% to about 117 concussions per month (p < 0.001).

The majority or 60.5% of the whole sample with diagnosed concussions were male but the difference between the sexes started to decrease in 2013–2014 (Table 1). A total of 46.4% of all diagnosed concussions were in individuals aged 10–14. The age group with the least amount of injuries, constituting 23% of the whole sample was children aged 4–9.

For the school-incurred diagnosed concussions data, the two summer months, July and August were removed from the analysis. The difference between possible (suspected) concussions before/after the PPM is not statistically significant (< 1% increase). However, the difference between diagnosed concussions was almost twofold. Prior to the PPM the number of average monthly diagnosed concussions was about 23/month, then in the months following the PPM this number rose to 44 concussions per month (excluding the estimated fall months). Figure 2 shows the percentage of ED visits for school-incurred diagnosed concussions for each month, and Fig. 3 shows the number of school-incurred diagnosed concussions relative to all locations incurred diagnosed concussions by year. Averaged out on a monthly basis, school-based concussions accounted for about 28% of all ED diagnosed concussions before and including March 2014 (PPM intro date). However, for the post-PPM months, this number rose to almost 42%. This before-after difference in the proportion of ED school-incurred diagnosed concussions was significant at p < 0.001. Overall school-incurred concussions accounted for more than 34% of all diagnosed concussions (Table 1). Some months had an almost 60% school-incurred diagnosed concussion rate out of all diagnosed concussion ED visits. The proportion of all school-diagnosed concussions increased throughout the study reaching almost 50% in 2016.

Table 2 indicates locations within schools where diagnosed concussions occurred. The greatest proportion of all school-based, diagnosed concussions, occurred on the playground (23.7%), followed by the gymnasium (22.1%) and the sport field (20.2%). The hallway variable, which included the waiting area, foyer, and emergency room was combined with stairs and ramps. The lowest number of concussions happened in other school areas. These areas were collapsed into one category for ease of interpretation and included: the roadway, sidewalk/bus stop waiting area, school parking lot, bathroom, dining area, kitchen, office, veranda/porch, and other unspecified exterior/interior areas. Most of these areas had fewer than five diagnosed concussions over the 8-year study period.

Table 3 shows the partial output from running the AUTOREG program. A maximum likelihood model was used, with backstep stepwise selection option, and using the log likelihood value for the model in the output, with assumed lag time set to 12. Overall, the rates of ED school-incurred diagnosed concussion visits were slowly trending upward at a statistically significant degree. The coefficient for the “policy” was significant, as could be also seen from Fig. 4, indicating an associated increase in the rate of ED school-incurred diagnosed concussion visits after the introduction of PPM #158.

CHIRPP does not represent all ED visits for youth concussions in Ontario since it only accounts for five hospitals. Thus, this study was not able to capture children that were treated elsewhere, and CHIRPP is known to have a variable capture-rate, explained elsewhere [42]. Thus, there is likely an underestimation selection bias, whereby the findings in this paper are conservative and under-representing the population at risk. However, we believe that the numbers are somewhat representative of Ontario children, especially when compared to OSBIE data. For example, in our CHIRPP school data in 2011 there were 265 confirmed, diagnosed concussions and 510 suspected concussions. OSBIE reported 634 instances of concussions or possible concussions, from all participating Ontario schools [43]. The 265 confirmed CHIRPP concussions were reported in only the five participating Ontario hospitals. That five hospitals reported more ED visits for concussion than official OSBIE reports for almost five thousand schools suggests significant underreporting or underestimation of youth concussion rates [44, 45]. Since OSBIE often works together with the Ontario Physical and Health Education Association (OPHEA) and other sports and athletics organizations across the province to develop various safety guidelines, it uses the insurance reports as the backbone or foundation for these guidelines [46]. However one report found evidence that actual youth concussion rates, at least in some sports (e.g. hockey) are 40 times greater than the officially reported OSBIE numbers [46]. It is unclear why this is the case but it raises questions about using insurance data for surveillance, and suggests that other data sources need to be considered when examining concussion trends. Only data for the five pediatric hospitals in Ontario was given to the investigators and there was no possibility to compare data from parts of Canada where there was no concussion policy or protocol in place. The CHIRPP database that was given to the primary investigators did not have a unique identifier for each hospital therefore we were unable to look at ED visits by hospital or perform a nested count analysis. Hospitals in other provinces might have had additional or different concussion regulations or criteria in place that would have made any cross-comparison results questionable. Furthermore, the diagnostic protocols in the five children’s hospitals in Ontario are supposed to be harmonized, although there is always the possibility of differing rates of concussion diagnoses. Nevertheless, Ontario hospitals use standardized diagnostic protocols that have been tested extensively by independent research organizations [31, 47].