A National Study on the Effects of Concussion in Collegiate Athletes and US Military Service Academy Members: The NCAA–DoD Concussion Assessment, Research and Education (CARE) Consortium Structure and Methods

The current CARE Consortium structure is illustrated in Fig. 2. In brief, an Executive Committee consisting of representation from the DoD, NCAA, and National Institutes of Health (NIH) National Institute of Neurological Disorders and Stroke (NINDS) oversee the overall function of the Consortium but have no input into the data collection, analyses, interpretation, or dissemination of any of the Consortium studies. The three CARE principal investigators (PIs) [SB, TM, MM] chair the CARE Consortium Operating Committee, comprised of site PIs of Consortium-participating institutions with expertise in the science and clinical management of concussion. The Operating Committee has representation from a wide array of NCAA conferences and the US Military Service Academies. A Scientific Advisory Board advises the Operating Committee and serves as a resource to the Executive Committee. Composition of the Scientific Advisory Board includes representatives with research and clinical experience in sports and military mild TBI, biostatistics and informatics, and a former NCAA student athlete. An Administrative and Operations Core (AOC), Longitudinal Clinical Study Core (CSC), and Advanced Research Core (ARC) are directed by the three Consortium PIs and their teams at the three lead institutions (Indiana University School of Medicine, University of Michigan, and Medical College of Wisconsin, respectively) and are briefly described below.

The AOC functions as the administrative and operations arm of the CARE consortium. The AOC provides Consortium members with oversight and assistance in regulatory and financial functions, maintains the Consortium’s website (http://​www.​careconsortium.​net), standard operating procedures (SOPs), and supports the various Consortium committees (e.g. the Executive Committee, Scientific Advisory Board, Operating Committee, and Publication Committee). Additionally, the AOC houses three support teams to assist in the aims of the CSC and ARC: Biostatistics and Data Management, Bioinformatics, and Biospecimens.

Housed at the University of Michigan, the CSC oversees the foundational study on the 6-month natural history of concussion. In the Fall of 2014, the CSC initiated 15 performance sites for data collection, with an additional six schools (21 in total) added in the 2015–2016 academic year, and an additional nine sites (30 in total) added for the 2016–2017 academic year (Fig. 2). Prior to data collection at any site, all site personnel were trained on a standardized protocol for preseason baseline testing and post-injury assessments (described below), and school-level Institutional Review Board (IRB) approval and participant consent were obtained. Protocols approved by the IRB of each performance site also underwent review and approval by the DoD Human Research Protections Office (HRPO).

The ARC, coordinated by the Medical College of Wisconsin, oversees the CARE foundational study on the neurobiology of concussion. The specific aim of the ARC is to conduct advanced scientific studies that integrate biomechanical, clinical, neuroimaging, neurobiological, and genetic markers of injury to advance our understanding of neurophysiological effects and recovery after concussion in college student-athletes and military service academy students. To complete this aim, a subset of four NCAA institutions (all years) and two military service academies (year 3) participating in the CSC are conducting a more extensive baseline and post-injury protocol incorporating neuroimaging, neurobiological, and/or genetic studies. Like the CSC, the ARC is a prospective, cohort study that includes pre-exposure baseline assessment of a large sample of enrolled student athletes, as well as acute and longitudinal follow-up assessment of a cohort with concussion, a control group of non-concussed student athletes from the same contact sport, and a non-contact-sport student-athlete control group. Table 1 provides an overview of the ARC procedures completed at baseline and each post-injury time point, over and above the clinical testing protocol included in the CSC. Scientifically, our interest in understanding the effects of returning to full participation in unrestricted activities (including exposure to head impact in contact sports) on neurobiological function prompted conduction of ARC procedures at 7 days following return to play rather than on the date of unrestricted return to play.

The ARC calls for enrolling and baseline testing of approximately 3500 student-athletes and cadets, with a detailed post-injury protocol in a projected 500 concussed student athletes, 225 contact-sport student-athlete controls, and 125 non-contact-sport (e.g. baseball, softball, track and field, etc.) controls. The ARC targets specific sports with a known high incidence of concussion, including men’s football, and men’s and women’s ice hockey, lacrosse, soccer, and rugby. The endpoint of concussed student athletes completing the ARC post-injury protocol was based on a conservative rate of concussion in the ARC-targeted sports at the participating sites, taking into account that not all those concussed would be enrolled (i.e. late reports, unreported injuries, failure to meet the study criteria, etc.) and that some degree of attrition would be encountered over the 6-month longitudinal follow-up phase of the study.

In addition to the concussed cohort, the ARC will enroll matched cohorts of contact-sport control student athletes (non-concussed student athletes with a history of head impact exposure from the same contact sport as the concussed student athlete) and non-contact-sport student athletes (non-concussed student athletes from track and field, baseball, etc., without a history of head impact exposure). Both control groups are matched to the concussed group on a combination of demographic, baseline performance, competitive, and exposure variables. The inclusion of these two control groups will enable CARE to differentiate the neurobiological effects of injury versus no injury with exposure (concussed versus contact-sport control) and exposure without injury versus no exposure or injury (contact-sport controls versus non-contact-sport controls).

As with similar large-scale efforts, attending to regulatory requirements across 30 sites within 3 years, as well as the parent funding agencies, is challenging. Two important strategies facilitated the implementation of this study: development of the template IRB application and informed consent documents, and utilization of an IRB deferral process where possible. The template CSC IRB documents were given initial approval by the University of Michigan IRB, with subsequent approval being issued by the US Army Medical Research and Materiel Command Human Research Protection Office (HRPO), as required by the DoD. The standardized application and consent documents were then distributed to all performance sites for local IRB submission, with only minor modifications allowed based on local IRB policies. The locally approved protocols were also reviewed and approved by the HRPO office. Performance site investigators who wished to conduct similar or concurrent studies obtained free-standing IRB protocol approval for add-on studies that did not interfere with the CARE research. In addition to the standardized forms, HRPO review of both ARC and CSC study documents was further expedited by having a single individual perform all reviews from the Consortium, which maintained review and approval consistency. For those sites also participating in the ARC, site IRBs deferred their review to the Medical College of Wisconsin IRB, and thus only one IRB approval (i.e. the Medical College of Wisconsin) was required to initiate the study. The IRBs of record described above (Indiana University, University of Michigan, Medical College of Wisconsin) operate in accordance with the standards of ethics outlined in the Declaration of Helsinki.

Launched in Fall 2014, the CARE consortium has completed 24 months of data collection. In the first 24 months of operation, 21 CSC performance sites (including the four ARC sites) have been engaged in data collection and have successfully enrolled 23,533 student-athletes and military service academy students (13,720 male, 9813 female) into the study, well on track to exceed enrollment expectations in both the CSC and ARC. During this time, a total of 1174 concussions have been reported in the CSC, with a post-injury data capture rate of 59% within 6 h of injury; 81.6% at the 24–48 h post-injury time point; 84.1% when the student athlete begins the return-to-play protocol; 79.6% at the return-to-play time point; and 75.6% 6-months post-injury (evaluations between 3 and 9 months post-injury). Similarly, enrollment and injury accrual in the ARC are on track at both the NCAA and MSA ARC sites, with more than 1500 participants enrolled and over 90 concussed student athletes studied under the detailed ARC post-injury protocol. Completeness of the ARC data elements (biomarkers, MRI, genetics) at baseline and the post-injury time points has also been high, despite the intense demands of the protocol.

To ensure consistency and quality in data collection, each site was provided with a manual of operations providing additional information for every question in the CRFs and detailed instructions on how to best administer each of the assessments outlined above. In addition to this, all site PIs completed in-person training on the assessments with the study PIs, with the requirement that the information would be relayed to their local study teams. Biannual investigator meetings address data collection strategies and concerns at the group level and with individual sites in order to ensure consistent practices across sites and mitigate rater drift and site effects. Lastly, baseline data collection in the first year was completed using pen and paper CRFs which, while effective, had an additional time cost with hand entry into the centralized database. Moving into the second year of the study, the paper CRFs were converted to an electronic format, allowing for direct entry by the study participant. The shift to direct entry has resulted in, overall, 3.9% less missing data. In addition, the electronic data capture allows for range checks, specified response lists and values, and question/response logic that has resulted in overall higher data quality.

To improve the chances of success, data-quality checks have been implemented into the electronic data capture portal. These include a variety of rules and logic checks with additional data scrubbing completed by the BDMT, and with greater detail in preparation for data analysis. In addition, protocol compliance and data integrity is monitored by yearly informed consent documentation reviews and audits to verify that appropriate informed consent has been obtained from our research participants and to ensure that our documentation is being maintained in a compliant manner. Lastly, site visits have been completed to standardize and monitor student-athlete enrollment, data collection, and data storage procedures.

As in all multicenter neuroimaging studies, CARE has required an intense effort, both centrally within the ARC MRI subcore team of experts and among the ARC site investigators, to ensure maximum standardization, performance, and data quality. Early in the project, the MRI subcore consulted with experts involved in other large-scale, multicenter studies that involved advanced neuroimaging (e.g. ADNI, TRACK-TBI). The collaborative input from these experts was invaluable with respect to sharing of lessons learned in optimizing multimodal imaging protocols across sites and MRI vendor platforms. Our approach toward quality control and integration is designed to closely parallel that of TRACK-TBI, with the goal of eventual cross-comparison of our neuroimaging findings in concussion and TRACK-TBIs in civilian mild TBI.

Additional to developing the specific pulse sequences for our multimodal imaging protocol, the MRI subcore has had to address issues related to MRI scanner upgrades, software updates, head coil functionality and wearability (particularly in large student athletes), and other technical challenges. Furthermore, there was a pressing need to be attentive to the demands on student-athletes and cadets, particularly the total scanning time required at each visit (<1 h). Ultimately, the ARC MRI subcore has followed a philosophy to deploy the most cutting-edge MRI protocol best-suited for the study of concussion, balanced against the technological constraints encountered in optimizing a standardized protocol across multiple sites. Our ability to address new and ongoing issues inherent to multicenter imaging studies in a data-driven and nimble manner has been key to our success.

As noted above, one scientific objective of the ARC is to characterize the concussion biomechanics and establish repeated head impact exposure associated with contact- and collision-sport participation. Although some work has been carried out in characterizing the biomechanics of concussion and head impact exposure associated with contact sports such as American football [56‐58] and ice hockey [19, 20, 59], little work has been carried out in non-helmeted sports such as soccer because of limitations in currently available technology. Two non-helmet-based options evaluated by the ARC team were determined to have issues that are currently preventing implementation. An alternative mouthguard-based system is currently being evaluated for validity, reliability, and student-athlete ‘usability’. Other non-helmet-based systems are not commercially ready and/or do not have a proper data transfer interface that is required for this study [60].

Given the anticipated number of total participants, the depth and breadth of data collected, data collection methodologies, and the sensitivity of the data collected, the data storage platform was carefully considered. Indeed, at the time of writing, over 57,709,940 data points had been collected, making a flat data storage platform ineffective. Instead, a relational database maintained by Quesgen was selected. The CARE data capture application is a scalable, browser-independent, web-based application that provides both a researcher interface allowing for paper/pencil data entry from the CRFs, and a participant portal allowing for direct entry of self-reported measures by the study participants. Extensive measures and controls were implemented to ensure security and privacy of the data collected, in addition to accuracy and quality. The application complies with all Federal (i.e. Health Insurance Portability and Accountability [HIPAA]) and Joint Commission on Accreditation of Healthcare Organizations (JCAHO) standards for data access, integrity, confidentiality, auditing, and security.

As with any data-centric endeavor of this magnitude, analytic challenges are expected. For example, examination of the baseline data revealed skewed data distributions for many of the instruments, some of which have a large percentage of zero values (e.g. no symptoms) among the student athletes. Thus, statistical models for zero-inflated data are required for valid comparisons among different groups of student athletes. The primary questions of interest will be concerned with post-injury data and the recovery periods for the student athletes who had concussions. Missing data early in the recovery period will be an issue to be dealt with since logistic issues and athlete underreporting often make the <6 h evaluation difficult. Survival analysis methods will be used for time to recovery and will have to account for the correlation among concussions for student athletes with more than one injury. Analysis plans for longitudinal data analysis of assessments measured repeatedly during recovery will also need to take these correlations into account, as well as possible non-linear recovery trajectories. Classical distributional assumptions, especially Gaussianity of these measures noted above, is also suspect. Many of the repeated measures are also observed for different lengths of time for different subjects. Thus, for each analysis we anticipate data issues that will both inform and complicate the choice of statistical model to be used [61].