Introduction

Traumatic spinal cord injury (TSCI) is a devastating injury to the patient, family and society, as a TSCI results in multi-system complications, impaired health-related quality of life and poor prognosis for neurologic recovery.1, 2, 3 The incidence of TSCI ranges from 3.6 to 195.4 patients per million around the world4 and 25.5 per million per year in developing countries.5 A prerequisite to the development of effective therapies for TSCI is a detailed understanding of the pathophysiological events occurring after injury. The pathophysiology of TSCI can be well divided into two phases, a primary and a secondary injury. The primary injury involves the initial mechanical damage, which is followed by a delayed onset of a secondary phase comprising a cascade of molecular and cellular events that inhibits the regeneration and also extends the injured site. As a result of the many technical limitations associated with the study of TSCI pathophysiology in humans, almost all of our detailed understanding of TSCI pathophysiology is based on animal studies. Translation of the obtained outcomes to the clinic could be achieved through well-designed animal studies and consideration of the variables, which are decisive in determining the outcomes of animal studies.

A systematic review in the field of evidence-based medicine is the highest level of medical evidence, and accordingly the required standards have already been developed.6 Recently, systematic reviews in the animal experimentation field have attracted great attention,7, 8, 9 although there are not adequate guidelines for assessing the quality of experimental animal studies. In this regard, a few groups of researchers focused on the development of publication guidelines in order to improve the reporting transparency, which makes future systematic reviews more feasible and allows the researchers to replicate previously published papers.10, 11, 12, 13 Also, Hooijmans et al.14 developed SYRCLE’s risk of bias tool based on the Cochrane Collaboration Risk of Bias (RoB) Tool, by introducing aspects of bias that are probable to have a role in animal studies. In this study, we aimed to identify variables that potentially influence the outcomes of experimental animal studies regarding TSCI pathophysiology through a modified two-round Delphi study.

Materials and methods

Study design

A two-round modified Delphi approach was used to explore the opinions of experts on a list of variables. Twenty-one experts were selected from departments/research centers of trauma, neurology, physiology, neuroscience, biomedical science, neurobiology, spinal cord and brain injury and neural injury from different countries, and they do not have any joint publication. Also, the selected scientists were the first or the corresponding author of at least three papers regarding TSCI. Forty-four percent of the participants had >10 years of experience in the field of animal study regarding SCI, and 50% had 4–10 years of experience. In the first round, in order to develop a questionnaire, a literature search was conducted in PubMed. As we aimed to explore experimental factors that can influence the pathophysiology of TSCI in animal experimentations, the proposed items about the therapeutic variables in the related papers were excluded from our initial questionnaire. Identification of therapeutic variables was conducted through several meetings. The aim of these meetings was to discuss the topics to be included in the first round of the Delphi study, after which a subset of members developed the final questionnaire. In the second round, the prepared questionnaire was sent to participants by E-mail, and, each participant was asked to score each item with a number from 0 to 10, where 10 is defined as ‘the most important item that a study could not exist without’ and 0 indicated ‘the least important item in experimental TSCI studies’. Furthermore, the opinion of each reviewer was received as a note/comment on each item.

Statistical data analysis

The median value of the scores assigned to each item was calculated using Microsoft Office Excel 2009. Consensus was achieved through the following criteria: (1) the median had to be at 7.5 or higher, and (2) at least 70% of participants needed to give a rating of 7 or higher. The inter-rater reliability analysis was performed to determine consistency among raters using the Kappa coefficient and Cronbach's alpha about 240 papers discussing pathophysiology of TSCI in animal studies. The details of these 240 papers are presented in Supplementary Table S1.

Results

Among the papers found from the literature survey, the published papers discussing the improvement of standards of reporting in animal studies,10, 11, 12, 13 a recently introduced RoB tool14 and CAMARADES checklist15 were selected to develop the initial questionnaire. By excluding the therapeutic variables from the proposed items in the aforementioned papers, a 47-item checklist was developed (Table 1). These items were divided into three categories of animals, housing and assessment variables and are described in the Discussion section. The result of the second round of the Delphi study, by considering the two mentioned criteria, was 15 items, which are highlighted in Table 1. Of these 15 items, 4 were from animal category, 10 were from assessment and 1 item was from housing variables.

Table 1 The list of variables that was included in the questionnaire sent to participants as well as the result of the Delphi study
Full size table

The inter-rater reliability for the raters was found to be Kappa=0.82 (P<0.001), which showed high internal consistency. Also, a Cronbach's alpha of 0.9 for all the questions indicated high internal consistency. Furthermore, evaluation of 240 animal studies regarding the pathophysiology of TSCIs, based on the present checklist, revealed that only species was reported in all articles (Figure 1).

Figure 1
figure 1

The percentage of the presence of the 15 potential variables affecting the quality of animal studies regarding the pathophysiology of TSCI in 240 related articles.

Full size image

Discussion

In this study, a final 15-item checklist has been developed using the Delphi method to reach a consensus on variables, which can potentially influence the results of animal studies regarding the pathophysiology of TSCI.

Ranking the initial 47 items included in our questionnaire has been conducted based on the median value of scores assigned to each item and the percentage of the scores ⩾7. For statistical analysis, we calculated ‘median’ because it is easy to compute and comprehend, and it is not distorted by outliers/skewed data; in addition, it can be used to determined ratios, interval and ordinal scales, and it is a special average used in qualitative phenomena, which are not quantified but ranked.16

The importance of the included 47 items is supported by evidence.13, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37 In the following, an overview is given of the literature evidence, which proves the significance of these items.

Species

The kind of species has a crucial effect on molecular and cellular responses after TSCI, which can be related to their differences in physiology, inflammatory responses and behavior.17, 18

Strain

The strain could be critical, depending on the study area. For instance, it has been shown that there are qualitative differences not only in the cellular immune response to trauma but also in cardiovascular responses between Sprague–Dawley and Lewis rats.19, 20

Genetic background

It is important to ensure that genetically modified strains are appropriately backcrossed and/or that littermates are used to rule out any variability as a confounding factor in terms of outcome.

Experimental unit

Definition of experimental unit is important, especially when animals housed in one cage receive the medication through a common source (for example, food and/or water). In these conditions, any change in the diet can affect all animals housed in one cage.

Number of animals per group

Number of animals per group is the most important item to conduct a power analysis. Significance is determined by both the number of animals as well as the effect size.17, 21, 22

Gender

It has been shown that nerve regeneration can be influenced by sex hormones (for example, estrogen).23

Age/weight

Age-related differences in functional outcomes, regenerative, glial and immune responses have been reported previously.24 It has been shown that neonatal rodents exhibit significantly faster rate of recovery than adult ones.25 In addition, significantly improved functional recovery has been reported for adult rats (60-day old) compared with older ones (12-month old).26 It is worth noting that, in some studies, weight of animals at the beginning of study is used as a reflection of age.27, 28, 29

Housing

The housing conditions are specified in the ethical approval. The exact values for temperature, humidity, lighting, noise and amount of food and water should be within a certain range. Also, any food or water restrictions should be specified. Importance of bedding varies with the injuries, and in dorsal horn injuries and pain studies softness of bedding should be mentioned too.

In addition, cage enrichment should be specified because having wheels, toys, large cages with more than a few animals, opportunities for exercise and so on can make a difference in locomotor recovery. Furthermore, frequency of handling for bladder expression can affect pain, spasticity and behavior assessments.

Method of allocation to treatment

Describing the method of allocating the animals to study groups (control or injury) has been recommended.21, 22, 30 Apart from allocation, it is also important to report whether the treatment and handling the animals were the same across experimental groups. The best practice should include reporting the methods of randomization of animals to the various experimental groups. Random blocked samples or computer-based numbers are suggested instead of alternate randomization, which has high risk of bias and changes the study design from randomized to quasi-randomized.13

Blindness

Blinding should be considered in allocation concealment, conduct of the experiment and assessment of outcomes.13, 21, 22 Although blindness is one of the major issues associated with RoB, inadequate reporting of this aspect of study design remains widespread in experimental animal studies (Figure 1).

Methods, level and severity of injury

There are differences between myelin pathology and the amount of the extension of pathophysiological events following contusion and transection.31 Furthermore, the level of injury can affect pathophysiological events, especially when neurons are evaluated after injury, and the amount of apoptosis after axotomy depends on the distance of the injured site from the cell body.32 Moreover, in compression/contusion model of injury, the applied force and duration of compression are the most effective factors on the severity of injury.33, 34 Also, precise details of all procedures carried out should be provided for each experiment.35

Exclusion of animals from the experiment

Exclusion from the analysis, reasons and the number of exclusions from each treatment group should be stated, because this may affect interpretation of statistical outcomes.

Control groups

Description of the control groups in the experiment and an explanation of why these specific control groups are important to answer the research question should be provided. In addition, the selection and source of animals of control group needs to be reported, including whether they are true littermates of the test groups.13

Regulations and ethics

Ethical statement indicates the ethical review permissions, relevant licenses and national or institutional guidelines for the care and use of animals. Therefore, description of compliance with an independent organization within the institute (for example, Institutional Ethics Committee) improves the quality of animal studies.

Statistics

Statistics section provides information about replications, sample size calculation, the expected difference between groups, the expected variance, the planned analysis method and the desired statistical power. It is a brief part of a paper but has an effect on the reviewers’ judgment about the quality of reported evidence(s).

Using appropriate tests to answer the research question and hypothesis

The research question and hypothesis should be clearly defined when reporting the study. Moreover, the evaluation methods and tests used to explore the research question should be adequately convincing, which provide sufficient evidence for the main objective(s) of the study.

Method and drugs used for anesthesia/antibiotics

Antibiotics are mostly administrated for the first 10–14 days after the TSCI to avoid infection.35 The used antibiotics, as well as anesthetic agents, should be mentioned because some antibiotics (for example, β-lactam antibiotics) and anesthetic agents (for example, propofol, isoflurane) are neuroprotective, which can interfere with the desired intervention.36 However, the effect of these agents can be ignored as long as they have also been used in the control group.

Bladder expression

Bladder dysfunction is a consequence of TSCI, which varies depending on the level and the severity of the injury.37 Therefore, the protocol of the bladder expression should be stated within the context of the experiment.

Method of killing

Method of killing is especially important when histological/immunohistological results are going to be evaluated.

Conclusion

In clinical trials, there are well-established standards6 that facilitate clinicians to compare different clinical trials and gain a better insight into their research question. However, in animal studies, there is not any established standard yet, and just recently a few groups attempted to introduce the criteria to be considered when reporting an animal research. Kilkenny et al.11 introduced the ARRIVE, a general guideline to improve reporting animal studies. A similar checklist has been published by Hooijmans et al.10 This checklist introduces the essential information that should be provided in each part of a paper (that is, title, abstract, introduction, method, results and discussion) as a general guideline for reporting animal studies. Recently, Lemmon et al.12 proposed a specified reporting standard for spinal cord injury experiments specifying animal type, details of intervention, as well as the details of methodology. In our case, we developed a 15-item checklist through a modified two-round Delphi study, which can be used specifically by the scientists who are exploring the pathophysiological events after TSCI.

Data archiving

There were no data to deposit.