Abstract
Study design:
This is a Delphi study.
Objectives:
Defining variables that potentially influence the outcomes of an animal study regarding pathophysiology of traumatic spinal cord injury (TSCI).
Setting:
This study was conducted in Iran.
Methods:
A modified two-round Delphi study was conducted. As the first round, an initial questionnaire was developed on the basis of literature and a series of focus group discussions. In the second round, the participants were asked to score the items through a 10-point scale. Consensus was achieved through the following criteria: (1) the median of scores has to be at 7.5 or higher, and (2) at least 70% of participants need to rate 7 or higher. Also, the inter-rater reliability analysis was performed to determine consistency among raters using the Kappa coefficient and Cronbach's alpha.
Results:
Twenty-one experts participated in our study. From the first round of the study, a 47-item checklist was developed. By considering the aforementioned criteria for consensus building on extremely important factors, we reached a 15-item checklist including species, strain, method and level of injury, control group, genetic background, severity of injury, attrition, use of appropriate test, blindness, method of allocation to treatments, regulation and ethics, age/weight, bladder expression, number of animals/group and statistics. The inter-rater reliability for the raters was found to be Kappa=0.82 (P<0.001). A Cronbach's alpha of 0.9 for all the questions indicated high internal consistency.
Conclusion:
This study introduces a checklist of variables that potentially influence the outcomes of animal studies regarding TSCI pathophysiology and describe its validity and reliability.
Similar content being viewed by others
Introduction
Traumatic spinal cord injury (TSCI) is a devastating injury to the patient, family and society, as a TSCI results in multi-system complications, impaired health-related quality of life and poor prognosis for neurologic recovery.1, 2, 3 The incidence of TSCI ranges from 3.6 to 195.4 patients per million around the world4 and 25.5 per million per year in developing countries.5 A prerequisite to the development of effective therapies for TSCI is a detailed understanding of the pathophysiological events occurring after injury. The pathophysiology of TSCI can be well divided into two phases, a primary and a secondary injury. The primary injury involves the initial mechanical damage, which is followed by a delayed onset of a secondary phase comprising a cascade of molecular and cellular events that inhibits the regeneration and also extends the injured site. As a result of the many technical limitations associated with the study of TSCI pathophysiology in humans, almost all of our detailed understanding of TSCI pathophysiology is based on animal studies. Translation of the obtained outcomes to the clinic could be achieved through well-designed animal studies and consideration of the variables, which are decisive in determining the outcomes of animal studies.
A systematic review in the field of evidence-based medicine is the highest level of medical evidence, and accordingly the required standards have already been developed.6 Recently, systematic reviews in the animal experimentation field have attracted great attention,7, 8, 9 although there are not adequate guidelines for assessing the quality of experimental animal studies. In this regard, a few groups of researchers focused on the development of publication guidelines in order to improve the reporting transparency, which makes future systematic reviews more feasible and allows the researchers to replicate previously published papers.10, 11, 12, 13 Also, Hooijmans et al.14 developed SYRCLE’s risk of bias tool based on the Cochrane Collaboration Risk of Bias (RoB) Tool, by introducing aspects of bias that are probable to have a role in animal studies. In this study, we aimed to identify variables that potentially influence the outcomes of experimental animal studies regarding TSCI pathophysiology through a modified two-round Delphi study.
Materials and methods
Study design
A two-round modified Delphi approach was used to explore the opinions of experts on a list of variables. Twenty-one experts were selected from departments/research centers of trauma, neurology, physiology, neuroscience, biomedical science, neurobiology, spinal cord and brain injury and neural injury from different countries, and they do not have any joint publication. Also, the selected scientists were the first or the corresponding author of at least three papers regarding TSCI. Forty-four percent of the participants had >10 years of experience in the field of animal study regarding SCI, and 50% had 4–10 years of experience. In the first round, in order to develop a questionnaire, a literature search was conducted in PubMed. As we aimed to explore experimental factors that can influence the pathophysiology of TSCI in animal experimentations, the proposed items about the therapeutic variables in the related papers were excluded from our initial questionnaire. Identification of therapeutic variables was conducted through several meetings. The aim of these meetings was to discuss the topics to be included in the first round of the Delphi study, after which a subset of members developed the final questionnaire. In the second round, the prepared questionnaire was sent to participants by E-mail, and, each participant was asked to score each item with a number from 0 to 10, where 10 is defined as ‘the most important item that a study could not exist without’ and 0 indicated ‘the least important item in experimental TSCI studies’. Furthermore, the opinion of each reviewer was received as a note/comment on each item.
Statistical data analysis
The median value of the scores assigned to each item was calculated using Microsoft Office Excel 2009. Consensus was achieved through the following criteria: (1) the median had to be at 7.5 or higher, and (2) at least 70% of participants needed to give a rating of 7 or higher. The inter-rater reliability analysis was performed to determine consistency among raters using the Kappa coefficient and Cronbach's alpha about 240 papers discussing pathophysiology of TSCI in animal studies. The details of these 240 papers are presented in Supplementary Table S1.
Results
Among the papers found from the literature survey, the published papers discussing the improvement of standards of reporting in animal studies,10, 11, 12, 13 a recently introduced RoB tool14 and CAMARADES checklist15 were selected to develop the initial questionnaire. By excluding the therapeutic variables from the proposed items in the aforementioned papers, a 47-item checklist was developed (Table 1). These items were divided into three categories of animals, housing and assessment variables and are described in the Discussion section. The result of the second round of the Delphi study, by considering the two mentioned criteria, was 15 items, which are highlighted in Table 1. Of these 15 items, 4 were from animal category, 10 were from assessment and 1 item was from housing variables.
The inter-rater reliability for the raters was found to be Kappa=0.82 (P<0.001), which showed high internal consistency. Also, a Cronbach's alpha of 0.9 for all the questions indicated high internal consistency. Furthermore, evaluation of 240 animal studies regarding the pathophysiology of TSCIs, based on the present checklist, revealed that only species was reported in all articles (Figure 1).
Discussion
In this study, a final 15-item checklist has been developed using the Delphi method to reach a consensus on variables, which can potentially influence the results of animal studies regarding the pathophysiology of TSCI.
Ranking the initial 47 items included in our questionnaire has been conducted based on the median value of scores assigned to each item and the percentage of the scores ⩾7. For statistical analysis, we calculated ‘median’ because it is easy to compute and comprehend, and it is not distorted by outliers/skewed data; in addition, it can be used to determined ratios, interval and ordinal scales, and it is a special average used in qualitative phenomena, which are not quantified but ranked.16
The importance of the included 47 items is supported by evidence.13, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37 In the following, an overview is given of the literature evidence, which proves the significance of these items.
Species
The kind of species has a crucial effect on molecular and cellular responses after TSCI, which can be related to their differences in physiology, inflammatory responses and behavior.17, 18
Strain
The strain could be critical, depending on the study area. For instance, it has been shown that there are qualitative differences not only in the cellular immune response to trauma but also in cardiovascular responses between Sprague–Dawley and Lewis rats.19, 20
Genetic background
It is important to ensure that genetically modified strains are appropriately backcrossed and/or that littermates are used to rule out any variability as a confounding factor in terms of outcome.
Experimental unit
Definition of experimental unit is important, especially when animals housed in one cage receive the medication through a common source (for example, food and/or water). In these conditions, any change in the diet can affect all animals housed in one cage.
Number of animals per group
Number of animals per group is the most important item to conduct a power analysis. Significance is determined by both the number of animals as well as the effect size.17, 21, 22
Gender
It has been shown that nerve regeneration can be influenced by sex hormones (for example, estrogen).23
Age/weight
Age-related differences in functional outcomes, regenerative, glial and immune responses have been reported previously.24 It has been shown that neonatal rodents exhibit significantly faster rate of recovery than adult ones.25 In addition, significantly improved functional recovery has been reported for adult rats (60-day old) compared with older ones (12-month old).26 It is worth noting that, in some studies, weight of animals at the beginning of study is used as a reflection of age.27, 28, 29
Housing
The housing conditions are specified in the ethical approval. The exact values for temperature, humidity, lighting, noise and amount of food and water should be within a certain range. Also, any food or water restrictions should be specified. Importance of bedding varies with the injuries, and in dorsal horn injuries and pain studies softness of bedding should be mentioned too.
In addition, cage enrichment should be specified because having wheels, toys, large cages with more than a few animals, opportunities for exercise and so on can make a difference in locomotor recovery. Furthermore, frequency of handling for bladder expression can affect pain, spasticity and behavior assessments.
Method of allocation to treatment
Describing the method of allocating the animals to study groups (control or injury) has been recommended.21, 22, 30 Apart from allocation, it is also important to report whether the treatment and handling the animals were the same across experimental groups. The best practice should include reporting the methods of randomization of animals to the various experimental groups. Random blocked samples or computer-based numbers are suggested instead of alternate randomization, which has high risk of bias and changes the study design from randomized to quasi-randomized.13
Blindness
Blinding should be considered in allocation concealment, conduct of the experiment and assessment of outcomes.13, 21, 22 Although blindness is one of the major issues associated with RoB, inadequate reporting of this aspect of study design remains widespread in experimental animal studies (Figure 1).
Methods, level and severity of injury
There are differences between myelin pathology and the amount of the extension of pathophysiological events following contusion and transection.31 Furthermore, the level of injury can affect pathophysiological events, especially when neurons are evaluated after injury, and the amount of apoptosis after axotomy depends on the distance of the injured site from the cell body.32 Moreover, in compression/contusion model of injury, the applied force and duration of compression are the most effective factors on the severity of injury.33, 34 Also, precise details of all procedures carried out should be provided for each experiment.35
Exclusion of animals from the experiment
Exclusion from the analysis, reasons and the number of exclusions from each treatment group should be stated, because this may affect interpretation of statistical outcomes.
Control groups
Description of the control groups in the experiment and an explanation of why these specific control groups are important to answer the research question should be provided. In addition, the selection and source of animals of control group needs to be reported, including whether they are true littermates of the test groups.13
Regulations and ethics
Ethical statement indicates the ethical review permissions, relevant licenses and national or institutional guidelines for the care and use of animals. Therefore, description of compliance with an independent organization within the institute (for example, Institutional Ethics Committee) improves the quality of animal studies.
Statistics
Statistics section provides information about replications, sample size calculation, the expected difference between groups, the expected variance, the planned analysis method and the desired statistical power. It is a brief part of a paper but has an effect on the reviewers’ judgment about the quality of reported evidence(s).
Using appropriate tests to answer the research question and hypothesis
The research question and hypothesis should be clearly defined when reporting the study. Moreover, the evaluation methods and tests used to explore the research question should be adequately convincing, which provide sufficient evidence for the main objective(s) of the study.
Method and drugs used for anesthesia/antibiotics
Antibiotics are mostly administrated for the first 10–14 days after the TSCI to avoid infection.35 The used antibiotics, as well as anesthetic agents, should be mentioned because some antibiotics (for example, β-lactam antibiotics) and anesthetic agents (for example, propofol, isoflurane) are neuroprotective, which can interfere with the desired intervention.36 However, the effect of these agents can be ignored as long as they have also been used in the control group.
Bladder expression
Bladder dysfunction is a consequence of TSCI, which varies depending on the level and the severity of the injury.37 Therefore, the protocol of the bladder expression should be stated within the context of the experiment.
Method of killing
Method of killing is especially important when histological/immunohistological results are going to be evaluated.
Conclusion
In clinical trials, there are well-established standards6 that facilitate clinicians to compare different clinical trials and gain a better insight into their research question. However, in animal studies, there is not any established standard yet, and just recently a few groups attempted to introduce the criteria to be considered when reporting an animal research. Kilkenny et al.11 introduced the ARRIVE, a general guideline to improve reporting animal studies. A similar checklist has been published by Hooijmans et al.10 This checklist introduces the essential information that should be provided in each part of a paper (that is, title, abstract, introduction, method, results and discussion) as a general guideline for reporting animal studies. Recently, Lemmon et al.12 proposed a specified reporting standard for spinal cord injury experiments specifying animal type, details of intervention, as well as the details of methodology. In our case, we developed a 15-item checklist through a modified two-round Delphi study, which can be used specifically by the scientists who are exploring the pathophysiological events after TSCI.
Data archiving
There were no data to deposit.
References
Saadat S, Javadi M, Divshali BS, Tavakoli AH, Ghodsi SM, Montazeri A et al. Health-related quality of life among individuals with long-standing spinal cord injury: a comparative study of veterans and non-veterans. BMC Public Health 2010; 10: 1–7.
Eslami V, Saadat S, Habibi Arejan R, Vaccaro AR, Ghodsi SM, Rahimi-Movaghar V . Factors associated with the development of pressure ulcers after spinal cord injury. Spinal Cord 2012; 50: 899–903.
Rahimi-Movaghar V, Moradi-Lakeh M, Rasouli MR, Vaccaro AR . Burden of spinal cord injury in Tehran, Iran. Spinal Cord 2010; 48: 492–497.
Jazayeri S, Beygi S, Shokraneh F, Hagen E, Rahimi-Movaghar V . Incidence of traumatic spinal cord injury worldwide: a systematic review. Eur Spine J 2015; 24: 905–918.
Rahimi-Movaghar V, Sayyah MK, Akbari H, Khorramirouz R, Rasouli MR, Moradi-Lakeh M et al. Epidemiology of traumatic spinal cord injury in developing countries: a systematic review. Neuroepidemiology 2013; 41: 65–85.
Furlan AD, Pennick V, Bombardier C, van Tulder M, Group ftEBotCBR. 2009 Updated method guidelines for systematic reviews in the Cochrane Back Review Group. Spine 2009; 34: 1929–1941.
Haberzettl R, Bert B, Fink H, Fox MA . Animal models of the serotonin syndrome: a systematic review. Behav Brain Res 2013; 256: 328–345.
Krishna V, Konakondla S, Nicholas J, Varma A, Kindy M, Wen X . Biomaterial-based interventions for neuronal regeneration and functional recovery in rodent model of spinal cord injury: a systematic review. J Spinal Cord Med 2013; 36: 174–190.
Akhtar AZ, Pippin JJ, Sandusky CB . Animal studies in spinal cord injury: a systematic review of methylprednisolone. Altern Lab Anim 2009; 37: 43–62.
Hooijmans CR, Leenaars M, Ritskes-Hoitinga M . A gold standard publication checklist to improve the quality of animal studies, to fully integrate the Three Rs, and to make systematic reviews more feasible. Altern Lab Anim 2010; 38: 167–182.
Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG . Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol 2010; 8: e1000412.
Lemmon VP, Ferguson AR, Popovich PG, Xu XM, Snow DM, Igarashi M et al. Minimum information about a spinal cord injury experiment: a proposed reporting standard for spinal cord injury experiments. J Neurotrauma 2014; 31: 1354–1361.
Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW et al. A call for transparent reporting to optimize the predictive value of preclinical research. Nature 2012; 490: 187–191.
Hooijmans CR, Rovers MM, Vries RBd, Leenaars M, Ritskes-Hoitinga M, Langendam MW . SYRCLE’s risk of bias tool for animal studies. BMC Med Res Methodol 2014; 14: 1–9.
Batchelor PE, Skeers P, Antonic A, Wills TE, Howells DW, Macleod MR, Sena ES . Systematic review and meta-analysis of therapeutic hypothermia in animal models of spinal cord injury. PLoS ONE 2013; 8 p e71317.
Manikandan S . Measures of central tendency: median and mode. J Pharmacol Pharmacother 2011; 2: 214–215.
Garland T, Adolph SC . Why not to do two-species comparative studies: limitations on inferring adaptation. Physiol Zool 1994; 67: 797–828.
Macleod MR, Fisher M, O'Collins V, Sena ES, Dirnagl U, Bath PM et al. Good laboratory practice: preventing introduction of bias at the bench. Stroke 2009; 40: e50–e52.
Popovich PG, Wei P, Stokes BT . Cellular inflammatory response after spinal cord injury in Sprague-Dawley and Lewis rats. J Comp Neurol 1997; 377: 443–464.
Landrum LM, Thompson GM, Blair RW . Does postsynaptic alpha 1-adrenergic receptor supersensitivity contribute to autonomic dysreflexia? Am J Physiol 1998; 274: 1090–1098.
Macleod MR, O'Collins T, Horky LL, Howells DW, Donnan GA . Systematic review and metaanalysis of the efficacy of FK506 in experimental stroke. J Cereb Blood Flow Metab 2005; 25: 713–721.
Macleod MR, O'Collins T, Howells DW, Donnan GA . Pooling of animal experimental data reveals influence of study design and publication bias. Stroke 2004; 35: 1203–1208.
Smith PG, George M, Bradshaw S . Estrogen promotes sympathetic nerve regeneration in rat proximal urethra. Urology 2009; 73: 1392–1396.
Kumamaru H, Saiwai H, Ohkawa Y, Yamada H, Iwamoto Y, Okada S . Age-related differences in cellular and molecular profiles of inflammatory responses after spinal cord injury. J Cell Physiol 2012; 227: 1335–1346.
Brown KM, Wolfe BB, Wrathall JR . Rapid functional recovery after spinal cord injury in young rats. J Neurotrauma 2005; 22: 559–574.
Gwak YS, Hains BC, Johnson KM, Hulsebosch CE . Locomotor recovery and mechanical hyperalgesia following spinal cord injury depend on age at time of injury in rat. Neurosci Lett 2004; 362: 232–235.
James ND, Bartus K, Grist J, Bennett DL, McMahon SB, Bradbury EJ . Conduction failure following spinal cord injury: functional and anatomical changes from acute to chronic stages. J Neurosci 2011; 31: 18543–18555.
Arvanian VL, Schnell L, Lou L, Golshani R, Hunanyan A, Ghosh A et al. Chronic spinal hemisection in rats induces a progressive decline in transmission in uninjured fibers to motoneurons. Exp Neurol 2009; 216: 471–480.
Navarro R, Juhas S, Keshavarzi S, Juhasova J, Motlik J, Johe K et al. Chronic spinal compression model in minipigs: a systematic behavioral, qualitative, and quantitative neuropathological study. J Neurotrauma 2012; 29: 499–513.
Horn J, de Haan RJ, Vermeulen M, Luiten PG, Limburg M . Nimodipine in animal model experiments of focal cerebral ischemia: a systematic review. Stroke 2001; 32: 2433–2438.
Siegenthaler MM, Tu MK, Keirstead HS . The extent of myelin pathology differs following contusion and transection spinal cord injury. J Neurotrauma 2007; 24: 1631–1646.
Jin Y, Tay D, So KF . The effect of distance of axotomy in the spinal cord on NOS expression and neuronal survival in the lateral vestibular nucleus of rats. Soc Neurosci Abstr 1997; 23: 899.
Poon PC, Gupta D, Shoichet MS, Tator CH . Clip compression model is useful for thoracic spinal cord injuries: histologic and functional correlates. Spine 2007; 32: 2853–2859.
Jazayeri SB, Firouzi M, Zadegan SA, Saeedi N, Pirouz E, Nategh M et al. The effect of timing of decompression on neurologic recovery and histopathologic findings after spinal cord compression in a rat model. Acta Med Iran 2013; 51: 431–437.
Rahimi-Movaghar V, Yazdi A, Karimi M, Mohammadi M, Firouzi M, Zanjani LO et al. Effect of decompression on complete spinal cord injury in rats. Int J Neurosci 2008; 118: 1359–1373.
Jain KK . The Handbook of Neuroprotection. Springer, New York. 2011, p 547.
David BT, Steward O . Deficits in bladder function following spinal cord injury vary depending on the level of the injury. Exp Neurol 2010; 226: 128–135.
Acknowledgements
We thanks Jan M Schwab, Artur SP Varejão, Hamid Mobasheri, Hideyuki Okano, Keith K Fenrich, Hideki Moriyama, Nina Weishaupt, Alexander G Rabchevsky, Marc J Ruitenberg, Phil Waite, Junfang Wu and Michael A Lane for their contributions in our survey. This work was funded by a grant (93-03-38-26795) from Sina Trauma and Surgery Research Center, Tehran University of Medical Sciences.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no conflict of interest.
Additional information
Supplementary Information accompanies this paper on the Spinal Cord website
Supplementary information
Rights and permissions
About this article
Cite this article
Hassannejad, Z., Sharif-Alhoseini, M., Shakouri-Motlagh, A. et al. Potential variables affecting the quality of animal studies regarding pathophysiology of traumatic spinal cord injuries. Spinal Cord 54, 579–583 (2016). https://doi.org/10.1038/sc.2015.215
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/sc.2015.215
This article is cited by
-
Reprogramming of astrocytes to neuronal-like cells in spinal cord injury: a systematic review
Spinal Cord (2024)
-
Sustained delivery of chABC improves functional recovery after a spine injury
BMC Neuroscience (2022)
-
Alginate scaffolds improve functional recovery after spinal cord injury
European Journal of Trauma and Emergency Surgery (2022)
-
The effect of low-level laser therapy on pathophysiology and locomotor recovery after traumatic spinal cord injuries: a systematic review and meta-analysis
Lasers in Medical Science (2022)
-
Combined application of neural stem/progenitor cells and scaffolds on locomotion recovery following spinal cord injury in rodents: a systematic review and meta-analysis
Neurosurgical Review (2022)