Procedures for assessing psychological predictors of injuries in circus artists: a pilot prospective study

When time-varying exposures and covariates are considered important, one could use two general approaches to minimize bias. First, one could ask patients to recall how they felt before the injury. The control group could consist of non-injured participants at the time of injury (case–control study, where control participants are sampled at the time a case is diagnosed, and therefore can later be considered cases), or the same participants recalling their “states” a few days earlier (i.e., case-crossover study) [35]. However, these study designs are subject to potential recall biases.

Alternatively in a prospective design one measures psychological variables regularly (e.g. daily or weekly), where the analytical approach could be a longitudinal repeated-measures design (e.g. time-series) or a nested case-crossover analysis. In this approach sample size requirements are dependent on both the frequency of injury events and the correlation of psychological states over time (higher sample sizes if states are relatively stable because there is less information on the changing exposures). However, this option involves logistical challenges associated with frequent collection of psychological data. Some psychological questionnaires may evoke scepticism among athletes, staff and coaches [39], and there may be variations due to translation [40], need to minimize time commitment [40, 41], sport context, and participants’ understanding for the need to avoid response distortion (either faking good or bad) [42]. Completing daily questionnaires may result in fatigue, with answers provided in an unvarying or random manner [43], or drop-out. There are also challenges in managing daily data collection, and transmitting it to a central data management unit in any sport setting. Therefore, it is paramount to establish procedures and practices that facilitate these processes.

As a consequence, the primary objective of this pilot study was to test the feasibility of a prospective study specifically designed to test if psychological states are predictive of injury. More specifically, our objectives were to determine 1) time required to complete psychological questionnaires distributed at baseline (assessing traits), daily (assessing states), and once per week (assessing states including an additional questionnaire to assess mood); 2) optimal distributing and collecting of questionnaires to be completed daily or once per week; 3) questionnaire comprehension and acceptability and; 4) intra-individual correlation for daily psychological states necessary for sample size calculations in a larger longitudinal repeated-measures investigation.

The study population were artists from the show “O” produced by Cirque du Soleil (CDS). These artists come from a variety of athletic backgrounds including swimming, diving, gymnastics and acrosport, as well as performing arts including dancers and musicians. We chose this population because injury records and exposures are stored electronically, and conducting the study appeared feasible. Overall injury rates have been estimated to be 9.7 injuries per 1000 artist-exposures (similar to Men’s NCAA Basketball [44]), with 4.4% of injuries resulting in missing 15 or more performances (approximately equivalent to 10 days) [45]. We therefore expect these results to be useful in other athletic contexts involving regular psychological assessments for an extended period.

The McGill University Institutional Review Board approved this pilot study. The questionnaires could only cover a limited number of constructs because participants were required to complete them daily. To determine which constructs we should investigate in our context (see Table 1), we used an iterative process. We first individually interviewed CDS personnel that were familiar with circus-related injuries: Director of Performance Medicine responsible for shows that change location, Director of Performance Medicine responsible for shows that don’t change location, the head coach, a choreographer, a supervising physiotherapist for one show, and the Senior Performance Psychologist. Following these meetings, we outlined areas of interest, their potential mechanisms of action, and identified corresponding psychological constructs. These variables and constructs were later verified with the same CDS personnel.

Table 1 illustrates the constructs identified as most likely to be important through the interviews with CDS personnel and a review of the literature, questionnaires chosen from the literature where they existed, and timing/frequency for questionnaire distribution. The constructs included trait and state anxiety, trait ability to cope, trait and state mood, and state confidence, where traits were measured at baseline and states were measured daily. We also included questions related to several state behavioural habits believed to influence injury risk. Our objective was to have artists complete the baseline questionnaires within 15 min, weekly questionnaires within 2 min and daily questionnaires within 1 min. Details of the specific questionnaires are described below.

Approximately one month prior to the study, we approached all CDS artists ≥ 18 years of age working at the show “O” located in Las Vegas, Nevada. During a regular weekly meeting, we gave a brief 10-min presentation explaining the general principles of the study, and distributed consent forms. We emphasized that we could prepare reports for each individual showing how their daily profiles changed over time. In addition, because artists often rely on each other (e.g. catcher and flyer in a trapeze act), we emphasized that the information may be important to the team, even if the individual did not feel it was personally important (which would be analogous in some team sport contexts). Artists were asked to submit signed consent forms within one week if they were interested in participating.

We provided the questionnaires in the four principal languages of the artists: English, French, Japanese and Russian. We used existing translations where available, and used the professional CDS translation service when necessary. At least one artist with conversational English language skills and whose first language was the translated language reviewed all translated questionnaires. Although we recognize that cultural differences exist between populations using these languages that might increase the heterogeneity of responses between participants, this should not affect our ability to reach our primary objectives of determining feasibility, and variance estimates for sample size calculations of a definitive study.

After discussions with the stage manager and artistic directors, baseline questionnaires including State-Trait Anxiety Inventory (STAI) [50], Ways of Coping [40, 47] and Profile of Mood States (POMS) [48] were distributed at the time the artist provided informed consent, and were completed at home over the ensuing 2–4 weeks. On each working day, we placed a short “states” questionnaire (see Additional file 1) that included either 7 or 9 items (i.e. 2 additional questions included on the “weekly” questionnaire) at each artist’s make-up station (each had their own station). The items included in these questionnaires came from validated questionnaires where possible and included the constructs conflicts, anxiety, self-confidence, sleep and physical capacity (Table 1). Artists completed the questionnaire prior to the day’s performances, and dropped them into a locked drop-box or handed them to the supervising therapist. To minimize missing data, distributed questionnaires included the date and the artist’s identification number. Completed forms were scanned and emailed to the primary investigator for data entry.

Artists were encouraged to provide feedback throughout the study, with special reference to the time required to complete questionnaires, undesirable questions, logistical problems, workload, and possible duration for a definitive study.

To track injury and exposure data, we used the same methods as our previous studies [45]. In brief, CDS therapists use electronic charting for all injuries and treatments (provided free of charge to artists) through in-house injury tracking software. We classified injuries according to commonly used criteria in the sport injury literature [56‐58] and we have used in the past with the same company [45] and are common in sport medicine epidemiology: Medical Attention (injury reported to a show therapist) and Time-Loss-1 (TL-1) if the injury resulted in at least one missed performance. Anytime an artist misses a performance, it must be documented as either a work-related injury (included in our analyses and for which the artist is covered by health insurance), or a personal health condition (excluded from our analyses). Exposure data were obtained from in-house software that keeps track of when artists are performing.

As a pilot study designed primarily to assess feasibility, we provide descriptive statistics only. For continuous variables, we report the median and interquartile region (IQR) because the data were skewed, and we report percent for categorical variables. In addition, we report the direct feedback elicited during weekly meetings. Our sample size was determined by the number of artists within the show and was not modifiable. For practical considerations due to vacation time, we used an 8-week period that began one week after a vacation and ended within two weeks of the next vacation.

To calculate sample sizes for the definitive study, we used linear mixed models and generalized estimating equations (GEE) to account for the intra-participant correlations for the psychological variables over days (stability of states across time), and took a precision-based approach for the estimate via simulations based on psychological daily data that corresponded to our observed data. The observed injury rates (shown in results) were much less than previous studies in the general CDS population. Given the short time frame of the pilot study, we felt the results from our previous study would provide a more reliable estimate. We estimated a baseline TL-1 injury rate of 1.6 injuries per 1000 artist-exposures for the company [45], and a minimal important difference (MID) of 25% (i.e. 1.2 injuries per 1000 artist-exposures). We calculated sample sizes based on at least a 2-category change in an ordinal scale (all daily questions used essentially ordinal data). We varied the number of artists (between 180–1000) and the number of observation days (40–250) to explore different assumptions (always setting alpha to 0.05, 2-sided). All simulations and calculations were conducted in the open-software R [59] with the lme4 [60] and geepack [61] libraries. In addition, we estimated sample size for a case-crossover study using a matched case–control design using free open source statistical software [62] and the following parameter values: 2-sided alpha = 0.05, power = 0.8, 10:1 ratio of controls to cases, risk of exposure (categorized dichotomously) in controls at 0.3, and an MID of 25% and 50% increased risk of exposure in cases.

To facilitate the process, identification numbers on questionnaires were numbered in the same order as the participating artists’ make-up tables, but were not sequential (i.e. two proximal artists might be 63 and 66) in order to enhance confidentiality. Distributing the questionnaires required ~10 min and was usually done ~2 hours before the night’s first performance. In general, feedback from the artists suggested the ability to simply drop completed questionnaires in a locked box located near the door was well received.

The baseline questionnaire was completed in the targeted time (<15 min) by more than 60% of artists but required more than 30 min by two artists. We expected the weekly questionnaire to be completed in <2 min and the daily questionnaire to be completed in <1 min, but ~20% of artists required more time. That said, 45-50% of the artists required less than half the expected time. When we cross-correlated the results from the different questionnaires, the two participants requiring >30 min for the baseline questionnaire were not the same participants requiring longer periods for the weekly or daily questionnaires. Of the seven participants requiring >2 min for the weekly questionnaire, the time required for the daily questionnaire was <30 s for two artists, between 31–120 s for three artists, and >2 min for two artists.

There were no questions that artists felt should not be asked. No artist felt the workload onerous, but a couple of artists felt it was bothersome to answer the same questions daily. Of the 26 participants who answered how much longer they would be willing to continue completing the questionnaires if the study was repeated, five responded as long as needed, five responded ~2-3 months, two responded one month, two simply said they would continue for now, three were unsure and nine said they would no longer continue.

Based on artist feedback, we made important changes to the text and format of the daily questionnaire within the first couple of weeks (see Additional file 1). The original versions of questions 5 and 6 were re-worded to clearly indicate they referred to the upcoming performance not yet done. We also added “current role” to question 6 to clearly indicate we were interested in the optimal level of anxiety relevant to their current role at CDS.

Despite our encouragement for feedback, the first question of the daily questionnaire (overall well-being, “How do you feel today?” with 7-point Likert response from Very very good to Very very bad) was left unanswered on 68/1297 collected questionnaires; several artists provided feedback saying it was difficult to give an “overall” assessment. The question regarding sleeping was left unanswered 42/1297 times, likely due to questionnaire formatting because we did not receive any comments. We believe adding a dotted line from the question to the answer choices might draw the artists’ eyes to the question-answer group.

Over 2660 artist-performances, there were a total of nine Medical Attention injuries (3.4 injuries per 1000 artist-performances) and three TL-1 injuries 1.1 TL-1 injuries per 1000 artist-performances.

Intra-participant correlations were higher than expected, and three artists had no variability within the study for some psychological factors. With the low TL-1 injury rate and low variability, sample size calculations for both the linear mixed and GEE models were very high, with the linear mixed model predictably showing slightly less power than the GEE results. For example, using 180 artists measured over 100 days would provide only 13% power under a GEE model. Increasing the number of artists to 1000 only increases power to 30%. Increasing the number of days of observation increases power as well, but following 750 artists for 250 days yields only 30% power, and following 1000 artists for 250 days yields only 61% power.

Based on these results, we consider it infeasible to reliably follow participants daily for so long. Therefore, we also calculated the sample size for a case-crossover study where we would measure artists only once or twice per week, and then again post-injury. Based on the parameters discussed in the methods, and the estimated intra-participant correlations ranging from 0.16 to 0.58 for the different variables measured (Table 3), we would need between 828 and 1301 cases (injuries) if the MID is 25% increased risk. The number of artist-performances needed to occur is equal to the required number of cases divided by the injury rate per 1000 artist exposures. With an expected TL-1 injury rate of ~1.5 per 1000 performances one would need to follow between 550,000 (for “ill past 24 hours”) to 870,000 artist-performances (for “confidence”). With recruitment similar to our pilot study of 35 artists per show on six shows (210 artists) and assuming an average of 470 performances per year, we would need to follow the artists between 67–105 months. If the MID is 50% increased risk, the number of cases and artist-exposures is greatly decreased (to ~30% of the above values), but we would still need to follow artists for 20–32 months.

If instead one were interested in medical attention injuries, the estimated injury rate from previous studies is ~10 per 1000 performances. In this context, we would need to follow 210 artists for ~10-16 months (82,800 to 130,1000 artist performances) for an increased risk of 25%, but only ~ 3–5 months (24,200 to 38,900 artist performances) for an increased risk of 50%. Increasing the number of artists through improved recruitment per show, or by including additional shows would further decrease the amount of time required to complete the study.

To maximize acceptability and participation by the artists and their support staff, we used a participatory research approach [63]. We sought input from the stakeholder’s key personnel in both the identification of the theoretical model and selection of psychological measures. We used an iterative process to ensure we captured all the salient features, and to prioritize the traits and states of most interest. To minimize dropouts, we targeted specific questionnaire completion times based on our previous experience in other similar studies, and on the opinions of the stakeholder. During the study, we altered the formatting and wording of some questions based on participants’ feedback.

In general, the majority of artists completed the questionnaires within the target time and we obtained data on state anxiety, mood, self-confidence, and physical factors that will be presented in other reports. From informal conversations, some artists were more sensitive to nuances within their own psychological states and required more time to distinguish among the various choices in the questionnaires. For interval validity, the artist should be consistent in her/his approach throughout the course of the study. We would expect the acceptable time to complete questionnaires in other activities or sports to be generally similar, but there may be important differences based on the participating group’s culture, and research requirements. Therefore, we encourage investigators using protocols that require repeated psychological assessments to engage stakeholders to build support and determine acceptable questionnaire lengths, and other important insights. We chose to prioritize our data on the assessment of labile states, moods and feelings rather than on distal events (e.g. conflict with management, which is a cause of mood). We did not include self-efficacy [64, 65] because the analyses are expected to be confounded by ability, and therefore any interpretation would be questionable even if it also affects injury risk itself. Nor could we explore an artist’s body awareness (common in dancers), and nutrition habits. Although we have data on sleep patterns, CDS artists work at night and some regularly (or irregularly) take naps; these interim periods of sleep could theoretically reduce the effect of a poor night’s sleep.

Sample size requirements for a definitive prospective study on time loss injuries is prohibitively large, but using an MID of 50% increased risk, one could use the case-crossover design with prospectively collected control data to investigate the role of psychological states by following 210 artists for ~2-2.5 years for TL-1 injuries, and for ~3-5 months for medical attention injuries. These large sample size requirements (both in number of participants, and duration of study which could result in subject fatigue) represent an important challenge for any study investigating the role of psychological states and injury. Sample size requirements would be reduced if the injury rate is higher than 10 injuries per 1000 exposures, the MID is higher than 50%, or the correlation of states over time is lower than in our study.

Our very large sample size calculations appear to conflict with previous prospective studies showing statistically significant results for psychological factors as risk factors. However, as previously mentioned, there are important differences between the studies. First, the TL-1 injury rate in previous data from our artists was 1.5 injuries per 1000 artist-performances. The injury rates estimated from the various published studies (assuming 6 exposures per week per athlete) were approximately 5.1 injuries per 1000 athlete-exposures (games or practices) [37], 6.9 injuries per 1000 athlete-exposures [36], 11.1 injuries per 1000 athlete-exposures [38], and 4.9 injuries per 1000 athlete-exposures [33]. The much higher injury rate in elite soccer compared to circus artists in our data suggests sample size calculations would be closer to our calculated Medical Attention injury definition. Second, our analyses address the question whether certain psychological states are risk factors for injury among participants who get injured (it is not possible to measure the effect in those that never get injured). Other published studies on psychological states [33, 36‐38] addressed whether psychological factors distinguish between athletes who get injured and athletes who do not get injured. The distinction is important because if the psychological state is important in those that get injured, interventions for the day of competition or training are promising for injury prevention. However, if the psychological state is not important in those that get injured and is just a marker for a psychological trait, then longer-term interventions must be designed to change the more stable psychological traits (if these are indeed causal factors rather than markers for true causal risk factors). Third, the previous studies compared injured to non-injured subjects, which is a case–control analysis. Because they did not use incidence-density sampling, the results may overestimate the effect if adverse psychological states decrease during the study. Finally, previous studies have focused on the Daily Hassles questionnaire (measured weekly) whereas we focused on daily states related to confidence, anxiety and mood. Ivarsson [38] estimated the ICC for the Daily Hassles questionnaire to be 73% in elite junior soccer players, which is higher than the ICC for the variables we report in Table 3. This suggests the required sample size would be higher rather than lower for questions addressing whether the psychological state affects injury risk in those that get injured.

Although retrospective case-crossover design is possible, the risk of recall bias is high. Another alternative is to use a hybrid approach where one prospectively collects “control” exposure data on psychological states only once or twice per week, and contrast this with psychological state data collected for the “case-event” after the injury occurred. In this study design, some of the cases would happen to occur on days where pre-performance exposure data had already been collected. Thus, one could estimate the magnitude of “case” recall bias by comparing exposures collected retrospectively after the injury with exposure data collected prospectively before the performance. There is research demonstrating athletes can provide accurate retrospective reports on past performances, at least for anxiety and pre-performance moods [40].

Finally, recent technological advances allow data to be entered directly via smartphones and tablets. Although these methods may appear promising, surveys on electronic devices are sometimes read/interpreted differently from paper-based systems, and response rates may or may not be improved [66‐68]. Investigators should evaluate the advantages and disadvantages of all methods within the context of their study.