A team-based competition for undergraduate medical students to learn radiology within the virtual world Second Life

The Medical Master Island reproduces a university campus, with a central esplanade surrounded by trees and walkways, educational buildings, open-air auditoriums (Fig. 1), and underwater settings (caves, palaces, submerged cellars, etc.) used to give a playful aspect to some learning activities.

In February 2019, the game was introduced as a voluntary online activity for students enrolled for the first time in the four-month Radiology course. Students were informed that their participation would have no effect on their grades. They were invited to 2-h training sessions on Second Life, held on March. Several PDF files were provided with instructions for using Second Life and the rules of the game. Those students who wanted to participate in the game had to propose together with a partner, and the organisation randomly unified the pairs into teams of four. Participants were asked to submit their team name and colour, logo image, and choose a captain.

This study carried out within the framework of the Educational Innovation Project of the University of Malaga # PIE17-113 received the corresponding approval from the Vice-Rector's Office for Teaching and Research Staff. The students gave their explicit consent when they voluntarily agreed to participate. No additional ethical permission was needed.

The League of Rays team-based game was held from April 1 to May 19, 2019, organised in 6 stages of 7 days, from Monday to Sunday. Between the second and third stages, the game was interrupted for the Easter holidays. The first three stages were dedicated to radiological anatomy and the next three to radiological semiology. Thorax, abdomen, and musculoskeletal were treated successively in each 3-week block.

During the first 5 days of each stage, educational contents, elaborated from the syllabus and seminars of the course, and online resources with owned copyrights were presented in a set of 3 panels with 50 self-explanatory slides each. Each week, 6 sets of panels were arranged in the central esplanade of the island, available continuously for participants (Fig. 2). On the afternoon of the fifth day, the educational panels were replaced by single panels with 15 multi-choice questions. A database of 180 slides with multi-choice questions, 30 questions for each thematic block, was used for this purpose. Twelve test variants were elaborated, homogeneously distributing the 30 corresponding questions, so that each of them was in 6 test variants. Each week, a test variant was assigned to each participant, who had to answer it by sending a notecard to the teacher's avatar. Test variants were not repeated within the same team. Correct answers added one point and incorrect answers subtracted 0.25 points for each participant. The teams, ordered by the sum of their members' scores, received from 0 to 12 points according to their position. The points accumulated by each team determined the classification after each stage. The test panels were arranged in various places (among the trees, in the sky or under water), to give variability to the game (Fig. 2). In the last three stages, dedicated to semiology, an additional task called "Normal or Pathological" (N/P) was included, consisting of a panel with 20 radiographs, of which 12 were normal. The teams had to correctly diagnose pathological cases and identify normal ones and their captain had to send the notecard with the answers. Zero to 12 additional points were assigned to the teams, using the same methodology as in the individual tests. In this way, the maximum points achievable in the competition were 108 (12 points × 6 tests × 3 tasks). Figure 3 shows a complete diagram of the game to facilitate understanding. The evolution of the contest was publicised on the virtual campus (Moodle platform) of the Radiology course. All participants received a certificate, specifying their participation in an educational innovation project for 18 h.

A 9-point Likert scale from 1 (very, very low mental effort) to 9 (very, very high mental effort) [29] was used to evaluate the cognitive load related to the review of the teaching content, the performance of the game tests, and the use of Second Life, at the end of the three blocks corresponding to anatomy and semiology. The test scores of each stage made it possible to measure the short-term knowledge profile of the participants. The correct answers were not provided during the game. One month after the game ended, all the students in the course were invited to a review seminar in the classroom, where the certificates and prizes of the participating teams were publicly given. The 180 questions of the game were reviewed except for 60 questions (10 randomly selected from each stage) that integrated a post-exposure test with 30 s to answer each question. Additionally, course scores were recorded to correlate with game scores. The results of participants and non-participants were compared.

Students who participated in the game were asked to complete an experience evaluation questionnaire, based on that from previous studies [28], consisting of 23 statements with answers on a five-point Likert scale (7 about Second Life, 8 about the game, 4 about the presentation of the different stages and 4 about the multi-choice tests), an evaluation of 1 to 10 points of various aspects of the project and a text box to add open comments (See Additional file 1: Appendix 1). No personal identification was included in the evaluations, and they were analysed anonymously.

The open comments included in the questionnaire were analysed through collaborative systematic coding by group consensus agreement [30], considering the codes obtained in previous studies on League of Rays [28]. In a first consensus meeting, four first-layer codes were established: positive, negative, suggestions, and team. The last, indicating that the comment made specific reference to teamwork. Second-layer subcodes were proposed and finally agreed upon in a second consensus meeting. The same comment could contain more than one different codes and/or subcodes.

Fifty-two students out of 185 (28.1%), organised into 13 groups of 4 students, participated in the game. Table 1 shows the results of the weekly tests and the N/P tasks, together with the final score of each team. The correlation between the game score and the sum of accumulated correct answers for each team was excellent (Pearson coefficient = 0.973). There were 21 undelivered weekly tests and 4 undelivered N/P tasks.

The questions about cognitive load during the anatomy and semiology stages were answered by 46 (88.5%) and 31 (59.6%) participants, respectively. The mental effort to use Second Life was significantly lower during the first three weeks (cognitive load = 3.8 ± 2.5 vs 5.7 ± 2.0; p < 0.001). No significant differences were found in the mental effort involved in reviewing the teaching content (5.7 ± 1.1 vs 5.8 ± 1.5) or performing the tests (6.0 ± 1.8 vs 6.5 ± 1.6). The mean percentage of correct answers in the game tests was higher in the three stages of anatomy than in the three stages of semiology (75.5 ± 19.5 vs 68.5 ± 20.5; p = 0.009), although the lowest percentage of hits were obtained in thoracic anatomy (Table 2). The percentage of correct answers in the N/P tasks was 72.0 ± 17.0, with the lowest result in the chest tasks (Table 3).

The post-exposure test was carried out by 45 participants and 83 non-participants. Participants obtained superior results than non-participants, with significant differences (p = 0.009) in the semiology questions (Table 4). The percentage of answers left blank for participants and non-participants was 2.4 ± 5.2 and 4.6 ± 6.7, respectively (p = 0.007). Forty-nine participants (94.2%) and 96 non-participants (72.1%) took the final examination in the June convocation, whose grades (mean percentage ± standard deviation) were 79.2 ± 15.3 and 71.3 ± 19.5, respectively (p = 0.021). The mean of correct answers by teams showed a low correlation with the final grades (Pearson's coefficient = 0.425) but moderate correlation with the post-exposure test (Pearson's coefficient = 0.572) (Fig. 4).

Thirty-five participants (67.3%) completed the evaluation questionnaire. The questionnaire presented high internal consistency in the three constructs that comprise it: the students' experience in Second Life (Cronbach alpha = 0.83), the perception of the game (Cronbach α = 0.81), and the global evaluation of the project (Cronbach α = 0.95). Kendall's τ-b test correlation matrices can be seen in Additional file 1: Appendix 3. Its analysis revealed a positive correlation between all pairs of variables with respect to the evaluation of the global experience (τ-b > 0.41 p < 0.01) and with respect to participants' opinion about the experience in Second Life (τ-b > 0.25; p < 0.05 in 19/21 pairs combination). In exchange, the results were disparate regarding the perception of the students about the game, finding a positive correlation between variables 8, 9, and 10, related to the design, information, and the adequacy of the contents to medical training (τ-b > 0.34; p < 0.05). In the factor analysis of the questionnaire, the results indicated that the sampling adequacy for the items related to the students' experience in Second Life was moderate (KMO index = 0.743), for the items related to the students' perception of the game it was medium (KMO index = 0.612), and for the overall evaluation of the project it was excellent (KMO index = 0.902). The opinion of the students about Second Life was positive in general (mean values ≥ 4.0 ± 1.0 points on a five-point Likert scale), except for the statement “you managed easily enough in Second Life”, to which 8 students (23%) disagreed (responses 1–2), obtaining a mean value of 3.6 ± 1.2 (Fig. 5). The opinion of the students about the game (Fig. 6) was very positive in terms of design, information, and the adequacy of the contents for medical training (mean values ≥ 4.5 ± 0.7). The participants agreed (answers 4–5) in having worked as a team during the game and in learning better in competitive games by 83% and 74% , respectively. There was some dispersion in the answers about the difficulty of the contents, 34% agreed that they were difficult, 23% gave a neutral answer (3), and 43% did not find them difficult. Most of the students found the educational presentations and the evaluations during the game interesting, appropriate to the learning objectives, adequate, and easy to follow or perform (Fig. 7). Finally, the qualification of the experience, from 1 to 10 points, obtained very positive values (mean values ≥ 7.8), highlighting the teacher, the educational content, and the usefulness for their training (Table 5).

Fourteen questionnaires (40%) included open comments (see Additional file 1: Appendix 2). Ten of them included positive comments; 8 were subcoded as appreciation, since they included expressions such as "great initiative", "very interesting", or "highly recommended"; 7 reflected aspects related to the formative nature of the experience, with expressions such as “I have learned a lot”, “great help to practice with cases'' or “it has helped me to understand radiology”; 5 were subcoded as fun, as students found it an entertaining or fun learning method. Other positive comments included words of thanks, availability to participate in future editions, and motivation, expressly indicating this aspect of the experience. Some participants included several positive subcodes in their comment, for example: “I think that learning through competition is something that young people are very interested in. Generally, we do not like to lose, especially if it is against friends who we are going to see the next day in class. I think it is a very fun way to keep track of the subject and stay motivated throughout the semester". Five questionnaires included comments coded as negative. On four occasions, referred to the game's schedule, indicating that it ended near the examination period. Other negative subcodes were technical limitation to handle Second Life, content difficulty, lack of time due to having a lot of work with other courses. Comments coded as suggestions were found in eight questionnaires. Four of them proposing rules of the game, some almost describe a completely different game, others suggest an earlier schedule. Finally, there were four comments coded as team: team-choice, indicating that teams should be chosen by the members; team-demand, referring to the fact that more demand for participation should be requested from the teams; team-help, expressing the help provided by their teammates to solve technical problems; or team-inconvenience, verbatim as follows: “… team play is somewhat more cumbersome than individual participation, but in the end the result has been good."