Sensor-based jump detection and classification with machine learning in trampoline gymnastics

An exercise in trampoline sport consists of ten elements (jumps) and the order in which the elements are performed is at the discretion of the gymnast. Each jump consists of various twists and somersault rotations and their number and combination determine the difficulty of each jump. According to the regulations of the Fédération Internationale de Gymnastique, the degree of difficulty, execution, time of flight, and horizontal displacement scores define a final total value using of the following equation:

For a long time, the total value in trampoline competitions consisted of two variables: the degree of difficulty and the overall skill execution. In order to make trampoline gymnastics more attractive and the evaluation of the gymnasts more objective, the technical committee of the Fédération Internationale de Gymnastique established the time of flight as a new performance value in 2010 and the horizontal displacement value in 2017 (Ferger, Helm, & Zentgraf, 2020). The increased efforts to objectify performance measurement in trampoline gymnastics through the introduction of the time of flight and horizontal displacement has led to the development of a new measurement system (called HDTS: horizontal displacement, time of flight, and synchronicity) and changes in the international scoring rules (Ferger & Hackbarth, 2017; Ferger, Hackbarth, Mylo, Müller, & Zentgraf, 2019; Ferger et al., 2020). In addition to these objectively measurable parameters, two other parameters are currently being collected with the help of judges. The execution, and, thus, the quality of the movement (E), is evaluated by four judges, and the difficulty of elements (D) by one judge. The duty of the judge of difficulty is to check the elements and difficulty values entered on the competition cards. The difficulty of each element is calculated based on the number of twist and somersault rotations. Each ¼ somersault and each ½ twist increases the difficulty score by 0.1. A fully completed somersault is given 0.5 of a difficulty point, a double somersault 1, a triple somersault 1.6, and a quadruple somersault 2.2 points. If a jump type has both somersault and twist rotations, the difficulty points are added up. Individual somersaults of 360–630° without twists, with straight or piked execution, receive an additional 0.1 points. Multiple somersaults of 720° or more, with or without a twist, executed in a straight or piked position, receive an additional 0.1 points per somersault (FIG Executive Committee, 2016).

Therefore, this judge has to recognize the execution of ten jumps in rapid succession, which can be a very challenging task, for example, the distinction between “Full-in Full-out” (one twist in the first somersault, one twist in the second somersault—822) and “Half-in Rudy out Fliffis” (half-twist in the first somersault, one and a half-twists in the second somersault—813). Furthermore, athletes and coaches have to recognize the exact execution of and, even more important, deviations from these jumps during their training sessions. Currently, there is no reliable method to automatically recognize the various jump types.

However, current developments in sensor technology make it possible to measure complex whole-body movements as they occur in technical compositional sports in order to assist the evaluation of movement quality (Camomilla, Bergamini, Fantozzi, & Vannozzi, 2018). Thus, in addition to technique and match analysis, one finds, among other things, a sensor-based recognition of flight elements in half-pipe snowboarding (Harding, Small, & James, 2007) as well as the use of IMUs for detection of training load and classification of gymnastic elements in trampoline gymnastics (Campbell, Bradshaw, Ball, Hunter, & Spratford, 2021; Helten, Brock, Müller, & Seidel, 2011).

From a practical coaching perspective, the use of wearable inertial sensors or sensor-based systems should add value to everyday training and competition. The IMUs are particularly well-suited to monitor performance in a reactionless way, in real time, and without cumbersome set-up procedures, such as calibration (Baca, 2006; Chambers, Gabbett, Cole, & Beard, 2015; Hood, McBain, Portas, & Spears, 2012; Knight et al., 2007; Li et al., 2016; Mendes, Vieira, Pires, & Stevan, 2016). These sensors are capable of measuring physical quantities related to the movement of a body and their measurements can be used to estimate temporal, kinematic, and dynamic parameters. This means that the systematic, objective, and reliable monitoring and evaluation of performance can strengthen the link between research and practical coaching, particularly in high-performance sports (Camomilla et al., 2018). However, the use of sensor-based jump detection in training practice has been scarcely practicable up to now. Complex mathematical calculations for relatively simple movements and a large number of sensors prevent the systematic use of these methods in training and competition (Harding et al., 2007; Helten et al., 2011).

New opportunities have arisen to advance the automated classification of sensor-based data with the expansion of machine learning (ML) and data science into other areas of research. This also includes recent research at the intersection between biomechanics, mobile sensors, and ML. Current research offers new possibilities in this area and indicates the potential for integrating ML systems for application in sports, particularly in the analysis of requirements for complex movements (Ancillao, Tedesco, Barton, & O’Flynn, 2018; Camomilla et al., 2018; Stetter, Krafft, Ringhof, Stein, & Sell, 2020). However, the use of ML methods for the detection of complex jump movements, such as those occurring in trampoline gymnastics, using mobile sensor data has been insufficiently explored up to now.

This paper presents the feasibility of the automatic classification of trampoline jumps using data from a data logger with an integrated IMU manufactured by 2D Datarecording (Datasheet 2d-Datarecording, 2021). We also discuss how to transform raw inertial data into meaningful characteristic features that underlie the detection of a jump and its assignment to a jump type. The duration of a jump is defined by jump limits from the acceleration data (ACC) which determine the start and end of the movement. The conditions for the execution of the respective jumps cannot be derived from the characteristic acceleration values alone; it is necessary to use the angular velocities of rotation of the gyroscope (gyro) about its three axes for this.

We tested eight different approaches to automatically classify jumps based on the sensor data. These are common ML techniques used in manifold applications, including sensor-based data (Huang & Perry, 2016; Meyer et al., 2019; Woltmann et al., 2022). The primary goal of the ML models is to automatically detect different types of jumps (somersaults, twists, and combinations thereof) in the ongoing training of trampoline gymnastics. Another goal is to simultaneously detect and record the various conditions of execution (tucked, piked, straight) and, thereby, expand the analysis of individual techniques in the long term. Finally, the classification of elements and determination of the difficulty of an exercise in competition could be automated.

All eight ML models were tested on the eight datasets, originating from the four different percentual splits and the use of raw or preprocessed data. This leads to 64 experiment values expressed in the final accuracies of each model on the test data specified. The detailed results can be found in Table 3. For brevity, we only present the data based on the best percentual split and the feature sets being used.

The best model is the DFF with just the raw data with 20% steps. It reaches an accuracy of 96.4% and, therefore, outperforms all other ML models by 0.3 to 7.5% points. The CNN does not perform better, even though it should be able to model complex tasks better than the DFF. The higher complexity is also shown in the training times. Therefore, the DFF is preferable to the CNN.

From this conclusion, we want to analyze the influences of the features on the prediction. Accordingly, we calculated the SHAP values on the best model, i.e., the DFF from Table 2. Figure 4 details a single decision for a back tucked somersault (somersault C). The chart illustrates the influence of each feature on the decision of the model to say ‘somersault C’. Each bar shows how much the feature contributes to the model towards this decision. The grey values next to the feature names are the actual measurements. The features ‘20_mean_Gyro_y_R’ and ‘20_mean_Gyro_y_Fil’ have the highest impact. These are the measurements for the rotation around the y‑axis during the phase from 20 to 40% of the jump. Therefore, we argue that the rotation around the y‑axis in the phase from 20 to 40% is the most expressive part of the jump type ‘somersault C’ not only for the model but also in general. This was tested for several other jumps and the SHAP values were able to identify several similar arguments, for example, the combined rotation around the y‑axis in the first 20% and the z‑axis in the first 40% is the most expressive part of a back full for the ML model. With this feature analysis and the averaged representative jump, there is a lot of potential for the application of the models in training and competitive scenarios. These applications are discussed in the following section.

This article has presented and discussed eight different ML models for the classification of trampoline jumps. Based on raw inertial data from an inertial sensor, ACC along the three axes of motion and angular velocities (GYRO) around the three axes of motion were collected for selected jumps (Table 1). Utilizing the approach for automated jump recognition presented herein, the training process can be augmented regarding the achievement of a target value of skills and skill combinations (e.g., difficulty value of a freestyle or compulsory exercise in trampoline gymnastics) and related to the qualitative development of the performance of motor tasks with additional parameters, such as the application of force (horizontal displacement, time of flight, and synchronicity) and quality of execution. It shows that the systematic, objective, and reliable monitoring and evaluation of performance using IMUs can strengthen the link between research and practical coaching, particularly in high-performance sports (Camomilla et al., 2018). Furthermore, the accuracy of the DFF model of 96.4% shows that the use of sensors for automated jump detection is rewarding despite complex mathematical calculations (Harding et al., 2007; Helten et al., 2011).

With our experiments, we have demonstrated that the DFF works best with the data based on raw values and that the CNN works best with the domain knowledge included. Additionally, this work also demonstrates that using the raw data and a DFF is slightly advantageous for the use case overall. However, there is no general recommendation for the use of a particular calculated feature set because each model requires its own adapted feature space. The NC performs badly, because, as expected, it does not incorporate any knowledge about the data except the a priori distribution of the jump types. Furthermore, we cannot recommend a specific model per se. On the one hand, the DFF performs best, but, on the other hand, the SVC has the shortest training time while having an acceptable accuracy. However, all models except the DFF require preprocessed data, which adds computational complexity to the data processing. Therefore, the application of a specific model depends on the particular use case.

Another interesting conclusion is the fact that the KNN performs well. Therefore, we can assume similarities of jumps of a certain jump type in their vector representation and raw measurements. This leads us to the assumption that these jumps might follow the same visual patterns and an averaged representative jump can be derived for each jump type. This representative jump can be used to advise athletes in training and recognize the phases of a jump type.

This uncertainty in model and data selection and their influence is a common conclusion in other current work classifying jumps (Bitén, 2021; Echterhoff, Haladjian, & Brügge, 2018). One general notion of these works is the discussion that an ML solution should be tailored to the problem by experimental design. This is also independent of the type of sport as both publications show. Bitén also acknowledges the vast number of models that can be used and limits the studies to a certain subset. In this work, we use a similar argument to support our extensive combination of models and features in the experiments and the conclusions drawn in the results and discussion. Additionally, this problem makes explainable artificial intelligence (XAI) a strong candidate for further research, as shown by our use of SHAP values. These are already used to back up some of our claims. Other works support similar arguments by applying XAI to relevant sports-related analysis, like performance of basketball teams (Wang, Liu, & Liu, 2022). To the best of our knowledge, no work for jump classification includes explainable ML argumentation.

The documentation of training data in trampoline gymnastics has so far been limited, for example, to the collection of the training duration, athletics, number of exercises, number of individual jumps and jump combinations, whereas the technical quality of elements is not systematically recorded. The application of mobile sensors combined with predictive models for jump detection offers new possibilities in this area and indicates the potential for integrating ML systems for application in sports, particularly in the analysis of requirements for complex movements (Ancillao et al., 2018; Camomilla et al., 2018; Stetter et al., 2020).

The automated determination of the difficulty of jumps according to international scoring rules is a special challenge for competition. However, the approach presented here also indicates prospects for supporting the difficulty judges and enables a direct derivation and formulation of individual target values for training. Finally, the competition data represent relevant factors influencing performance and can be made available in real time.

Machine learning methods can be used to detect jumps using sensor data. The application particularly of mobile sensors in combination with prediction models for jump detection has been insufficiently researched up to the present. The approach proposed herein basically shows considerable potential for expanding mobile applications in a sport with complex movement requirements. Future work is planned to apply these techniques to provide immediate feedback through which an athlete’s performance is evaluated or the difficulty judge is supported.