Development and Evaluation of a Quality Assessment Tool for Laparoscopic Sleeve Gastrectomy Videos: A Review and Comparison of Academic and Online Video Resources

Video recordings of surgical procedures have been increasingly used for training, demonstration of new techniques, presentation of unique cases or determination of outcomes, as well as for legal reasons [1, 2]. Laparoscopic sleeve gastrectomy (LSG) is one of the most common procedures in bariatric surgery [3, 4] and many academic and commercial videos have been published. Video analysis of LSG technique identified associations between technical variations and postoperative complications, including staple line leak and haemorrhage rates [5]. These findings highlight the importance of operative videos in measuring surgical quality. Recordings of operations are also utilised for educational purposes and play an important role in simulation training [1, 6].

Video recordings have the advantage of demonstrating operative skills remotely, allowing surgeons in training to use videos to prepare for operations and quickly learn skills and procedures [2, 7]. Additionally, it permits retrospective review and monitoring of progression. Consequently, there is growing evidence that videos enhance surgical skills and techniques [1, 8] and improve surgical training [9]. From a clinical perspective, videos can link postoperative complications to intraoperative techniques and identify errors [5]. These recordings are presented in various methods, including academic publications (e.g., video case reports), on commercial platforms (e.g., YouTube) and educational resources [2, 10]. However, these platforms present videos in different form such as full length, highlights, narrated or with subtitles, and without audio. Heterogenous reporting of videos can reduce the educational value of them, impair the communication of the intended learning or clinical outcome and limit the possibility of auditing them.

The steps of a gold standard of LSG procedure have been established by expert consensus in the Bariatric Metabolic Surgery Standardization (BMSS) [11] to improve consistency in surgery, data collection and outcome reporting. However, the utilisation of these guidelines in practice and the demonstration of best practice in academic and commercial videos is unknown. Our aim was to determine the quality of both academic and commercial videos by using a checklist based on BMSS guidelines and describe the degree of heterogeneity in both peer-reviewed publications and LSG videos on YouTube.

A total of 716 articles were identified. The flowchart of the search is presented in Figs. 2 and 3. Overall, twenty-six academic videos and sixty-seven commercial videos were included in the analysis.

The median time of the LSG videos was 13 min and 43 s. Specifically, videos uploaded in peer-reviewed literature had a median time of 7 (5–12) minutes whilst LSG videos uploaded to YouTube had a median time of 17 (2–120) minutes.

The total median score for academic videos was 5 (1–10) and 4 (1–10) out of a total of 11 for academic and commercial videos, respectively. None of the videos analysed, either peer-reviewed or on YouTube, described the volume of the resected sleeve specimen or gained a full score of 11. Table 1 shows the percentages and scores for each component of the developed checklist, comparing academic and commercial videos.

All videos describe their aim or reason for uploading the video, which was primarily demonstrative or educational. None of the YouTube videos described short-term outcomes, whereas half of the peer-reviewed articles did. High percentage of agreement and good inter-rater reliability were demonstrated between the six assessors from various level of expertise. Regardless of the level of expertise there was an agreement in nearly all elements of the checklist, except for distance from pylorus, as demonstrated on Table 2. The raw total score for each assessor is provided in Supplementary File 1.

There were notable differences in the type and frequency of audio elements. Narration was more prevalent in academic videos, whereas commercially available videos often lacked audio or used music instead of formal narration to describe the operation (Fig. 4). Statistically significant differences were observed in the use of narration (P < 0.001) and the absence of audio (P < 0.001) between the two groups. However, no statistically significant differences were identified in the use of music between academic and commercial videos (P = 0.72), as demonstrated in Table 3.

The analysis demonstrated inconsistent reporting in the videos uploaded in both peer-reviewed literature and the YouTube platform. Videos published in the academic literature had a greater tendency to describe technical information in detail. They provided information regarding the surgical instruments used and patient outcomes. There was no audio or background music found in more than 85% of YouTube videos, compared to 26% of academic videos. There is potentially higher educational value in narration, for example, through the expert description of surgical instrumentation. This is supported by the LAP-VEGaS consensus statement [10]. The findings of this study therefore suggest that videos clips in the academic literature have more educational value than commercial platforms.

Further analysis revealed that all videos failed to mention the resected volume of the sleeve. The exact importance of this has not yet been established. There is some evidence in literature to support the relationship between resected sleeve volume and weight loss [15‐17], although other authors have disputed this and described the resected volume to be related to the initial body mass index, rather than as a predictor of weight loss [18, 19].

This qualitative review has led to the development of an evidence-based checklist for LSG video reporting using BMSS guidance. Although there are no established video reporting guidelines yet, checklists are routinely used in the literature to assess observational studies, systematic reviews and meta-analyses. For example, the STROBE statement comprises of a checklist of items that should be included in articles reporting observational research [20] and the MOOSE checklist contains specifications for reporting meta-analyses of observational studies in epidemiology [21]. To our knowledge, this is the first reporting guidelines checklist for surgical videos. We have presented a method for LSG video reporting, of which a similar approach can be applied to report other surgical videos. The overall inter-rater agreement, along with the agreement between surgeons and trainees at various levels of expertise, indicates that the checklist functions effectively across different skill sets. This suggests that the checklist is accessible and valuable not only for experienced surgeons but also for those with less expertise, including trainees. Consequently, this tool could be useful in helping both surgeons and trainees monitor their performance and identify any changes or improvements over time. The utility of this checklist across diverse levels of surgical expertise highlights the potential of video-based quality assessment as a broad-based tool for enhancing surgical proficiency and outcomes. Therefore, a new initiative was initiated to move the medical field forward in televisual education. The SPRINT (Standards for Presenting and Reporting clinical InterveNtions Televisually) guideline is currently under development as a universal checklist for all clinical interventional videos, registered with the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network (www.​sprintguidelines​.​com). The aim of SPRINT is to improve the reliability and quality of published video research by promoting transparency and quality of surgical videos in the literature.

The applications of surgical videos are broad and variable, having a strong impact on clinical practice and patient outcome. Intraoperative video-based technical skills assessment, analysed in a systematic review by Bavardi et al. [22], supported the association between superior surgical techniques and lower postoperative morbidity. The authors proposed video analysis as an approach to surgical quality improvement. There is a significant opportunity to improve e-learning strategies through robust studies with clear reporting outcomes and standardized metrics. Some resources such as WebSurg IRCAD’s online University has proven to be a high-quality, peer reviewed e-learning resource which adapts to LAP-VEGaS guidelines [23, 24] providing content that meets Health on the Net Foundation (HONCode) ethical requirements for quality, confidentiality, neutrality, transparency, community and visibility, guaranteeing reliable and expert health information [25]. These peer reviewed and readily available educational videos are becoming an important part of surgical education. Beyond educational importance of videos, there are other essential benefits from recording operative procedures. Videos can potentially identify errors or complications, and therefore aid in the improvement of patient safety by learning from those experiences. Moreover, videos can possibly be used in the future as documents or evidence in medicolegal cases.

One of the strengths of this study is the objectivity of the designed evidenced-based checklist to aid comparing uploaded LSG videos. However, there are important limitations that must be addressed. This study included a relatively low number of videos used in the analysis, especially from peer-reviewed literature, and a lack of expert consensus agreement of the developed reporting guidelines video checklist. It must also be taken into account that videos on commercial platforms do not undergo a strict peer-review process prior to upload onto the website.

Surgical videos of LSG are increasingly becoming more important and accessible on the internet, however there is no reliable quality assessment tool available. It is possible to develop an evidence-based checklist that can assess the quality of these surgical videos based on previous research and society guidelines. The checklist presented here delivers consistent results across users of varying expertise levels, demonstrating its usability. Academic videos were overall more detailed and more commonly narrated suggesting higher quality and educational value. Most commercial videos lack important elements such as description of technique and narrations of the procedure. Furthermore, there is a lack of peer review to act as quality control on these platforms. These findings demonstrate that there is a need for a reliable quality assessment tool beyond LSG in order to promote transparent and accurate reporting of surgical videos. Future studies should aim to achieve expert consensus, through the Delphi technique, on the development of optimal video reporting guidelines in surgery.