Reliability Exercise of Ultrasound Salivary Glands in Sjögren’s Disease: An International Web Training Initiative

Primary Sjögren’s syndrome (pSS), also recently renamed Sjögren’s disease (SD) [1], is a multisystem autoimmune inflammatory disease that affects multiple systems. It is characterized by the infiltration of lymphoid cells into the exocrine glands, particularly the salivary and lacrimal glands, leading to decreased gland function and resulting in symptoms such as dry eyes and mouth. Oral dryness is one of the most common manifestations of SD. The 2016 ACR/EULAR (American College of Rheumatology/European Alliance of Associations for Rheumatology) classification criteria for diagnosis of SD is based on five items (positivity for anti-SSA (Ro) antibodies, focal lymphocytic sialadenitis with focus score (FS) ≥ 1 foci/mm² in a minor labial salivary gland biopsy, abnormal ocular staining score ≥ 5, Schirmer test ≤ 5 mm/5 min and an unstimulated salivary flow rate ≤ 0.1 ml/min) but do not consider ultrasound sonography (US) of the salivary gland (SG) [2]. More recent data from a large international survey have demonstrated the usefulness of major salivary gland ultrasonography (SGUS) in classifying patients. The inclusion of SGUS with a similar weighting to other minor items improves the performance of the 2016 ACR/EULAR classification criteria [3]. The diagnosis of SD may be challenging in patients with suspected Sjögren’s disease, due to the absence of pathognomonic signs and many other conditions can produce similar presentations [4]. Various imaging techniques, including sialography, parotid scintigraphy, parotid magnetic resonance imaging (MRI), and most recently SGUS, have been proposed to improve the diagnostic approach to SD.

First proposed in 1988 [5], SGUS has proven to be a valuable tool for assessing damage in patients with SD, with its usefulness demonstrated since 1992 [6]. SGUS is a simple, non-invasive, non-irradiating, low-cost and effective examination that can easily be repeated over time. SGUS is performed at both parotid glands (PG) and submandibular glands (SMG). SGUS has demonstrated its good metric properties as an outcome measure for diagnosing SD, and may be helpful in evaluating SG parenchyma and in defining the presence of SG involvement [4, 7]. Studies have reported SGUS abnormalities (grade 2 or higher) in 62.8% of patients with SD, with no significant differences according to disease duration [8]. The SGUS score assesses greyscale (B-mode) features such as echogenicity, homogeneity, location of an (hypo)echoic area, hyperechoic bands. More recently, power Doppler mode (D-mode) has been used to evaluate inflammatory activity in SD [9‐11]. These two new simple scoring systems for SGUS (B-mode and Doppler-mode) have been developed by the OMERACT (Outcome Measures in Rheumatology) ultrasound subgroup in Sjögren’s disease.

Since the development of SGUS for SD diagnosis, several scoring systems have been developed, but no significant differences have been found in terms of reproducibility, sensitivity or specificity [12]. Several studies analyzed the impact of adding SGUS criteria for salivary gland involvement in SD, showing an increase in sensitivity without decreasing specificity [3, 8, 13, 14]. Currently, SGUS is routinely used only by expert sonographers. Few studies have studied intra-reader and inter-reader reliability, particularly among less experienced sonographers. De Vita et al., in 2020, from an European experience, demonstrated an excellent intra-reader reliability (Light’s kappa of 0.86) and a correct inter-reader reliability (Light’s kappa of 0.77) in a group of SGUS expert sonographers [15, 16].

The objective of our study was to assess the reliability of SGUS interpretation, using some independent items and a comprehensive new simplified OMERACT scoring system, among sonographers from different countries and different levels, following web-based training. This first study is a prelude for the beginning of an international SGUS study to evaluate Modification Abnormalities of Salivary glands in SD According to disease duration (MASAI study).

We conducted an international ultrasound reliability exercise in 12 expert centers for SD from nine countries. These centers routinely perform SGUS. The exercise included a total of 14 sonographers with different levels of experience in SGUS from France (Brest, Montpellier, Lille), Norway (Bergen), Germany (Munich), England (Birmingham), Serbia (Belgrade), Holland (Groningen), Italy (Pisa), South Korea (Gwangiu, Seoul) and USA (Baltimore), in two sessions of web-based training in July and September 2019. A web-based exercise was then conducted in September and December 2019. Among them, two participants were qualified as most experienced sonographers (MES) due to their participation in previous SGUS training and their experiences as a sonographer. One expert of SGUS from the center leading the project (SJJ), did not participate in the reliability study due to their role in training and ultrasound images selection.

All sonographers participated in the entire session’s program. All of them read all 60 images twice with an interval of 4 months between two reliability exercise sessions for intra-observer reliability.

In this international reliability exercise, we confirm good intra-reader and inter-reader reliability for the assessment of ultrasound greyscale abnormalities of Salivary glands in SD using a videoconferencing training session. First of all, teaching was done in two steps, using video-conferencing in web training sessions followed by a reliability exercise on static images with the different experts. This approach was chosen to evaluate the effectiveness of videoconferencing training in an international SGUS exercise for use in future clinical trials. It is worth noting that previous studies assessing the reliability of sonographers in SGUS were solely based on expert panels and did not include sonographers with different levels of experience. In our study, we aimed to fill this gap by including sonographers with varying levels of experience. Web-based training for ultrasonography has already been described in the literature, with satisfactory results, which seems comparable to traditional classroom training [18‐20].

The results of our study showed satisfactory results, especially for reliability of scores on the salivary gland parenchyma homogeneity, which was closely related to the SGUS diagnosis appreciation. This indicates that videoconferencing using static ultrasound grey images of salivary glands is effective for training in performing these ultrasounds. In accordance with the literature, we confirm that homogeneity is the most reliable item for both MES and LES [21]. However, it is important to note that homogeneity or heterogeneity alone is not sufficient to diagnose SD. On the contrary, the OMERACT scoring system is applicable to patients with SD, taking into account all elementary pathological elementary lesions and it should be emphasized that homogeneity is only a part of the OMERACT scoring system. Our findings suggest that the assessment of heterogeneity items, which include different elementary lesions such as hypoechoic areas and hyperechoic bands, may be relatively easier based on the observed kappa values. However, the evaluation of hypoechoic areas can sometimes be challenging, as they may not be well defined, and the assessment of hyperechoic bands can also pose difficulties. Hyperechoic bands in normal conditions correspond to fascia between lobule in parotid glands. When numerous hyperechoic bands are present, the gland becomes heterogeneous and hyperechoic, indicating the presence of fat deposits or fibrous tissue. This glandular state (grade 3 fibrous), defined by OMERACT scoring system is difficult to assess in an independent item. In summary, while homogeneity is a reliable item in SGUS assessment, it is important to consider the complete OMERACT scoring system, as it incorporates various pathological elements. The evaluation of heterogeneity including hypoechoic areas and hyperechoic bands can be more challenging but is crucial for a comprehensive assessment of salivary gland abnormalities in SD. The lower kappa values observed for hyperechoic bands in the structural damage assessment of the OMERACT scoring system can be attributed to the challenges in defining and evaluating these bands consistently. This could explain the lower reliability in this aspect of SGUS evaluation. Regarding the comprehensive OMERACT scoring system, the results showed substantial intra-reader reliability and moderate inter-reader reliability for MES, while fair to moderate inter-reader reliability was observed for LES. However, when the binary OMERACT scoring system was employed, categorizing grades 0–1 as no or low parenchyma damage and grades 2–3 as major parenchyma damage, the reliability increased, with a substantial to almost perfect intra-reader reliability and substantial inter-reader reliability in MES. The good results obtained with binary OMERACT system is very useful and shows that the OMERACT scoring system could be simplified between 0 and 1 normal and 2 and 3 abnormal. However, fibrous appearance and hypoechoic areas aspect would not be described in this simplified classification. The moderate reliability observed for the 0–3 OMERACT scoring system, primarily due to a one-point difference between observers, may not significantly impact the patient’s overall status in routine practice and this is corroborated by the good reliability in ultrasound diagnosis appreciation among MES but also among LES.

The OMERACT scoring system is a new simple evaluation of salivary glands for diagnosis and may also be useful in assessing change in patients with SD over time. In routine practice, this scoring system is very helpful because it does not require scoring of all the pathological features in SD described by Jousse-Joulin et al. and it can be easily applicable for novice ultrasonographers [21]. However, results of LES are lower than MES, confirming that it is necessary to train and practice salivary glands ultrasound regularly in order to have a better accuracy. It should be noted that, due to the variation in experience level of sonographers, only kappa coefficients have been calculated and Light’s kappa coefficient has not been calculated because Light’s kappa would have been biased by extreme results.

Our study has some limitations. Firstly, the definition of MES based on their participation in previous SGUS training and their experience as sonographers could introduce some subjectivity and lack of consensus to determining their expertise level. Secondly, the use of static ultrasound images differs from live images acquisition, which may impact the interpretation and assessment of SGUS findings. Lastly, no color Doppler US was used because the results of the new OMERACT scoring system was not yet published [22] and it would be interesting to explore the inclusion of color Doppler in future studies to further enhance the evaluation of inflammatory activity in SD. SGUS diagnosis was a subjective appreciation of sonographers on the diagnosis of Sjögren’s disease based only on SGUS images.

Finally, the reliability exercise demonstrated that SGUS can be a valuable tool for evaluating salivary gland parenchymal damage in patients with SD, particularly among MES, but can be useful for less trained sonographers. This SGUS reliability exercise is a prerequisite to a future international SGUS study to evaluate modification abnormalities of salivary glands in SD according to disease duration, using web training.

Videoconferencing training sessions in an international reliability exercise could be an excellent tool to train expert and non-expert sonographers. SGUS homogeneity is the most reliable item and remains a good feature to distinguish normal from abnormal SG parenchyma independently of diagnosis. However, structural damage evaluation by the new comprehensive OMERACT scoring system to diagnose SD can be easily used by non-expert sonographers in a binary method.