Enhancing endoscopic measurement: validating a quantitative method for polyp size and location estimation in upper gastrointestinal endoscopy

In our previous research, we introduced a novel quantitative method, which consists of integrating electromagnetic tracking technology¹ with a conventional endoscope (Pentax EPK-i), as illustrated in Fig. 1, and is complemented by a newly developed computer-vision-based algorithm. This algorithm can yield the 3D coordinates of a detected polyp’s border in real scale. By fitting an ellipse to these 3D point coordinates, one can derive both the longest length of the polyp and the distance from its centre to a specific anatomical landmark, as detailed in our previous work [23]. Evaluated in a simulated upper gastrointestinal model and an artificial rounded polyp, a root mean square error (RMSE) of less than 1 mm for size and 3 mm for location estimation were achieved.

The sensor accuracy test results indicated no significant degradation in sensor performance in endoscopic settings, underscoring its potential applicability. Moreover, through sensitivity analysis, we provided recommendations for capturing endoscopic images from a polyp to ensure optimal accuracy with our proposed method. This includes avoiding large relative movements (specifically, relative translation > 30 mm or relative rotation > 30 degrees), minimal movements (displacement < 3 mm), or pure forward–backward movements.

Following the promising initial results, the quantitative method was evaluated in this study, while endoscopy procedure was performed using ex vivo pig stomachs. While the core principles of the quantitative method remain intact for this evaluation, a few modifications were necessary to adapt to the unique characteristics of the new setting. These modifications related to the two steps of the computer vision-based algorithm including finding corresponding points and polyp segmentation.

While the movement of the polyp cannot be controlled, signs of its movement might be detected when multiple image pairs are used for a single polyp size determination. Therefore, here a simple approach of capturing multiple images and subsequently multiple estimations to compensate for potential polyp movements between taking image pairs were investigated. If the polyp remains stationary, the 3D coordinates of the polyp’s centre would be expected to stay consistent across all measurements. Consequently, outliers in the estimated 3D coordinates of the polyp centres were identified using quartiles, and those image pairs were excluded from the final estimation, while the average of remaining estimations was considered as final estimation.

In the following sections first, two adapted steps including polyp segmentation and corresponding points will be described briefly, and subsequently, the experimental design for method evaluation will be explained.

Given the diverse characteristics of polyps, which reflect the variability observed in actual polyps, here we employed a pre trained DL-based model called Polyp-PVT (Pyramid Vision Transformers) [24], for polyp segmentation purposes in our experiment. This model consists of a pyramid vision transformer [25] as an encoder to extract multi-scale feature maps where the image divided to smaller patches in each level. The model consists of three other modules, a cascaded fusion module (CFM) to fuse the high-level features through progressive integration, a camouflage identification module (CIM) to capture low-level features using an attention mechanism, and a similarity aggregation module (SAM) to integrate low and high-level features for final segmentation. The Polyp-PVT model aims to extract more powerful and robust features to improve polyp segmentation regarding accuracy and generalisation ability. The model achieved a Dice similarity coefficient (DSC) of 0.917 and 0.937 on the Kvasir-SEG [26] and CVC-ClinicDB [27] that consist of 1000 polyp images and 612 polyp frames, respectively. For our ex vivo experiment, we employed this model and subsequently applied some image post-processing techniques to further improve the segmentation quality including, basic morphological operations such as opening, and filling operations, to eliminate small detected regions of interest (ROI), and fill in small holes within the ROIs, or deformable segmentation based on active contour models [28] in situations where the model output covers the polyp partially. Figure 2 illustrates some examples of segmentation outputs on polyp images as explained above.

Repetitive and indistinctive texture surfaces in endoscopic images can adversely affect the feature detection and matching algorithms. Consequently, in this paper, we explore a new approach for determining corresponding points based on shape context, which is unaffected by the texture-less nature of the endoscopic images.

An object’s shape can be represented by a discrete set of points, denoted P = \({p}_{1}\), \({p}_{2}\), …, \({p}_{n}\) where \({p}_{i}\) ∈ R2 from the internal or external contours of the object. In the case of a polyp, the shape can be defined by a set of points located on its border. Given the set of points P, with n points of the polyp shape in one image and the set of points Q, with m points in the other image, the shape context method [29] can be used to find one-to-one correspondences between these two 2D sets of points, P and Q. The shape context is a vector of local geometric information that characterises the distribution of points around a reference point on the shape boundary. The shape context can be defined as a histogram using a log-polar coordinate system. The log-polar system represents a point based on the logarithm of its distance from the origin (log r) and its angle (θ) from a reference direction. This is illustrated in Fig. 3 with a log-polar coordinate system of 5 bins for log r and 12 bins for θ.

Two shapes of the polyp border in two endoscopic images are presented by sample points. Considering a log-polar coordinate system; each shape context is a log-polar histogram of the coordinates of the rest of the point set measured using a reference point as the origin of the log-polar coordinate system. As shown in Fig. 3, the shape context of relatively similar points (i.e. green and red) on the polyp border is visually more similar, while the shape context of another point (i.e. blue) is quite different. This concept was used to find corresponding points along the border of a polyp in the present study. The result of finding corresponding points for one polyp in the pig stomach experiment using this approach is illustrated in Fig. 4.

The sample size was determined based on the correlation coefficient and considered a two-tailed hypothesis: Null Hypothesis: The correlation coefficient is zero. Assuming a power of 0.8 (β = 0.2), a significance level of α = 0.05, and an anticipated correlation coefficient of C = 0.7, a preliminary sample size was calculated. To account for unforeseen circumstances, an additional 10% was added to this initial calculation, resulting in a final sample size of 15.

Five pig stomachs were utilised in one of which three polyps were created by tying off sections of the stomach wall, in various sizes representing the common real polyp sizes. For this validation, endoscopy procedure was simulated on ex vivo pig stomachs, which were placed in a container and secured at the oesophagus end and sealed at the pylorus end, as depicted in Fig. 5. Notably, the stomach remained unfixed, permitting potential random movements. This procedure, executed by an experienced expert, aimed to mimic real endoscopy in terms of navigation and environment. The stomach was inflated with air and actions such as water jet cleaning and suctioning were performed to replicate actual endoscopic conditions.

The extracted recommendations from model experiment in our previous work were used as a feedback mechanism to achieve the higher accuracy. Five different images of each polyp in a selected perspective made by the expert were taken and used for size and location estimation by the proposed quantitative method. The location of the oesophagus-gastro junction was also recorded, and the locations of the polyps with respect to the junction were determined using the proposed method. This procedure was repeated twice to compute the test–retest reliability for both size and location estimations. After each simulated endoscopy, polyps were measured with a digital calliper. Because the polyps are asymmetric, viewing perspective can affect length measurements. To reduce bias, we aligned the calliper perspective with the captured images. Although calliper measurements are not the most reliable for irregular and non-rigid polyps, we compared them to our method for validation, expecting general agreement between both techniques.

The fifteen polyps examined in this study had an average size of approximately 12 ± 4.3 mm (mean ± SD), ranging from 5—20 mm. The Intraclass Correlation Coefficient (ICC) for polyp size estimation between test–retest data was assessed using a two-way mixed effects model [30]. The ICC value for absolute agreement was 0.991 (95% CI 0.973 to 0.997, p < 0.05). To explore the agreement of test–retest data for both size and location estimations, we utilised Bland & Altman (B&A) analysis [31]. The differences in the B&A plot can be expressed as units or percentages (i.e. (test–retest)/average %). Given the standard deviation of the differences (s) and the mean of the differences (d), the two upper and lower agreement limits were computed as d ± 2s. Considering the normal distribution of the differences, 95% of differences are expected to lie between d + 1.96s and d − 1.96s.

The normal distribution of differences for test–retest data was confirmed using the Shapiro–Wilk test [32]. Assuming a two-tailed test with a p-value of 0.05, the confidence intervals for the mean line were derived.

Figure 6 shows the Bland & Altman plots for both size and location estimations using the proposed method, presented in both unit and percentage forms. For size estimation, the mean difference stands at − 0.17 mm or − 2.03%. This difference is not statistically significant, given that the line of equality lies within the 95% confidence interval. Consequently, 95% of the differences between test–retest data lie between − 1.31 and 0.96 mm or from − 13 to 8.9%.

For the estimation of polyp location (i.e. the distance from the oesophagus-gastro junction), the Bland & Altman plot indicates a mean difference of − 0.4 mm or − 0.21% between test–retest data, which is not statistically significant. The limits of agreement, considering a 95% confidence interval, range from − 4.7 to 3.9 mm or from − 3.7 to 3.3%.

Figure 7 presents the Bland & Altman plot comparing polyp size estimations made by the proposed method and the calliper. According to the plot, no significant bias is evident (0.2 mm or 2.22%), with 95% of differences fall between − 1.4 and 1.8 mm, or − 13 to 17.4%.