Are there differences among operators in false-negative rates of endosonography with needle aspiration for mediastinal nodal staging of non-small cell lung cancer?

Clinical tumor-node-metastasis (TNM)-staging in non-small cell lung cancer (NSCLC) is a pivotal factor for deciding the treatment plan and assessing prognosis [1, 2]. Although computed tomography (CT) and positron emission tomography (PET) are recommended for evaluation of TNM staging, invasive mediastinal staging techniques are still the gold standard approach for confirming N-staging because imaging techniques have low sensitivity and specificity to predict metastasis to mediastinal and hilar lymph nodes (LNs) [1]. Recently, endosonography (endobronchial ultrasound [EBUS] and esophageal ultrasound [EUS]) with needle aspiration (EBUS/EUS-NA) was recommended as the first choice for mediastinal nodal assessment over surgical staging because of its cost effectiveness, minimal invasiveness, and high sensitivity for evaluation of N-stage in patients with NSCLC [1, 3, 4].

However, this recommendation is based on the operator having appropriate experience and skill level. The experience of the operator performing the needle biopsy is known to have a significant impact on the diagnostic yield [5‐7]. Moreover, additional surgical N-staging such as mediastinoscopy is recommended if the results of EBUS/EUS-NA are negative but there are suspicions of mediastinal nodal metastasis because the sensitivity of EBUS/EUS-NA is not high enough to completely rule out nodal metastasis [1, 4, 8]. Therefore, it is important to manage the test quality of EBUS/EUS-NA to achieve appropriate diagnostic yield, reduce false negative results, and avoid unnecessary surgical N-staging.

There are many published studies about patient-related and tumor-related predictors for false negative results of EBUS/EUS-NA [9, 10], but few reports about operator-related factors. Although many studies have evaluated how fast individual trainees adjust to the procedure as assessed by various evaluation tools [7, 11, 12], there is a lack of studies analyzing the false negative rates of EBUS/EUS-NA depending on operator and using surgical staging as the reference value. In this study, we aim to evaluate the false negative results of EBUS/EUS-NA to N2 or N3 LNs depending on operator and accumulated experience.

We retrospectively identified 705 patients with NSCLC who had negative N2/3 by EBUS/EUS-NA performed by six operators and subsequently underwent MLND. This study population included 111 (15.7%) false negative cases. In our institution, an operator can independently perform EBUS/EUS-NA without a supervisor after at least 200 cases of observation and assistance over a period of 6 months. Nevertheless, procedure-related factors such as duration of procedure, frequency of combination with EUS, number of evaluated lesions, node size, and obtained core tissue per node varied among operators. There were no statistically significant differences in baseline characteristics of age, sex, location of primary tumor, histologic type, and evaluated nodal station according to operator. False negative rates to N2/3 LNs according to operator were not statistically different. In addition, there was no trend in the change of false negative rate according to accumulation of operator experience. Therefore, although there are differences in procedure style according to operator, there appears to be no difference in false negative rates after the operator has completed sufficient training.

In the present study, the rate of false negative results on N2/3 LNs by EBUS/EUS-NA was 111/705 (15.7%). Among the patients with false negative results, 24/111 (21.6%) had false negative results to inaccessible LNs, and these rates were similar to those reported in previous studies [9, 10]. Previous studies have identified several factors associated with false negative results of EBUS/EUS-NA [9, 10, 17], including primary tumor location (central or left lung), abnormal findings of mediastinal LNs on CT and/or PET, inadequacy of the sampling, internal necrosis in the LNs, and rare cancer types. Although we did not perform multivariate analysis to identify the independent patient and tumor characteristics associated with false negative results, young age, female gender, inaccessible stations by EBUS, and larger size of LNs had a trend for higher false negative rates (Tables 1 and 2).

There are several studies on the learning curve of EBUS/EUS-NA. Some studies have concluded that optimal accuracy of the procedure can be achieved after a relatively short learning curve (about 10 procedures) [12, 18]. However, another study demonstrated that the minimal diagnostic yield was achieved with 60 procedures, but optimal results were obtained after 100 procedures [19]. In another study, significant variation was seen in the learning curves of individual operators, with ongoing improvement in EBUS/EUS-NA skill even after 200 clinical cases [7]. In the present study, there was no difference in the false negative rate among the operators despite differences in procedure style. The differences in interpretation of the learning curve for operators between these studies may be explained by the fact that most of the previous studies involved EBUS novices, whereas our study was conducted on operators with experience of observation and assistance for least 200 cases. Recent research showed that simulation-based training using an assessment tool is effective for improving the EBUS performance metrics [11, 20, 21]. However, no study has evaluated whether the skills demonstrated on a simulation assessment are transferred to an improvement in clinical skills as performed in patients [22]. We think that sufficient clinical experience, even indirect experience such as observation and assistance, is necessary to increase the diagnostic performance in real clinical practice.

There are several limitations of this study that affect the generalizability of our results. First, this is a retrospective study performed in only one hospital, and EBUS/EUS-NA was performed without general anesthesia or ROSE. However, we evaluated cases performed by six operators with various degrees of experience and different operator characteristics such as duration of procedure, frequency of combination with EUS, number of evaluated lesions, node size, and obtained core tissue per node. We think that the variation in operator characteristics may overcome some of the limitations associated with single-institution research. Second, we were not able to evaluate the complication rate according to each operator and experience accumulation. Third, we have no data on false negative rates to N1 LNs. Although differentiation between N0/1 and N2/3 is the most important factor in N-staging of NSCLC because it greatly influences the decision of treatment policy, some recent papers report that differentiation between N0 and N1 is also important because it may influence delicate decisions such as neoadjuvant clinical trials for stage II NSCLC, selective MLND, and selective limited resection [23, 24]. Therefore, further research is needed to evaluate the false negative rates to N1 LNs. Finally, we did not confirm whether 200 cases of observation and assistance during 6 months is the optimum training for the EBUS/EUS-NA procedure.

In conclusion, if the operator has sufficient experience of bronchoscopy and at least 6 months of experience of observation and assistance in EBUS/EUS-NA, the false negative rate to N2/3 LNs might be not affected by the operator. In addition, false negative rates to N2/3 LNs may not be reduced by additional experience. However, further studies are needed to evaluate the minimum amount of observation and assistance required before performing the independent EBUS/EUS-NA procedure.