External validation of a prognostic model based on total tumor load of sentinel lymph node for early breast cancer patients

The sentinel lymph nodes (SLN) biopsy is consolidated as an effective procedure in the staging of solid tumors, especially in breast cancer [1‐3]. For years, the exhaustive study of these lymph nodes through immunohistochemical techniques, and particularly molecular methods [4] have resulted in an increased detection of minimal neoplastic involvement, which promoted research on its biologic and prognostic relevance in early diagnostics of breast cancer.

Several studies were conducted to determine the total tumor load (TTL) that would allow to avoid unnecessary axillary lymph node dissection (ALND), based on the prediction of non-SLN affectation [5‐8]. The ACOSOG Z0011 study published in 2011 allowed to identify a group of breast cancer patients with positive SLN in which ALND could be omitted. This resulted in a considerable increase in the number of patients with affected SLN and no ALND [9‐11].

Nodal involvement (pN) has been always considered one of the main prognostic predictors in early breast cancer [12]. However, due to the previously mentioned research works, its study is now restricted in many cases to the SLN (pN_(sn)), which might compromise its prognostic value. The PLUTTO study [13] was developed as the next logical step to collect evidence of the prognostic value of the TTL, defined as the sum of the copy number of CK19 mRNA detected in every SLN examined, expressed as a concentration (copies/μL).This study concluded that the TTL allows to distinguish two groups of patients with different overall survival (OS) and disease-free survival (DFS), independently from other known prognostic factors, using a threshold of 25,000 copies/μL. From this evidence, prognostic models were developed for DFS, loco-regional DFS (LRDFS) and OS, based on TTL, the patient’s age, and a risk score derived from tumor characteristics.

The primary objective of the present study was to conduct an external validation of a previously developed model [13] for the prediction of 5-year DFS in patients with a diagnostic of breast cancer that had undergone SLN biopsy with TTL determination from one step nucleic amplification (OSNA) method. Secondary objectives were the validation of analogous models for LRDFS and OS.

We collected longitudinal data of a historic cohort of patients with breast cancer from eight different Spanish hospitals, who underwent SLN biopsy January 2008 to June 2014, and evaluation of all SLN by OSNA (without conventional histological evaluation). Exclusion criteria were carcinoma in situ, tumors with no expression of Cytokeratin 19 (CK19), cases with systemic primary treatments (chemotherapy or hormonotherapy), lack of information related to demographics, tumor, sentinel lymph node biopsy, breast cancer treatment and follow-up in the clinical records, or when OSNA evaluation of SLN was not made with all, complete, excised SLN. The study was approved by the Ethical Review Board of each institution (reference IRB: 2017-4-3-HCUVA, Hospital Clínico Universitario Virgen de la Arrixaca, Murcia, Spain). Given its retrospective nature, informed consent was not obtained, but patient privacy was protected by using a patient study number in the data collection and through a dissociated management of the data.

Data collected for each patient included clinicopathological characteristics as well as clinical follow-up. The risk score derived from tumor characteristics was computed by adding up the number of risk factors (size > 2 cm, lymphovascular invasion, ki67 > 20, HER2 positive, ER negative, PR negative, and tumor grading) [13]. Follow-up time was defined as the time from the SLN biopsy date to the last follow-up date or the event date (progression or death), whichever came first. Progression events were classified as local, regional or distant.

A total of 1089 patients from eight sites were considered for inclusion in the validation study. However, 200 patients were excluded due to violation of selection criteria (4), lack of data required for model validation (193) or both (3), so that the validation set included 889 patients.

Table 1 shows the main characteristics of patients in the training and validation sets. In the 889 validation patients, the median follow-up was 6.4 years (Q1 5.6; Q3 7.0), 84 had a DFS event, 67 had a LRDFS event, and 51 died. A total of 1367 lymph nodes were analyzed, 544 (61.2%) patients had zero positive nodes, 278 (31.3%) one positive node, 55 (6.2%) had 2 positive nodes, 11 (1.2%) had 3 positives nodes and 1 (0.1%) had 4 metastatic SLNs. TTL values ranged from 0 to 27 × 10⁶ CK19 mRNA copies/μL, with a median of 0 and IQR 3100. In 544 cases, TTL was 0. In the remaining 345 patients who had non-null TTL values, the quartiles were 1400, 10⁴, and 10⁵. These figures were similar to those reported previously for the training set [13].

Our results document the external validity of a model previously developed to predict DFS from age, a risk score based on tumor characteristics, and TTL, in early breast cancer patients in whom SLN analysis was conducted by molecular methods. This model performed well when applied to a new sample of patients (validation set) recruited from centers different from those involved in its construction (training set). The discrimination ability of the model (AUC ROC) ranged from 0.78 (at 2 years) to 0.73 (at 5 years) in the training set, and from 0.78 to 0.71, respectively, in the validation set. The quartiles of the MPP defined groups of patients with heterogeneous DFS curves (Fig. 2). The difference between the extreme groups in actual 5-years DFS probability was 14.4% in the training set, and 11.0% in the validation set (see Table 2 and text).

Our study also confirmed significant different 5-years DFS according to the 25,000 copies/μL TTL threshold. However, this was only done as a way of documenting that TTL may contribute valuable information for 5-years DFS, but we do not suggest the use of any threshold [17] for prediction or therapeutic decision making. In the model, TTL was introduced as a continuous variable, since categorization is unnecessary, and has been shown to be inefficient in prognostic model development [18].

Extensive work has been done on the prediction of non-SLN involvement from OSNA [5‐8, 17, 19‐22]. The prognosis of breast cancer in terms of DFS and overall survival, has been studied in clinical trials [2, 10], and observational studies proposing tools for prediction [3, 13, 23, 24], most of them basing the prediction on ALND and/or tumor markers.

The Nottingham Prognostic Index (NPI) was developed long ago, using tumor size, grade, and lymph node staging from 387 patients recruited in a single center [23]. Lymph node involvement was based on biopsy of “a lower axillary node, an apical axillary node and a node from the internal mammary chain”. The NPI was subsequently applied (using conventional axillary lymph node staging) to large samples of patients from other populations with a variable degree of success [25, 26], but these exercises did not follow the standard methodology for the validation of prognostic models, and measures of model discrimination were not reported. Recently, a newer version of the NPI (NPLI Plus) has been proposed to predict development of distant disease [3].

A recent study proposed a nomogram for the prediction of survival on the base of a modified lymph node ratio in breast cancer patients undergoing ALND [24]. In this study, patients that only received SLN biopsy were explicitly excluded (1328 out of 5736 patients that received surgery, or 23%), and therefore their model and ours should be considered complementary. Their model showed a good discrimination capacity (c-index 0.789 in the validation set), though this was assessed by split-sample rather than external validation.

Validating a prognostic model means establishing that it works satisfactorily for patients other than those from whose data the model was derived. Neither internal validation (data splitting or cross-validation) nor temporal evaluation (subsequent patients within the same centers) addresses the wider issue of the generalizability of the model. As the goal of validation is to demonstrate satisfactory performance for patients from a different population from the original, it is clearly desirable to evaluate a model on new data collected from an appropriate patient population in different centers. For this reason, external validation is considered to be the most stringent form of validation [15, 27].

As mentioned in the 8th edition of the American Joint Committee on Cancer (AJCC) for breast cancer, the biology of the tumor should be considered for clinical and prognostic information [28]. Axillary staging (pN) is still an independent prognostic factor in early breast cancer patients; however, after the publication of the ACOSOG Z0011 trial and others, the therapeutic significance of ALND became questionable limiting the prognostic information of axillary involvement only to sentinel lymph node (pN(sn). Several models show a good prognostic value for positive SNL nevertheless, our prognostic model for early breast cancer based in an objective factor as is TTL and other outcomes may improve the management and follow-up of breast cancer patients using this variable. The validated model helps to identify patients with a lower 5-years DFS.

Our study has several limitations. First, data were retrospectively collected and, in general, retrospective data are less accurate and more prone to bias than prospective data. However, because the data were prospectively recorded, and their nature is not subjective, we think there is little risk of important inaccuracies. High inclusion rate of estrogen receptor positive patients, 88.1% in the training set and 89.5% in the validation set, as a result of uniformity subtype group could be considered a prognostic bias, nevertheless, patients included are representing a real sample of the population. Concerning a potential risk of bias, given our selection criteria we cannot think of any mechanism that could introduce a distortion in the predictive ability of the model. Second, our model predicts DFS at 5 years, which is a minimum meaningful time to assess prognosis in early breast cancer. A longer time frame of 10 or 15 years would be of great interest, but the molecular analysis of SLN is a relatively novel method [4], and its wide adoption in our environment dates less than 10 year ago [6, 7]. Definitely, another limitation is the need to get OSNA method which is not available in all centers; however, the use of molecular methods is recommended for a better prognostic precision, provided that the center can have it [28].Last, despite our selection criteria were not very restrictive, not all breast cancer patients satisfy them.

A prognostic model of 5-years DFS based on age, a tumor risk score, and TTL from SLN, was shown to be valid when applied to patients in centers different from those involved in its construction. The model was useful to identify patients that are at higher risk of disease progression, which may be interesting in a context of restrictive indication of ALND.