PREDICT: a new UK prognostic model that predicts survival following surgery for invasive breast cancer

The primary analysis was based on data from patients with invasive breast cancer diagnosed in East Anglia, UK between 1999 and 2003 identified by ECRIC. ECRIC covers a catchment area population of approximately 5.5 million people and registers all malignant tumours occurring in people resident in East Anglia at the time of diagnosis. ECRIC also records prospectively demographic, pathologic, staging, general treatment and outcome information. Death certificate flagging through the Office of National Statistics provides the registries with notification of deaths. The lag times for this are a few weeks for cancer deaths and two months to a year for non-cancer deaths. In addition, ECRIC checked vital status by querying the National Health Service Strategic Tracing Service. Vital status was ascertained at the end of June 2008 and all analyses were censored on 31 December 2007 to allow for delay in reporting of vital status. Breast cancer specific mortality was defined as deaths where breast cancer was listed as the cause of death on Parts 1a, 1b, or 1c of the death certificate.

Information obtained from ECRIC included age at diagnosis, number of lymph nodes sampled and number of lymph nodes positive (categorised as 0, 1, 2 to 4, 5 to 9, and 10+ nodes positive), tumour size (categorised as <10 mm, 10 to 19 mm, 20 to 29 mm, 30 to 49 mm, 50+ mm), histological grade (I, II, III), oestrogen receptor (ER) status (positive or negative), mode of detection (screening vs. clinical), information on local therapy (wide local excision, mastectomy, radiotherapy), and type of adjuvant systemic therapy (chemotherapy, endocrine therapy, both). Exact chemotherapy regimens are unknown, but the majority of breast cancer patients in the ECRIC population received first or second generation chemotherapy during this time period. Patients who did not undergo surgery, patients with incomplete local therapy (wide local excision without radiotherapy) and patients with fewer than four nodes excised with a diagnosis of node-negative disease were excluded from the analyses, leaving a study population of 5,694 individuals (Table 1).

An independent validation dataset was comprised of women diagnosed with invasive breast cancer between 1999 and 2003 within the boundaries of the West Midlands Cancer Intelligence Unit (WMCIU). The geographic area served by WMCIU has a population of approximately 5.3 million individuals. Identical patient demographic information and study endpoints were retrieved from the WMCIU cancer registration database, with the same exclusions applied as for the ECRIC dataset. The total validation study population included 5,468 individuals (Table 1). As this was a large population-based study, with full anonymisation of all data, informed consent and ethical approval was not sought.

Breast cancer specific mortality and mortality from other causes (competing mortality) were modelled separately. For breast cancer specific mortality, a Cox proportional hazards model was used to estimate the hazard ratio associated with each prognostic factor. As the effect of ER status varies over time [11] ER negative and ER positive tumours were modelled separately. Nodal status, tumour grade and tumour size were modelled both as categorical variables and as ordinal variables. The models with ordinal variables fit the data better, and so these were chosen for the final models. Chemotherapy, endocrine therapy, and tumour detection by screening were treated as simple indicator variables. For the purposes of this study, screen-detected cancers were those discovered by screening mammography in the NHS Breast Screening Programme which at the time offered three-yearly mammography to women aged 50 to 64. In an exploratory analysis, age at diagnosis was included as a categorical variable in five age groups (<40, 40 to 49, 50 to 59, 60 to 69 and 70+) but these were not found to be significantly associated with breast specific mortality (data not shown) and age was excluded from subsequent models.

Competing mortality was modelled separately and adjusted for age at diagnosis. Exploration of the age specific beta-coefficients suggested that the effect varied exponentially with age; the best fit model was age to the power of 2.38.

We used the baseline survivor function from the ER negative and ER positive Cox proportional hazards models for breast cancer specific survival adjusted for the other prognostic factors to estimate the predicted number of deaths from breast cancer. Deaths from other causes were estimated from the baseline survivor function for competing mortality after adjusting for age. The total number of deaths at Years 5 and 8 after diagnosis was estimated by summing the breast-specific and competing mortality. Observed and predicted deaths were compared using a standard Chi-squared test. Model discrimination was evaluated by calculating the area under the receiver-operator-characteristic (ROC) curve (AUC) calculated for breast cancer specific and overall deaths at Year 8 past diagnosis. The ROC curve plots sensitivity against 1-specificity at different predicted risk thresholds. Model calibration was assessed using a simplified goodness-of-fit (GOF) method for the Cox proportional hazards model proposed by May and Hosmer [12] in which observed and model-based estimated deaths at Year 8 after diagnosis within deciles of risk score were compared. This provides a goodness of fit Chi-square test. As the baseline hazards and prognostic variable coefficients differed for ER positive and ER negative models, separate GOF tests were carried out for these models. In subgroup analyses, where numbers within deciles of risk score were small, quartiles of risk scores were used. Person-years lost were calculated by taking the area under the cumulative risk curve. Analyses were performed using STATA, version 9.2 (StataCorp, College Station, TX, USA).

The ECRIC data set was used to derive the primary prognostic models for breast specific and competing mortality. Beta-coefficients and standard errors for each prognostic factor in both the ER negative and ER positive models are provided in Table 2. The estimated relative hazard associated with treatment with adjuvant hormone therapy was smaller than the published estimate based on randomised clinical trials [7, 8] in women with ER positive tumours, and was associated with a poorer prognosis in women with ER negative tumours where no effect is expected based on clinical trial data. These differences are likely to represent bias due to clinical selection or patient non-compliance in the observational data.

As expected, this model was well calibrated. The model tended to over-predict mortality, but the difference between actual and predicted deaths was less than one percent at five and eight years after diagnosis (14.8 vs. 15.6 percent and 18.9 vs. 19.0 percent, respectively), differences that were not statistically significant (P = 0.10 and 0.83 respectively). There were 31,904 person-years of follow-up compared to 31,662 predicted. Model discrimination was also good - the calculated area under the ROC curve (AUC) for the overall model was 0.81 (SE 0.0074) (Table 3). Similarly, breast cancer specific actual and predicted mortality were within one percent at Years 5 and 8 past diagnosis (10.6 vs. 11.0 percent, P = 0.28 and 12.9 vs. 13.5 percent, P = 0.26, respectively; AUC = 0.84, SE = 0.008) (Additional file 1, Table S2). The ER positive and ER negative prognostic models were also well-calibrated overall and for all subgroups, and the goodness of fit tests suggest that the models fit well across different risk categories. The ER positive model provided better discrimination (AUC = 0.82, SE = 0.0111) than the ER negative model (AUC = 0.75, SE = 0.0171).

The WMCIU study population of 5,468 individuals was used for independent prognostic model validation. Overall the model was well calibrated. The tendency to over-predict mortality was slightly worse than with the ECRIC data but the difference between actual and predicted mortality was still small (<2 percent) at five years past diagnosis (15.8 vs. 17.4 percent, P = 0.004) and less than one percent at Year 8 past diagnosis (17.5 vs. 18.4 percent, P = 0.11) (Table 4). There were 25,917 person-years of follow-up compared to 25,809 person-years predicted. The overall model AUC was calculated as 0.79 (SE = 0.0079). Breast cancer specific actual and predicted mortality were within two percent at Years 5 and 8 past diagnosis (11.0 vs. 12.6 percent and 12.2 vs. 13.6 percent, respectively; AUC = 0.82, SE = 0.0083) (Additional file 1, Table S1).

Overall, the ER positive and ER negative prognostic models were well-calibrated, although both models predict more breast cancer deaths than observed. The overestimation was slightly greater for the ER negative model than the ER positive model. In ER negative disease, the Year 8 actual breast cancer mortality rate was 25.0 percent compared to 30.6 percent predicted; for ER positive tumours, Year 8 actual and predicted breast cancer mortality were within one percent (8.9 vs. 9.2 percent). Overall model fit was good (GOF P-values > 0.05), although the fit was less good for some sub-groups. Specifically, for ER positive disease, the fit was not so good in women aged <35 years (GOF P = 0.01) and 35 to 49-year-old age category (P = 0.045). For ER negative disease, the model fit in node negative disease (GOF P = 0.03), 30 to 49 mm tumours size category (GOF P = 0.02) and high grade tumours (GOF P = 0.001) was not so good (Additional file 1, Table S2).

Model discrimination was also good, again being somewhat better for the ER positive model (AUC = 0.81, SE = 0.0111) than the ER negative model (AUC = 0.75, SE = 0.0169). There were no significant differences between the ROC curves generated with the ECRIC and WMCIU data (ER positive χ² = 0.17, P = 0.68, ER negative χ² = 0.00, P = 0.95) (Figure 1).

We also explored the overall and breast cancer specific mortality within T1N0 and T2N0 good prognosis subgroups where decisions regarding adjuvant therapy can be difficult and challenging (Additional file 1, Table S3). In the WMCIU population, 1,931 individuals were diagnosed with T1N0 tumours, while 1,182 individuals were diagnosed with T2N0 tumours. For T1N0 tumours, actual and predicted five- and eight-year overall mortality rates were within 2.1 percent (5.5 vs. 7.6 percent and 6.1 vs. 8.2 percent, respectively); actual and predicted five- and eight-year breast cancer specific mortality rates were within one percent (2.4 vs. 3.3 percent and 2.8 vs. 3.6 percent, respectively). For T2N0 tumours, actual and predicted five- and eight-year overall mortality was within 2.5 percent (11.7 vs. 14.1 percent and 13.5 vs. 15.2 percent, respectively); actual and predicted five- and eight-year breast cancer specific mortality was within one percent (7.9 vs. 8.7 percent and 9.1 vs. 9.4 percent, respectively).

Given the difference in the estimates of the effects of hormone therapy from the ECRIC dataset compared to published clinical trial data, we also fit models (constrained models) with the relative hazard of hormone therapy constrained to the published estimate from the 1998 overviews (relative hazard 0.68 for ER positive tumours). Under this constrained model, the coefficient estimates for the other prognostic factor coefficients were similar to the original, data-driven model (Additional file 1, Table S4). Performance of the constrained model was slightly poorer in the ECRIC data than the full data driven model (Table 5), but the difference between actual and predicted mortality at eight years and between actual and predicted person-years of follow-up was still small. In the WMCIU validation dataset, the constrained model performed better than the full, data-driven model at predicting eight-year mortality and person-years of follow-up.

Finally, we tested the performance of models using the data derived coefficients for grade, node status, tumour size and mode of detection from the full and constrained models with the benefit estimates from the 1998 overviews (Table 5). First generation chemotherapy benefit estimates were applied in all these analyses. Again the models performed slightly poorer than the full, data-driven model in the ECRIC dataset, but somewhat better in the WMCIU validation dataset.