Cosmetic outcome as rated by patients, doctors, nurses and BCCT.core software assessed over 5 years in a subset of patients in the TARGIT-A Trial

Patients completed a self-reported cosmetic assessment at baseline (before radiotherapy) and annually thereafter for five years as previously reported [9]. At the same time points, a Radiation Oncologist and a Research Nurse independently completed the same cosmetic assessment for each patient, blinded to other rater scores. Doctor and nurse raters were also involved in patient care and were not blinded to treatment allocation. At the same time-points, digital photographs were taken of the unclothed torso from neck to navel, frontal view, with the patient standing, a method used by others [12, 22]. All baseline measurements were taken before radiotherapy, the majority being after WLE, except 3% in the TARGIT-IORT group and 18% in the EBRT group that were taken before WLE (where patients were randomised before their WLE (prepathology) earlier in the study - prior to postpathology becoming the more common approach at the Australian study centres).

All cosmetic assessments used the Global Harris Scoring System (also known as the Harvard Scale) of Excellent, Good, Fair or Poor (Additional file 1: Table S1) [10, 18, 23, 24]. Responses were dichotomized into Excellent-Good (EG) or Fair-Poor (FP) categories in order to facilitate comparison with other published studies of cosmesis. The digital photographs were analyzed separately [8] utilizing the validated objective BCCT.core software [14‐16, 25]. Scores are referred to in the subsequent text as doctor (Radiation Oncologist), nurse, patient and BCCT.core.

Statistical significance was set at the level of p < 0.01 to account for multiple comparisons [26, 27].

IBM-SPSS-V22 (SPSS Inc.,Chicago, IL) was used for: non-parametric analysis (Mann-Whitney U-Tests and Chi² tests) of raw unadjusted data, two sample t-tests for change between baseline and Year-5 scores, and inter-rater reliability analysis using the Kappa statistic to determine consistency among raters. Kappa scores were interpreted based on levels of agreement described by Landis and Koch; < 0 Poor, 0.0–0.20 Slight, 0.21–0.40 Fair, 0.41–0.60 Moderate, 0.61–0.80 Substantial, 0.81–1.00 Almost perfect [28]. Generalized estimating equations (GEE) with a variable covariance structure were used for the longitudinal dichotomized cosmesis endpoint scored by doctors and nurses using SAS-V9.3 (SAS Institute, Cary, NC).

Inter-rater reliability analysis revealed only 5 statistically significant Kappa scores out of the 36 observed time point pairings (Table 2). Four of these showed only fair agreement between raters. Only one time point scored moderate agreement which was between nurse and doctor scores at Year 1 with Kappa = 0.46 (p < 0.001), 95% CI (0.24, 0.68).

One can see that most of the variation in responses was in the EBRT arm. BCCT.core correlated well with patient scores in the TARGIT-IORT group but not in the EBRT group (Fig. 3).

Figure 3 illustrates that each rating system followed a similar trend, with overall cosmesis scores showing 25% variation between raters within each of the 5 time points. Rater disagreement was seen such that doctors gave the most positive scores, followed by nurses, then patients and then BCCT.core. The significantly different time points (p < 0.01) between raters were Year 1 (doctors and BCCT.core both gave worse scores than nurses; doctors and nurses scored cosmesis better than patients) and Year 2 (BCCT.core gave worse scores than nurses) (Table 3).

In terms of percentage agreement, nurse and doctor scores appeared to be the most closely related with an overall agreement of 80.8%; range 73.5% (Year 2) to 88.3% (baseline) (Table 2). The rating system that appeared most similar to the patient scores overall was the doctor scores, with an overall agreement of 78.6%; range 73.2% at Year 3 to 80.8% at baseline).

The effect of missing data on the patient scores at Year-5 was tested by substituting the previous years’ result. For the EBRT group, this increased the proportion of an EG score from 68.4% to 69% and for the TARGIT-IORT group it decreased the proportion from 90% to 88%.

Since its first use in 1998, intraoperative radiotherapy has been tested in randomised clinical trials and offered as adjuvant breast radiotherapy for over 20,000 women. Given the fact that breast cancer local recurrence outcomes are no different with TARGIT-IORT compared with EBRT, the obvious difference in patient experience (a single treatment instead of several weeks of daily treatments in the hospital) is of great importance.

Cosmetic outcome post various forms of intra-operative radiotherapy has been previously reported [8, 9, 30‐34] however no study has compared four different rating systems, even with other approaches of breast conserving therapy. The earliest cosmetic assessment of TARGIT-IORT utilised a satisfaction index by asking patients to give a score for what she expected (E) and another for what she observed (O). It was found that for appearance, there was a trend for better scores for TARGIT IORT boost compared with EBRT and no difference between the two treatments was found for the satisfaction indices for texture [35, 36]. The current TARGIT-A sub-study assessed the agreement in cosmetic outcome between one objective and three subjective rating systems, by investigating the proportion of patients scoring an Excellent-Good (EG) outcome in each treatment group. Overall, the majority of patients in both treatment groups scored an EG cosmetic result across all scoring systems, however 32% (12/38) of EBRT patients assessed at Year-5 self-reported a Fair-Poor result. This compares poorly to the 90% of TARGIT-IORT patients (27/30 patients) self-reporting an EG result at this time-point, but nearly fits within the previously reported expectation that overall, 70–80% of EBRT patients will have an EG result [37]. Of those not providing a score at Year-5 due to having withdrawn (n = 3) or non-compliance (n = 11), 5 out of 7 from the EBRT group had a previous EG score and 3 out of 4 from the TARGIT-IORT group had a previous EG score, suggesting that had they provided a 5 year score which maintained their previous score, the proportions may have been 69% and 88% respectively. This suggests study attrition did not cause the large difference at this time point. Further discussion about the patient-reported findings has been detailed elsewhere [9].

It is well known that EBRT can have a negative impact on long term cosmetic outcome. This was confirmed in this study, with the EBRT group failing to reach baseline proportions of EG scores across all follow-up time points and all rating systems. Although follow-up scores when compared to baseline scores were not statistically significantly different between the TARGIT-IORT and EBRT groups in the subjective measurements, TARGIT-IORT patients did reach baseline proportions of EG scores in all rating systems, from Year-3 or 4 onwards. This suggests IORT patients do experience an initial decline in cosmesis but this improves beyond the second year of follow-up. Baseline assessments were taken prior to surgery in 1 patient in the EBRT arm (2%) and 12 patients (18%) in the TARGIT-IORT arm, which may introduce some bias in the interpretation of these results. Previously reported BCCT.core data showed patients receiving TARGIT-IORT were significantly more likely to have an EG result at Year 1 and Year 2, compared to patients who received EBRT [8]. The only time point to have moderate agreement between two raters was Year-1, between nurses and doctors, when cosmesis scores were poorer than at other time points. This suggests that for both treatment groups, cosmetic outcome assessed by all raters reached a nadir at Year-1.

Overall, a higher proportion of patients in the present study had an EG result compared to other different modalities of intraoperative radiotherapy. The Montpellier study (IORT delivered via a linear accelerator) reported ‘Excellent to Good’ cosmetic scores at 6 months and a Mammosite study (utilizing a balloon applicator) found 84% and 78% Excellent to Good scores at Years 1 and 2 respectively [30, 31]. Cosmetic assessments for these two studies were made by a clinician from physical assessments and photographic review and were not patient reported. Cosmetic outcome data from the ELIOT study (using 21Gy electron intra-operative radiotherapy) was reported as ‘good’ in the majority of cases, scored by both patients and clinicians [32]. A South African study delivering Iridium¹⁹² via after-loader found that 74% of patients reported an EG score after 7 years of follow-up [33].

Historically, patients have been known to evaluate cosmetic outcome more favourably than their clinicians, possibly due to a range of psychological factors; not wanting to displease their clinicians (or their teams) is a common suggestion. It may also be that factors other than aesthetics influence a patient’s evaluation of cosmetic outcomes; it may be related to quality of life, expectations or the difference in interpretation of what the Harris Scale means between different raters [20, 38, 39]. In this study, we found that patients’ self-assessments were similar to the objective assessment of the BCCT.core software, but only in those who received TARGIT-IORT, perhaps influenced by the better breast related quality of life with TARGIT-IORT [9]. Previous studies utilizing BCCT.core [17, 38, 39] have found that patients receiving EBRT score better than BCCT software. In this study, doctors were most likely to report an Excellent or Good outcome, followed by nurses. These results confirm previous research that subjective Harris Scale scores reported by the patient, her doctor and a nurse should not be used interchangeably [12, 13, 17].

Guidelines produced by the EORTC in 2006 stated that since there was no ‘gold standard’, at least 3 measures should be used to assess cosmetic outcome: a subjective panel of 5 members using the Harris Scale; some form of objective measurement system; and some form of skin damage grading, however this may not always be practical [17, 40].

Some authors have stated that, as patients have to live with the outcome of treatment, the patient self-assessments are the most important; although some contest that due to the low reproducibility of such results and the high dependence on psycho-social factors, they should be measured in conjunction with an objective measurement system (13, 17, 19, 34). An approach used in the past has been blinded review by two or more radiation oncologists, however the BCCT.core system was used instead of blinded review in the present study. Even though BCCT software is an objective assessment, we posit that the most practical and perhaps most relevant measurements are those carried out by the patient herself. This is particularly true when it is used within the context of a randomised trial, as all other factors would be equally balanced between the two arms and any effect would be attributable to the randomised allocation and should reflect the real-world scenario. It would be ideal to use all four methods in every study, or a combination of at least two, but as the other methods are more resource intensive and if they don’t correlate with the patient perception, they are arguably less relevant.