Indicators of breast cancer severity and appropriateness of surgery based on hospital administrative data in the Lazio Region, Italy

Breast cancer is a major cause of cancer death among women in the region of Lazio (central Italy, including Rome), and in all of Italy [1, 2]. Several randomized clinical studies have shown that simple mastectomy is as effective as more radical treatment (Halsted's radical mastectomy) in terms of survival [3, 4]. Although national and international guidelines list treatment protocols specific to the tumor's stage [5, 6], the degree of adherence to these guidelines in Italy is not known, and in recent years the surgical techniques for breast cancer used in different Italian hospitals have varied widely [7, 8]. Clinical stage is the principal determinant of surgical treatment; specifically, there has been an under-use of conservative procedures with stage I and II tumors [9]. Stage I (up to 2 cm in diameter) and stage II tumors (more than 2 cm in diameter) can be treated with conservative surgical intervention (lymphoadenectomy and quadrantectomy) as effectively as with radical mastectomy [10, 11].

In Italy, regional hospital information systems (HIS) contain information on all hospital admissions. They are a valuable source of data for verifying the quality and appropriateness of health procedures and/or in evaluating the economic impact and frequency of use of new technology [12, 13]. Current literature which uses HIS data to evaluate breast cancer surgery is limited to geographic and temporal variability of various surgical procedures and their possible determinants [14, 15]. A recent study conducted in Australia described geographical variations in surgical treatments offered to women with breast cancer [16], while an American study showed that conservative treatment is most common among young women, and more frequently performed in university hospitals [17]. Data analyses from the London Tumor Registry showed noticeable differences in therapeutic procedures in the 42 hospitals involved in the 1980's study [18]. Age and education level appear to play important roles in the type of surgical intervention offered [19]. A direct relationship was observed between volume of hospital activity and adherence to guidelines [20, 21]. Various studies in Italy also have documented levels of inappropriateness according to hospital and patient characteristics [7, 22]. In Lazio, age, tumor severity and hospital volume were associated with the use of conservative therapy [8].

HIS data have important limitations for breast cancer surgery, mainly because they do not contain information on tumor stage, available only in medical charts, the crucial indicator for determining appropriate treatment, and more generally, because administrative data completeness and accuracy have not been established.

Despite the absence of tumour size in the HIS many authors used the combinations of primary and secondary ICD9-CM diagnosis to estimate breast cancer severity [21, 23].

The aims of this study are to: 1) validate a breast cancer severity indicator (SI) calculated from primary and secondary diagnoses (ICD-9-CM codes) from the HIS; 2) verify the differences in the SI, tumor stage, and tumor size in evaluating appropriateness of breast cancer surgery, and, 3) verify the accuracy and completeness of the HIS in recording breast cancer diagnoses and surgical procedures.

Clinical charts were obtained for 387 of the 398 cases included in the sample (97.2 %). Sixty cases (Eleven cases according to SI and 53 according to TNM stage and pT) could not be assessed, due to unclear or incomplete charts. According to the SI indicator, most of the cases had a medium-low level of severity (about 66% in class 2). Based on the TNM, more than 70% of cases were classified stage I or II; based on pT, most cases (48.1%) were classified as T1 (Table 3).

A positive correlation (ρ = 0.6; p < 0.001) was observed between the severity indicators SI and TNM and the distribution of tumor stage in each category of the SI, showed in Figure 1, suggest a rather good agreement between the two classifications. However, SI did not correlate as well with pT size (ρ = 0.3 p < 0.001). In fact in class 1 of the SI 77% of tumors were size Tis and 15% were size T1, in class 2 66% were T1 and 24% T2, in class 3 37% were T1, 44% T2 and 14% T4. Because only two cases belonged to class 4 the percentage was not calculable.

Based on pT, 28 % (94/334) of cases were under-treated and 5% (17/334) were over-treated; based on TNM stage, the percentages were 35% (117/334) and 5% (18/334) and on the SI they were 25% (82/334) and 6% (19/334) respectively (Table 4). The SI and pT (gold standard) agreed relatively well in classifying "under"- and "over-treated" cases (kappa = 0.53). The agreement between the "under" and "over-treated" cases classified on the basis of the TNM and tumor size pT was very high (kappa = 0.82). The accuracy of the SI in estimating under-treatment was fair (58% of all procedures classified as under-treatment using pT where also classified as such using SI; 67.1% of all procedures classified as under-treatment using SI where classified as such according to pT), while it was quite good for over-treatment (respectively 88.2% and 78.9%). The TNM staging system showed values of 96.8% and 77.8% for under-treatment and 100% and 94.4% for over-treatment (Table 4).

Lastly, the total agreement between HIS data and clinical charts was quite low (kappa = 0.35) and the PPVs (Table 5) were 11% for class 1, 75% for class 2 and 76% for class 3. Class 1, tumors in situ, in particular showed very low PPVs. A misclassification was observed for tumors in situ by the HIS; it misclassified 8 invasive tumors as cancer in situ, and failed to identify 14 out of 15 of them. As a result, 67 cases were listed as "localized tumors" in the HIS but as "non-localized" on the clinical chart.

The study indicates that with data from hospital discharge reports it is possible to construct an indicator of breast tumor severity that performs quite well compared to clinical and histological classifications. However, it needs to be used with caution in estimating the appropriateness of surgical treatment, because although it is sufficiently thorough in estimating over-treated cases, it tends to underestimate under-treatment. The validity of the severity indicator from the HIS depends explicitly on the completeness and quality of the data.

Our proposed indicator is based on ICD-9-CM codes of breast cancer diagnoses and procedures, and uses some of the criteria proposed by Kahn et al in 1996 [23]. Since the ICD-9-CM coding does not include tumor size, their indicator is based on the assumption that there is a positive correlation between the extent of the growth (localized, loco-regional, and distant) and the size of the tumor. Consequently, the criteria were based on the level of diffusion of the tumor from the ICD-9-CM codes, which served as a proxy for tumor size, and thus for severity. Kahn et al evaluated the accuracy of ICD-9-CM codes in correctly identifying localized breast cancer, and then in documenting the percentage of lymphoadenectomies that had been performed, comparing to the data from the cancer register. The authors calculated the raw agreement of the two information sources was 82%; the sensitivity and specificity was acceptable (above 85%). Thus, the data from the HIS were considered valid in evaluating appropriateness of surgery typology according to tumor severity (localized/not localized), but not for analyzing therapeutic outcomes[23]. Our study modified Kahn's SI to better differentiate degrees of severity: we considered cancer in situ as a category in and of itself (otherwise included in the same category with localized tumors) and we distinguished, in the non-localized tumor category, those with loco-regional metastasis from those with distant metastasis. A similar approach is found in the study conducted by Rohan et al [21].

The correlation of the SI with TNM stages was rather good (0.6; p < 0.001), but not as good with tumor size (pT) (0.3; p < 0.001). These results suggest that the SI can be considered an adequate proxy of tumor severity, but is not as precise an indicator of tumor size. Unfortunately, tumor size is usually the only indicator available to physicians when determining treatment.

We also evaluated the possibility of using the SI to evaluate appropriateness of treatment: the SI appears to classify cases of over-treatment relatively well but underestimates under-treated cases. We found that TNM and pT classifications are better indicators of under- and over-treatment; this may be because size is a component of TNM staging. It is, however, important to note that the SI seems a conservative indicator of both over-treatment and under-treatment, minimizing the risk of inappropriately categorizing difficult cases.

In order to rule out possible errors of classification, codification or input, data registered on an ad hoc schedule were used instead of the original HIS information to construct the SI. Subsequently however, the completeness and accuracy of the discharge reports were assessed comparing them to clinical documents: this study was a part of a larger study on quality of data from HIS, methods and results are available on technical report published on web [28]Hospital discharge records have been shown to have problems of under-notification of health data, which greatly decrease their value to analytic studies, [29, 30]. Furthermore, in our study there were a number of cases for which it was not possible to define the TNM stage due to lack of information on the clinical documents, which could have in part reduced the "validity" of our gold standard. The PPV of the HIS is overall quite low (Kappa = 0.35), mainly due to a high level of discordance in the first SI category (carcinoma in situ). The PPV in the other three categories comparatively is rather good.

Our findings support previous literature results that hospital discharge reports are potentially utilizable to define the clinical severity of breast cancer, mainly for the purpose of public health evaluations, such as studying variations in breast cancer treatment in key demographic groups. The limited ability of SI to predict tumor size and the questionable quality of HIS data do not support the use of this indicator to evaluate clinical outcomes or appropriateness of surgical treatments for breast cancer. Improvement of HIS coding accuracy and completeness could make it possible the use of administrative data for epidemiological purposes.