Comparison of accuracy of fibrosis degree classifications by liver biopsy and non-invasive tests in chronic hepatitis C

We recruited different populations with liver biopsy to evaluate the different diagnostic means. Thus, population #1 provided different pathologist skill-levels and blood tests. The large population #2 included only blood tests. The more recent populations #3 and #4 included Fibroscan and blood tests. The four populations were separately analysed due to initial differences in study designs; this allowed us to evaluate accuracy robustness given these differences.

Patients with chronic HCV hepatitis, liver biopsy, blood tests and available Fibroscan were consecutively recruited in different populations: #1 to #4 described in Table 1. Each population had different characteristics and fibrosis assessments. Inclusion and exclusion criteria are detailed in previous publications or below for new populations. Briefly, patients did not receive antiviral or known anti-fibrotic treatments. Liver biopsy, blood withdrawal and Fibroscan, when available, were performed within a maximum interval of 6 months. The study protocol conformed to the ethical guidelines of the current Declaration of Helsinki and was approved by local ethics committees. Patients gave written consent.

Population #1 included 205 patients recruited from primary, secondary or tertiary care centres as detailed elsewhere [10] for a diagnostic study. Liver biopsy was read initially by a local (first line) pathologist, then independently by an expert from the Metavir group and finally by two other experts with a consensus reading in case of disagreement.

Population #2 included 1,056 patients provided by five centres participating in the Sniff 17 study [11]. Thus, individual patient data were available from five centres, independent for study design, patient recruitment, and blood marker determination. Blood and pathological determinations were not centralized. Pathological assessments were performed twice by the same pathologist in Grenoble, once in Bordeaux and once each by two pathologists in Angers, Tours and PACA region, with a common final reading in cases of disagreement.

Population #3 included 458 patients provided by 19 centres participating in the Fibrostar study [12]. Blood determination and liver interpretation were centralized. Liver specimens were read by two senior experts, one of whom was from the Metavir group.

Population #4 included 349 patients provided by three centres participating in the Vindiag 7 study (exploratory set) [13]. Blood and pathological (one senior expert in each centre) determinations were not centralized.

Fibrosis was staged in liver biopsy according to Metavir staging [1] in all patients. This fibrosis stage classification was used as the reference for the calculation of accuracy. In population #1, where several readings were available, the consensus reading by two experts was the reference. "Expert pathologist" was defined as a senior pathologist specialized in hepatology. At least one expert pathologist was available in each study. Blood tests were determined in all studies; we only evaluated here those for which a detailed fibrosis class classification has been described, i.e., FibroMeter [14] (Biolivescale, Angers, France) and Fibrotest [5] (Biopredictive, Paris, France). Second generation FibroMeter (FibroMeter^2G) [14], the most widely studied, and a recent third generation FibroMeter (FibroMeter^3G) [8] were evaluated. Two studies also included Fibroscan (Echosens, Paris, France) as this technique has only been available since 2004; usual technical aspects have been described elsewhere [15]. All successful measurements of Fibroscan were included in the calculations.

We distinguished as fibrosis degrees the histological fibrosis stages and the fibrosis classes provided by non-invasive tests and including one or several fibrosis stages. Several fibrosis classifications were evaluated:

Data were reported according to STARD statements [16]. Quantitative variables were expressed as mean ± SD, unless otherwise specified. Metavir fibrosis staging was used either as a categorical variable or as a score (continuous variable) since we have shown a perfect linear correlation between Metavir fibrosis stages and fractal dimension of fibrosis which reflects quantitative architecture. For this reason, the results of fibrosis class classification were also evaluated as a score, e.g., F_M3/4 class was noted as 3.5. This score was only used in the reflection evaluation of Metavir staging (see the fourth figure). Multivariate analyses were based on binary logistic regression. The performance of each test was mainly expressed by the accuracy (i.e., true positives and negatives or correct classification). The diagnostic cut-offs used for significant fibrosis were determined by a posteriori maximum Youden index (sensitivity + specificity - 1). Discrepancy between diagnostic means can be evaluated as grade or score. The grade rate shows details, especially the grade of significant discrepancy (≥ 2 F_M stages). The discrepancy score took into account the magnitude of the error. This score was defined as follows: 0 for correct classification, then 1, 2, 3 or 4 as per the misclassification in F_M stages between the liver specimen and the fibrosis class classification by the non-invasive test. For example, a patient with histological F_M4 but classified as F_M0/1 by blood test was scored 3. The mean score permits a comparison between blood tests. A low score means a low discrepancy magnitude. Statistical software programs were SPSS version 17.0 (SPSS Inc., Chicago, IL, USA) and SAS 9.1 (SAS Institute Inc., Cary, NC, USA).

Population #1 was used to compare the accuracy of pathologists with different expertise levels or vs. blood tests. The prevalence of significant fibrosis was 49.3%.

Results are detailed in population #2 since it was the largest (1,056 patients) for blood tests.

Populations #3 and #4 were used to compare elastometry by Fibroscan and blood tests.

In population #2, the fibrosis class classification of FibroMeter^2G (expressed as score) was more closely correlated with F_M score than that of Fibrotest (Figure 4a/b). By ANOVA, the mean F_M score was significantly different as a function of fibrosis class classification of FibroMeter^2G (F = 188, p < 10^-4) and Fibrotest (F = 83, p < 10^-4). However, the post hoc comparison (by weighted Bonferroni test) showed highly significant differences between each pair of fibrosis classes for FibroMeter^2G, whereas this was not observed between several pairs of contiguous classes of Fibrotest (Figure 4a/b).

Results in population #3 were similar to those observed in population #2: significant discrimination between most contiguous fibrosis classes by FibroMeter^2G and any significant discrimination by Fibrotest (Figure 4c/d). Fibroscan classification was poorly discriminating between contiguous classes (Figure 4e).

The fibrosis class classification might offer some degree of imprecision in the classes including at least two F_M stages. Therefore, we evaluated the meaning of test score within the largest class observed, i.e., F_M1/2 class with FibroMeter^3G in population #2 (Figure 5). In this class, FibroMeter^3G score was 0.32 ± 0.11 in F_M1 vs. 0.37 ± 0.12 in F_M2 (p < 10^-3).

In this study, we have shown that the fibrosis class classification of an accurate blood test like FibroMeter^2G provides better accuracy than Metavir staging by local pathologists, which reflects clinical practice. Additionally, its accuracy was not significantly different from that of Metavir staging by a senior expert of the Metavir group. Surprisingly, fibrosis class classification of FibroMeter^3G provided a non-significantly higher accuracy than that of the senior expert of the Metavir group. This can be attributed to the poor inter-observer agreement of liver interpretation for fibrosis staging in clinical practice [9].

These results nonetheless deserve some comments. First, the accuracy of liver biopsy was significantly superior to that of the best performing non-invasive test when the diagnostic target was binary, such as significant fibrosis. In other words, the development of detailed fibrosis class classifications derived from F_M stages compensated for the lesser performance of non-invasive tests in binary diagnostic targets, as observed in the literature and in the present study. Second, fibrosis class classifications of non-invasive tests seem less precise at first glance; we discuss this important characteristic further on. Third, this study underlines the issue of reference, as an expert from the Metavir group underperformed the consensus reading considered as reference in the present study. Thus, who, or what, should be used as a reference? We have already observed that a consensus reading improved reproducibility and thus could be considered as a reference [9]. However, we do not know if a panel reading would be a more reliable reference. Liver biopsy does have innate limits, such as sampling error and sample size effect, which surpass those of liver interpretation. Indeed, two studies have recently shown that blood tests for liver fibrosis were better prognosis predictors than histological staging [17, 18].

Liver biopsy was used as the best standard [19]. Despite its limits, it can be considered as a good reference for the comparison between non-invasive tests since there are no data to consider that the biopsy error was not systematic (i.e., different between tests). In other words, the accuracy of non-invasive tests is probably underestimated but not their comparison. The results of the different populations are summarized in table 5. The accuracies of fibrosis class classifications were different among non-invasive tests in the present study in the following order: FibroMeter^3G > FibroMeter^2G > Fibroscan > Fibrotest. It should be underlined that these differences were observed in several independent populations. In addition, from one study to another, the rank of accuracy between tests was very reproducible. Thus, the present results are robust. It should also be noted that the authors of a recent study using a quite different methodology in a small series (four patients) observed an accuracy of less than 25% with the fibrosis stage classification of Fibrotest [20]. How thus can one explain this apparent discrepancy between the close accuracies of non-invasive tests for the usual binary diagnostic targets such as significant fibrosis, and the dissimilar accuracies in their fibrosis class classifications? First, a single binary diagnostic target necessarily (mathematically) includes fewer sources of errors than a multiple-stage classification. Second, the statistical methods used to develop the fibrosis class classifications have to be considered. We developed a new statistical method for the development of a fibrosis class classification [4]. Thus, we obtained a fibrosis class classification with FibroMeter^2G that included 6 classes, each one comprising only one or two Metavir fibrosis stage(s). It should be noted that the fibrosis class classifications of Fibrotest or Fibroscan have been reported but the statistical methodology used to establish them was not described [5, 6], nor their accuracy. The method used for three stage classification of Fibroscan accumulates the misclassification rates of each diagnostic cut-off. We used the cut-offs of Stebbing et al since their study was a large recent meta-analysis restricted to HCV. The method of fibrosis class classification that we developed for FibroMeter^2G [4] was validated in the present study by the reproducible accuracy measured in several independent large populations. Thus, before using a non-invasive test in clinical practice, it seems important to verify the statistical methodology behind the construct and its accuracy.

The present results indicate that the FibroMeter classification is robust, as its precision was expanded from 2 for significant fibrosis to 6 or 7 fibrosis classes at the expense of only a 4% relative decrease in FibroMeter^2G accuracy or a 12% relative increase in FibroMeter^3G accuracy (87% in the largest series) [8]. It should be noted that the accuracy/precision ratio was optimized only for FibroMeter^3G [8] but this optimization could also be applied to FibroMeter^2G. This contrasts with Fibrotest, which displayed a 49% relative decrease in accuracy in the largest series between the binary diagnosis and its 8-class fibrosis classification [8]. In addition, the FibroMeter^2G fibrosis class classification was more discriminant than those of Fibrotest or Fibroscan in distinguishing fibrosis classes, especially two successive classes (Figure 4). It has been suggested that the maximal theoretical accuracy may be around 90%, considering the limits of liver biopsy as a reference [21].

The discrepancy level between fibrosis class classifications of non-invasive tests and Metavir stages was reflected by the discrepancy score and the proportion of significant discrepancy (≥ 2 F_M), which markedly varied among tests in the present study. FibroMeter^2G and even FibroMeter^3G provided a significantly lower discrepancy score than Fibrotest or Fibroscan in all study populations.

The accuracy (correct classification in the whole population) of binary diagnosis was superior or equal to that of fibrosis class classification except for FibroMeter^3G. However, the level of classification precision (less fibrosis stages per class) also has to be examined. When the ratio between accuracy and precision is considered, fibrosis class classification seems to provide the best performance. Finally, the fibrosis class classification of FibroMeter^2G had a significantly higher correct classification (qualitative accuracy descriptor) and a significantly lower discrepancy level (quantitative accuracy descriptor better reflecting disagreement than the former) compared to local pathologists. In addition, FibroMeter^3G compared favourably with expert pathologist for those characteristics. This better accuracy for the fibrosis class classification of FibroMeters as compared to liver biopsy would seem to provide a strong argument for their use in clinical practice despite their lesser precision. In other words, FibroMeters had fewer errors than liver biopsy interpretation in clinical practice. Figure 6 also shows that a blood test has a robust diagnostic reproducibility in clinical practice, compared to other diagnostic means. However, this issue of precision can be refined.

Based on F_M stages, fibrosis class classifications provide multiple classes of F_M stages according to blood test values [4]. Thus, FibroMeter^2G fibrosis class classification provided the following new classes: F_M0/1, F_M1, F_M1/2, F_M2/3, F_M3/4 and F_M4. These correspond to the following FibroMeter fibrosis stages expressed in single Metavir score: F_M0.5, F_M1, F_M1.5, F_M2.5, F_M3.5, and F_M4. They can furthermore be translated into the following new FibroMeter^2G fibrosis (F_FM) stages: F_FM0, F_FM1, F_FM2, F_FM3, F_FM4 and F_FM5. This last classification assumes that there is less error with non-invasive tests than with liver biopsy, as suggested by several studies [22, 23]. Therefore, the interest of these new classifications, based on "blood" fibrosis stages, has to be tested independently of their native histological reference by using clinical events as an endpoint. This could be accomplished through a prognostic study as previously done for blood tests used as scores [17, 18] from which classifications are derived. Finally, it should be noted that within the largest FibroMeter^3G fibrosis class, the score progression of blood test well reflected the histological progression (Figure 5).

The prevalence of significant fibrosis in the four populations was close to that (48%) of a reference population of 33,121 patients with HCV and liver biopsy [24]. The studies including Fibroscan were not based on an intention-to-diagnose analysis since unsuccessful measurements were not included. This would decrease the accuracy by about 5% as already shown in another study [25] but not modify the hierarchy of tests regarding accuracy. It should be underlined that liver biopsy has other indications than liver fibrosis.