Technical validation of an RT-qPCR in vitro diagnostic test system for the determination of breast cancer molecular subtypes by quantification of ERBB2, ESR1, PGR and MKI67 mRNA levels from formalin-fixed paraffin-embedded breast tumor specimens

RNA for each sample was extracted from a single non-macrodissected 10 μm-thick FFPE section with the RNXtract kit (BioNTech Diagnostics GmbH). Pathologic review of a representative H&E-stained slide (biopsy or resection specimen) ensured the presence of sufficient tumor tissue with at least 20 % tumor cell content. A description of basic clinical and pathological characteristics of the samples used herein may be found in Additional file 1A. The RNXtract RNA extraction kit employs germanium-coated paramagnetic particle technology in order to segregate RNA from FFPE material with preference over DNA. The procedure begins with paraffin melting and buffer-controlled tissue lysis, followed by proteinase K digestion. Thereafter, lysates are incubated with magnetic beads under optimized buffer conditions which favour binding of RNA molecules. Contaminants are then progressively removed through sequential washing steps, during which paramagnetic particles with attached nucleic acids are kept magnetized, while the supernatant is removed. At the final step, RNA is released from the magnetic beads into 100 μl of elution buffer.

With MammaTyper reverse transcription of RNA and amplification of cDNA take place successively in one reaction mix, which contains all the necessary enzymes and hydrolysis primer/probe sets specific for the target sequences of interest. For the various analytical performance experiments forty cycles of nucleic acid amplification were applied and the quantification cycle (Cq) values of the target and reference genes were estimated as the median of triplicate measurements per RT-qPCR run. Median Cq values of the triplicate measurements (from now on simply Cq) for each of the 4 different genes of interest (GOI) were normalized against the mean expression of the two reference genes (REF) and presented as ΔΔCq values relative to the positive control [27]. The final values were generated by subtracting ΔΔCq from the total number of cycles so that test results are positively correlated, a format that facilitates interpretation for clinical decision making:

The test output comprises the normalized raw data of individual biomarkers and the molecular subtype of breast cancer, according to the algorithm depicted in Table 1. Each sample is assigned into Luminal A-like, Luminal B-like (HER2 positive or HER2 negative), HER2 positive (non-luminal) and Triple negative subtypes, after dichotomizing continuous RNA output values per marker into “Positive” and “Negative” results according to clinically established and qPCR device-specific cut-offs (Additional file 1B). Assay controls are included to ensure that specimens and RT-qPCR runs satisfy pre-specified quality thresholds. These controls comprise two internal reference genes (REF) for assessment of sample validity and normalization (B2M, CALM2) as well as one positive (PC) and one negative (no-template control; NTC) external control for qualification of the RT-qPCR process (Additional file 1C). The aforementioned cut-off values for ERBB2, ESR1 and PGR were established in a cohort of 135 breast cancer patients, based on the highest concordance achieved between relative RNA amplification and corresponding protein expression by high-quality IHC as the gold standard.

Due to the uncertainty of Ki-67 IHC data the MKI67 cut-off of the assay under development (Stratifyer Molecular Pathology GmbH) was set at the 3^rd quartile of the normally distributed MKI67 expression data from 90 FFPE breast cancer tumor samples. These had been previously analysed in the context of a clinical outcome prediction study [28]. The cut-off was then transferred from the assay under development to the MammaTyper IVD by parallel measurement of 135 clinical breast cancer samples and matching of cut-offs on this sample set.

For assessing the sensitivity of extraction, 3 FFPE samples (1 × 10 μm) were spiked with commercial 4 μl unique internal control RNA (Int-RNA) (primer design) at the initial extraction step so as to determine the recovery rate. The obtained Cq values were compared against a standard curve of Int-RNA serially diluted in RNXtract elution buffer. Since the original concentration of the Int-RNA was not known, extraction efficiency was estimated relative to that of the control sample (representing 100 % input). In addition, the extraction efficiency was validated with respect to variations in the range of input material. Two resection specimens and 6 standard core tumor biopsies (10 μm × 1.5 mm × 20 mm) were serially cut and 10 samples were prepared that differed by specimen type (resection versus biopsy), thickness of paraffin curls (5 μm versus 10 μm) and number of input curls (1, 2 or 3). The RNA concentration of the eluates was quantified with calibration to a standard curve as described above for specificity.

The analytical sensitivity of the RT-qPCR was validated under conditions of reduced RNA template and with respect to the test detection capacity according to the standard methods provided in the EP17-A guideline issued by the Clinical and Laboratory Standards Institute (CLSI) [29]. Serially diluted samples from 3 RNA eluates corresponding to different subtypes and RNA eluates from six tumor biopsies were analyzed and the acceptable range of B2M and CALM2 REF genes was used as a quality control of sample validity according to the instructions for use. For further characterizing the analytical performance of MammaTyper the following formal validation metrics were used. The LoB (limit of blank) i.e., the highest measurement that is likely to be observed for a blank sample, was determined separately for each assay mix and for each qPCR system on 20 replicates of NTC samples containing water (120 reactions in total). Assuming a Gaussian distribution of the Cq values, the LoB, was set at a mean Cq value of 40, observed 95 % of times for each assay mix. The LoD (limit of detection), defined as the smallest analyte concentration likely to be consistently distinguished from the LoB and at which detection is feasible with a certain degree of certainty was herein perceived as the amount of total sample RNA (positive for all targets) at which the Cq value was below the LoB with a probability of 95 %. The LoQ (limit of quantification) was here defined as the amount of target RNA for each marker producing a Cq value which was (a) lower or equal to the Cq determined for the LoD, and (b) displayed a distance to the regression curve of the dilution series lower than 0.7 Cqs for each dilution of the clinical sample or the IVT-RNA (in-vitro transcribed RNA). The distance value reflects previously reported PCR replicate variation range of 0.5 to 1.0 Cq [30]. The analytical measurement range for MammaTyper assay was determined as the Cq range over which the MammaTyper shows a linear signal (distance from regression curve ≤0.7 Cq) beginning at the LoD as the lowest input value. For the LightCycler the analytical measurement range is limited by the fix baseline settings (3–15).

A master sample prepared by pooling three eluates from a single patient block (3 FFPE sections per extraction) was measured with spectrophotometry (NanoDrop; Thermo Scientific) and serially diluted in carrier RNA (10 ng/μl). Dilution curves were analyzed with all three assay mixes on both amplification systems. The same procedure was carried out for MammaTyper IVT-RNA. The numbers of molecules corresponding to the highest concentration of the IVT-RNA were calculated online with a web-based molecular weight calculator (http://​www.​currentprotocols​.​com/​WileyCDA/​CurPro3Tool/​toolId-8.​html). The number of molecules in the master sample was extrapolated from the respective linear function of the regression curves of the IVT-RNA dilution series for each marker and reference gene. Concentrations corresponding to Cq values outside the analytical measurement range (as defined by IFU-based instrument settings) were excluded from the analysis.

In the context of the robustness study the extraction and RT-qPCR workflows were varied at various steps as described in detail in Additional file 1D. RNA was extracted from one FFPE clinical sample and was quantified using the standard curve method. The stability of RT-qPCR against pre-specified fluctuations of protocol parameters was determined as the concordance between subtypes generated under the varying conditions and of subtypes assigned under the standard protocol to 4 clinical breast cancer cases.

The following studies were designed in order to verify that variability in pre-analytical processing steps, including different methods for RNA extraction or the use of macro-dissected versus full-face tissue sections, do not substantially interfere with the stability of the RT-qPCR-based MammaTyper breast cancer subtypes. For this purpose RNA was extracted from 8 clinical FFPE samples with RNXtract and two other commercially available kits for extraction of RNA from FFPE tissues. Eluates corresponding to the different extraction methods were analyzed within the same RT-qPCR run.

Pathologically confirmed non-invasive tumor tissue including normal breast lobules or ductal carcinoma in situ (DCIS) is commonly part of full-face tumor sections used for RNA extraction and is generally considered a source of errors in gene-expression quantification by RT-qPCR. To assess the possibility that contamination by non-invasive tumor may affect the MammaTyper results, 9 clinical breast cancer cases were selected in order to represent samples with low tumor cell content (10–50 %) which were then enriched up to 80 % upon careful macrodissection. The differences between macrodissected and non-macrodissected samples were recorded as disagreement affecting the status of individual markers. By intention, all cases included DCIS of variable extent (10-70 %) adjacent to tumor.

For determining the reproducibility of MammaTyper, two studies were performed on the two compatible qPCR platforms, whereby the experiments were designed according to hierarchy of investigated factors (Study 1: Instrument/site – days – runs – replicates within runs; Study 2: Instrument/site – extraction – run (=days) – replicates within runs). The first study was conducted on four LightCycler 480 instrument II (Roche) devices utilizing pooled RNA extracted from breast tumors. The outset of the second study, conducted on three Versant kPCR Cyclers (Siemens) and one LightCycler 480 instrument II (Roche), was FFPE breast tissue sections and the aim was to investigate pre-analytical factors and variations between qPCR instruments from different suppliers. All operators participated in familiarization runs and were blinded to any characteristic of the test samples which might create anticipation for a specific output. Each sample was tested in triplicates during each run along with the specified positive and negative controls. One, 10 μm-thick tissue section was input for RNA extraction irrespective of tumor surface or tumor cellular content. The results of the two studies have been combined and arranged by qPCR system for presentation purposes.

This comparative three-site study was conducted with the MammaTyper by two operators on four LightCycler 480 II (Roche) devices to assess analytical performance as presented in Table 2. The four instruments were in place at the National Centre for Tumor Diseases in Heidelberg, the BioNTech Diagnostics GmbH in Mainz and the University Medical Center of the Johannes Gutenberg University Mainz. Reference RNA (RNA from several cancer cell lines) (Agilent) and RNA pools from seven breast tumor samples were generated from commercially available FFPE specimens for testing at each site. The samples were selected to include the entire range of the clinically anticipated expression levels of the target genes, including values adjacent to the cut-offs. Each sample was processed during six RT-qPCR cycles, run on consecutive days at sites 1 and 2 but not at site 3. The procedure was performed by the first operator at site 1 and by the second operator at the two other sites while at site 2 two different qPCR devices were used.

This comparative three-site study used replicate breast tumor tissue specimens from the same FFPE block for testing with the MammaTyper (Table 2). A 16-member panel of breast cancer samples (15 ductal carcinomas and 1 lobular carcinoma) obtained from commercial vendors except for one clinical sample and comprising all different tumor subtypes were analyzed. All tissue specimens were sectioned at BioNTech Diagnostics GmbH. The first and last section were analysed with MammaTyper at site 1 and showed nearly identical marker expression. After rearranging, sections were shipped to the other testing sites (Stratifyer Molecular Pathology GmbH, Cologne and Institute of Pathology, University of Erlangen). Each tissue sample was processed during three, 4-day cycles, starting with RNA extraction and aliquoting of eluates (day 1) followed by three RT-qPCR runs on consecutive days (day 2–4). The procedure was performed by one operator per site using a single instrument (Versant kPCR Cycler, Siemens), a single lot of RNXtract and a single lot of the MammaTyper Assay. At BioNTech Diagnostics six additional cycles were performed by the same operator, two with different MammaTyper lots, one with an alternative RNXtract lot and three on a LightCycler 480 II (Roche) qPCR device to account for related effects on precision.

Intra-run precision was estimated based on the variance of triplicate measurements. All calculations for the precision studies were carried out based on CLSI guideline EP 05-A3 [25] using a random effects model II ANOVA via PROC mixed in SAS Version 9.2.

Quantification of nucleic acids extracted from 20 FFPE samples showed preferential extraction of RNA with a mean concentration of 232.8 ng/μl (95 % CI: 132.8-332.8 ng/μl). By contrast, DNA levels remained consistently below 15 ng/μl with a mean concentration of 5.5 ng/μl (95 % CI: 3.7-7.3 ng/μl) (Additional file 2A).

For the amplification assay, specificity was determined by comparing the status of individual markers as well as subtypes in 3 pairs of DNase I-treated and non-treated samples. Under these varying conditions no difference was observed between DNA-free samples and samples potentially contaminated by DNA. Subtype and single marker concordance was 100 % and no detectable amplification was noticed for negative controls (Cq = 40) (Additional file 2B).

The recovery rate of the 3 samples spiked with the unique internal control was 73 %, 91 % and 99 %. Further, RNA isolation was optimal for all variations of the input material sensitivity experiment and remained consistently above 10 ng/μl. Accordingly, RNA-rich eluates were obtained even when samples were reduced to single core needle biopsy or single 5 μm-thick full-face slices. Amplification sensitivity was assessed on three serially diluted samples (Triple negative, Luminal B-like (HER2 negative) and HER2 positive (non-luminal)) and on RNA extracted from six standard core needle biopsies. All marker results of the diluted samples were in agreement with the results of the non-diluted specimen up to the limit of sample validity as defined by the pre-specified B2M and CALM2 Cq thresholds. Further, as shown in Fig. 1, some of the individual marker assignments remained stable up to a dilution factor of 256. All six biopsy samples were valid according to assay specifications resulting into efficient subtyping of the respective tumors.

Three 10 μm-thick sections were generated from each one of eight FFPE tissue specimens and manually processed with RNXtract and two additional commercially available kits for extraction of RNA from FFPE tissue in order to estimate the effect of different isolation methods on MammaTyper assay performance. Among the binary results resulting from 96 measurements (valid specimens: 100 %) there were three discrepant marker classifications involving PGR and MKI67 40-ΔΔCq values which were close to the respective cut-offs (Fig. 2).

For estimating the effect of tumor cellularity on the accuracy of MammaTyper output, pairs of dissected and non-dissected samples were compared with respect to relative gene expression and binary marker status. One sample did not satisfy validity criteria and was excluded from further calculations. As shown in Fig. 3, 40-ΔΔCq differences across the tested pairs remained very low for all markers, with an average difference of less than 0.47 Cq. Concordance was 100 % for ERBB2, ESR1, PGR whereas for MKI67, one case containing 10 % tumor cells was negative in the non-macrodissected sample. Macro-dissection of FFPE tissue sections for gene-expression analysis with the MammaTyper assay may be spared for samples containing more than 10 % tumor cells.

The various potential failure modes tested in the robustness study for both the RNXtract and the MammaTyper assays affecting binding and washing steps and PCR parameters did not significantly alter the output. For the extraction experiments this involved adequate yields of RNA at concentrations which did not differ significantly from standard conditions except when the protein digestion step was omitted. For the MammaTyper the method failed upon omission of the reverse transcription step. As expected, the failure was detected by out-of-range median Cq values of the positive control. For the other two failure modes there was perfect agreement with respect to single markers.

All blank measurements showed no amplification signal up to 40 cycles. The LoQ was equal to the LoD for all six assays on IVT-RNA as well as FFPE derived RNA and both platforms Versant kPCR and LightCycler 480 instrument II. Data for the other metrics are summarized in Table 3 for experiments run on the two different qPCR systems for the FFPE RNA master sample and IVT-RNA (positive control) respectively. All assays were linear at least up to 32.33 and 33.56 Cqs for the two qPCR platforms tested. Amplification efficiencies ranged from 99 to 109 %. The lower (Cq) border of the analytical measurement range (AMR) for IVT-RNA on the LightCycler platform was limited by the fixed instrument baseline settings of the test, whereas for the FFPE master-sample the lower (Cq) border of the dilution curve reflected the expression of the target genes which in turn was restricted by the actual concentration of RNA in the master-sample (96.7 ng/μl).

Precision was evaluated simultaneously for RNA extraction and for RT-qPCR with regard to various potential sources of analytical variability such as qPCR instrument, reagent lot/batch and site/operators. One hundred percent of all specimens and measurements yielded valid results according to assay specifications except in study 2 for one sample at site 1, for which extraction was repeated using a reserve FFPE section. As expected, no signals were detected for the negative controls up to a Cq value of 40. The calculated test results of the 16 specimens adequately represent a wide range of clinically encountered 40-ΔΔCq values of the individual genes, both positive and negative calls for individual markers.

Table 4 shows the estimated variance components (expressed as standard deviations) generated by analytical and pre-analytical factors as well as the overall variance, for studies run on LightCycler 480 instrument II platforms (Roche) (inter-run/site, intra-run data generated with 8 RNA-pools on 4 instruments, inter/within section/extraction data generated on 16 samples with 3 cycles as described for the Versant instrument). The assay specific inter-site standard deviation (SD) for the entire set of the test samples was between 0.14 and 0.20 Cqs for Study No1. In this study there were only minor differences in the variance reflecting different components of precision. This indicates that the actual triplicate measurement is the major source of analytical imprecision and further noise is introduced only to a minor degree when experiments are carried out on different days or different instruments. Moreover, variability seemed to be homogenously distributed across the measurements of the four target genes, indicating comparable assay performance.

The estimated parameter-specific and overall variance for studies on Versant qPCR Cycler platforms (Siemens) are depicted in Table 5. The estimates for assay specific inter-site SD for the entire set of the test samples ranged from 0.00 to 0.66 Cqs, whereby a value of 0 indicates that the higher-order variance component is completely covered by the lower order component. A small but noticeable gradual increase in variability was observed while progressing from lower to higher components of precision. The data presented in Tables 4 and 5, collectively reveal a characteristic difference in the noise involving all testing parameters between experiments conducted on different qPCR platforms. Interestingly, biological variability, represented by inter-extraction variance was lower or almost equal to the variability between different aliquots of the same RNA eluate (intra-extraction) for each one of the two instruments (study 2), but it was overall higher for the Versant instrument. Thus, the same sources of analytical variation tested under similar conditions but on different qPCR systems resulted in comparable but not identical variability of the measurements.

The gold standard for assessing concordance was set by the most prevalent binary result across all measurements between sites after applying the respective cut-offs (positive versus negative). The average marker specific between-site and between-instrument concordance was over 97 % (Table 6) (over 94 % for subtypes) and it reflected excellent reproducibility of relative gene expression for the individual targets as shown in Fig. 4, for Study 1.