Analytic performance studies and clinical reproducibility of a real-time PCRassay for the detection of epidermal growth factor receptor gene mutations informalin-fixed paraffin-embedded tissue specimens of non-small cell lungcancer

FFPET specimens of NSCLC tumors were obtained from US and European commercialvendors: Analytical Biological Services Inc. (Wilmington, DE, USA), Asterand,Inc. (Detroit, MI, USA), BioServe (Beltsville, MD, USA), Conversant (Huntsville,AL, USA), Cureline Inc. (South San Francisco, CA, USA), Cytomyx (Lexington, MA,USA), Discovery Life Sciences, Inc. (Los Osos, CA, USA), ILSBio, LLC(Chestertown, MD, USA), Indivumed (Hamburg, Germany), OriGene (Rockville, MD,USA), and ProteoGenex (Culver City, CA, USA). In addition, FFPET specimens ofNSCLC tumors were provided by Astellas Pharma US, Inc. (Deerfield, IL, USA). Allspecimens were aged between 3 and 10 years.

Human epidermal growth factor receptor (HER) plasmids: HER2, HER3, and HER4 werepurchased from Integrated DNA Technologies (San Diego, CA, USA).

K562, human genomic DNA, used as wild-type sequence, was obtained from the humanlymphoma cell line K562 (Promega, Madison WI; part number DD201X).

Methods described below were not used in the diagnosis or treatment of anypatients. Patient consent forms were obtained through the commercial vendor. RMSand the principal investigators from the external reproducibility study abidedby the International Conference on Harmonisation Good Clinical PracticeGuidelines and regulations of the US Food and Drug Administration (FDA) in theconduct of this study. Before the start of the study, the protocol and otherdocuments necessary for participating sites to perform the study was submittedto an independent Institutional Review Board (IRB) in accordance with FDA andlocal legal requirements. IRB approval was obtained at each site.

The cobas EGFR Mutation Test (“cobas EGFR test”, Roche MolecularSystems, Inc, Branchburg, NJ, USA) is a CE-IVD marked multiplex assay that usesallele-specific polymerase chain reaction (AS-PCR) to detect 41 mutations inexons 18, 19, 20, and 21 in FFPET specimens of human NSCLC. The test consists oftwo major steps: (1) a manual DNA isolation step, and (2) PCR amplification anddetection of target DNA using complementary primer pairs and oligonucleotideprobes labeled with fluorescent dyes (Figure 1). Thetest is designed to detect G719X (G719A, G719C, and G719S) in exon 18; deletionsand complex mutations in exon 19; S768I, T790M, and insertions in exon 20; andL858R in exon 21. The specific mutations detected by the assay are detailed inTable 1. A mutant control and a negative controlare included in each run to confirm the validity of the run. The test uses 150ng total input DNA, an amount which can typically be extracted from a single 5μm section of an FFPET specimen using the cobas DNA Sample Preparation Kit(Roche Molecular Systems, Inc, Branchburg, NJ, USA), following the standardpackage insert protocol [13]. Macrodissection of the tissue section is only required if theestimated tumor content is < 10% by pathological assessment.Mutation analysis is performed through real-time PCR analysis using the cobas4800 System (version 2.0), and the analysis of raw data and reporting of resultsare fully automated. The DNA isolation, amplification and detection, and resultreporting can be performed in less than 8 hours. Testing for this study wasconducted with cobas 4800 System Software (version 2.0.0.1028). Analysis wasperformed with the EGFR Analysis Package Software V.1.0

DNA from FFPET specimens using the same extraction method as described for thecobas EGFR test were extracted and amplified at Roche Molecular Systems, andsent out for 2× bidirectional Sanger sequencing (Sanger) by a ClinicalLaboratory Improvement Amendments (CLIA)-certified laboratory (SeqWright,Houston, TX, USA) using a validated protocol.

NSCLC FFPET-derived DNA blends and specimens that gave discordant cobas EGFR testand Sanger test results as well as a randomly selected subset of concordantspecimens were tested using a quantitative massively parallel pyrosequencingmethod (“MPP”, 454 GS Titanium, 454 Life Sciences, Branford, CT,USA) [14]. DNA was extracted and amplified at Roche Molecular Systems prior tobeing sent to a CLIA-certified laboratory (SeqWright, Houston, TX, USA) to besequenced using a validated protocol for EGFR mutation detection. The analyticalsensitivity of MPP for EGFR mutations was validated to a limit of detection of1.25%.

Six plasmid constructs containing the most frequently observed mutation for eachmutation group detected by the test were blended with K562 wild-type DNA suchthat each sample contained a ~5% blend of mutant plasmid at the copy numberequivalent of 50 ng/PCR. Serial dilutions of each specimen were prepared to makepanels with five members. DNA samples were diluted while leaving the percentmutation constant. An additional panel member containing 100% wild-type DNA wasincluded to each panel. Each of the six levels of the six plasmid DNA blendspecimens was tested with each of three unique cobas EGFR test lots. Threedilutions were formulated for each plasmid in each mutation group for each ofthe three reagent kit lots. Eight replicates of each of the three dilutionseries was tested for each of the three kit lots, yielding a total ofseventy-two replicates per panel member.

Analytical performance of the cobas EGFR test was compared against 2×bidirectional Sanger sequencing using 201 FFPET human NSCLC specimens.Correlation between the two methods was assessed by agreement analysis,including positive percent agreement (PPA), negative percent agreement (NPA),and overall percent agreement (OPA). Specimens with invalid results on eithermethod were excluded from the correlation analysis. The cobas EGFR test resultswere considered invalid if any or all of the mutation calls were reported asinvalid. Sanger sequencing results were considered invalid if any or all of thefour exons failed to provide a valid result for a specimen. Sanger sequencingresults were considered invalid if any or all of the four exons failed toprovide a valid result for a specimen. Specimens with discordant cobas EGFR testand Sanger sequencing results and a randomly selected subset of specimens withconcordant results were subjected to MPP.

The internal repeatability of the cobas EGFR test was evaluated using six NSCLCFFPET specimens: two EGFR wild-type and four EGFR-mutationpositive specimens (one exon 19 deletion, one with G719X and S768I mutations,one with L858R and T790M mutations, and one with exon 20 insertion mutation).Testing was performed in duplicate by two operators, using two different reagentlots and two cobas z 480 analyzers over 4 days. Each operator performed one runper reagent lot per day for 4 days, giving a total of 16 runs and 32 replicateresults for each of the six specimens.

The external reproducibility of the cobas EGFR test across three clinicallaboratories using three reagent lots and two operators per site was evaluatedusing a 13-member panel of NSCLC specimens containing five different deletionsin exon 19 and exon 21 L858R in the EGFR gene (Table 2). Operators performed blinded runs (two replicates of each panelmember/run), on five nonconsecutive days, using a single instrument per site. Atotal of 180 DNA replicate specimens were prepared for each panel member, andeach of the three sites performed a total of 780 tests. All statistical analyseswere performed using SAS®/STAT® software. The sample identificationnumbers of panel members were randomized using PROC PLAN in SAS, and the orderof panel members within each run was randomized. Only valid tests from validruns were included in the statistical analyses.

The mutation status and percent mutant alleles of the specimens chosen for therepeatability and reproducibility studies was determined using Sanger sequencingand MPP, respectively, and tumor content was estimated for each specimen byassessment from an external pathologist. DNA was extracted and blended to createsamples containing levels of EGFR mutation above the limit of detection for thecobas platform. The percentage of mutant alleles in the blended samples wasverified by MPP.

The effects on the performance of the cobas EGFR test from two endogenoussubstances (hemoglobin and triglycerides) and nine therapeutic drugs that may bepresent in human NSCLC specimens (albuterol, ipratropium, fluticasone,ceftazidime, imipenem, piperacillin-tazobactam, cilastatin sodium, povidoneiodide, and lidocaine) were investigated with 10 NSCLC FFPET specimens.Specimens were selected for mutation status based on Sanger and/or MPP. Fivespecimens were EGFR mutation-positive and five were wild type.Specimens were tested in the absence and presence of each potential interferent.Each potential interferent was spiked during the lysis step. Hemoglobin andtriglycerides were added to achieve 1× the upper limit of normalconcentration seen in common pathological conditions (as defined by the Clinicaland Laboratory Standards Institute [CLSI] EP7-A2 Guideline; 2 g/L hemoglobin and37 mM triglycerides) [15]. The therapeutic drugs were added to achieve a final concentration of3× the maximal plasma concentration (as defined by the CLSI EP7-A2Guideline) [15], if known. Povidone iodide was tested as a 10% weight by volumesolution; lidocaine was tested at a concentration of 12 μg/mL, asrecommended by the CLSI EP7-A2 Guideline [15].

The impact of tissue necrosis on the cobas EGFR test detection of mutations wasevaluated. Twenty NSCLC FFPET specimens were tested in duplicate: ten specimenscovering a range of percent mutation from the exon 19 deletion, S768I, L858R,G719X, and exon 20 insertion mutation groups, and ten wild-type specimens.Percent necrosis, as assessed by a pathologist, varied from 0% to 60% for mutantspecimens and from 5% to 85% for wild-type specimens.

To confirm that other gene sequences homologous to the targeted EGFRexons do not interfere with the performance of the cobas EGFR test, potentialcross-reactivity was assessed for three members of the ErbB family of receptortyrosine kinases (HER2, HER3, and HER4). The homologous sequences in HER2, HER3,and HER4 corresponding to the probe-targeted portions of exons 18, 19, 20, and21 in the EGFR gene were individually cloned into 12 plasmids (fourexon regions per HER gene) and evaluated with the cobas EGFR test. Wealso sought to determine if the assay, which is designed to detect 29 deletionsin exon 19 would also detect the rare exon 19 L747S point mutation, using aplasmid containing this mutation. Ten NSCLC FFPET specimens (four with EGFRmutations, six wild type) were evaluated in the presence (spiked to aconcentration of 15,850 copies/PCR well, the equivalent of 50 ng of genomic DNA)and absence of each of the HER plasmids as well as the plasmid containing theL747S mutation. The plasmids were spiked into individual replicates of each ofthe ten specimens after extraction; one replicate of each of the 10 specimenswas not spiked with plasmid and was used as the control.

The analytical sensitivity of the cobas EGFR test for exon 19 deletion, L858R,S768I, T790M, G719X, and exon 20 insertion mutations was assessed using NSCLCFFPET-derived DNA blends and six plasmid DNA blends. For the FFPET-derived DNAblends the lowest percent mutation level that was associated with ≥95% hitrate with 50 ng/PCR reaction ranged from 1.3% to 5.6% (Table 3). For the plasmid blends, the amount of DNA in 5% copy equivalentto achieve ≥ 95% mutation detected rate ranged from 0.78 and3.13 ng/PCR reaction (Table 4). Together, the datashow that the cobas EGFR test can detect the predominant mutation for each ofthe six mutation groups when it is present as 5% mutant alleles.

Of the 201 specimens evaluated in the methods correlation between the cobas EGFRtest and Sanger sequencing, 49 specimens gave invalid test results for one orboth methods produced an invalid result. Forty-eight specimens were invalid bySanger sequencing (23.8%). Six specimens (3.0%) were invalid by cobas EGFR testusing reagent lot 1 (5/6 of these specimens were also invalid by Sanger), andfive specimens (2.5%) were invalid using reagent lot 2 (4/5 specimens were alsoinvalid for Sanger).

The comparison of the remaining 152 valid results is shown in Table 5. The OPA between both cobas EGFR test lots and Sangersequencing was 96.7%, with five discordant specimens for each lot. All specimensyielding discordant resultants with either reagent lot were further analyzed byMPP. Discordant analysis results are listed in Table 6. Sanger sequencing detected two mutation calls (one G719A, oneexon 19 deletion) that were not confirmed by the cobas EGFR test or MPP. Twospecimens designated “mutation not detected” by Sanger were detectedby MPP (exon 19 deletion, exon 20 insertion). Both lots of the cobas EGFR testcalled one specimen “mutation not detected” that was called as G719Sby MPP at 1.1% mutation, which is below the 5% limit of detection of the cobasEGFR test. One specimen was detected as an exon 19 deletion by cobas EGFR lot 2,but not detected for both Sanger and cobas EGFR lot 1. This specimen wasdetected as an exon 19 deletion at 3% mutation by MPP, which is below the limitof detection of the cobas EGFR test. Lastly, cobas EGFR test lot 1 detected onespecimen with an exon 20 insertion. This specimen was called “mutation notdetected” by Sanger sequencing, cobas EGFR test lot 2, and MPP.

All runs from the internal repeatability analysis were valid across allspecimens, reagent lots, operators, and instruments combined. A single replicateof one specimen gave an invalid result. The specimen was repeated and the validresult replaced the invalid result, which was excluded from data analysis.Initially six (6) false calls out of 192 specimens were observed generating atotal percent accuracy of 96.9%. Two of the results were resolved to confirm theobserved result by the cobas EGFR test. Three of the false calls were confirmedby MPP; the L858R false call was not confirmed by MPP. With two of the six falsecalls resolved the assay delivered 188 correct calls out of 192 specimenstested, or an accuracy of 97.9%.

In the external reproducibility study, a total of 2,340 tests were performed onthe 13 panel members in 90 valid runs (see Table 2for list of panel member. No invalid results were obtained. No false positiveresults were observed, as all 180 replicates of wild-type specimens (95% CI[98–100%]) gave a Mutation Not Detected result. For the exon 19 and exon21 panel members with 5% mutation, one panel member (EX19_2240_2257del18) had ahit rate below 95% (62.8%, -95% CI [55.3–69.9%]),This may have been due topoor DNA quality in the tumor block used. Although this panel member appeared tohave a lower than 95% hit rate, the Ctr SD and CV(%) for this panel member werewithin the range of the remaining panel members. For all exon 19 and exon 21panel members with ≤10% mutation had 99.4% (95% CI [96.9–100])agreement. Overall the external reproducibility study showed little variation inthe cobas EGFR test performance at multiple clinical sites (Table 7).

No interference was observed for hemoglobin and triglycerides at CLSI-recommendedtest concentrations of 2 g/L and 37 mM for any of the 10 FFPET specimens. Nointerference by therapeutic drugs was observed on the performance of the cobasEGFR test.

No interference from necrotic tissue was observed when evaluating the performanceof the cobas EGFR test. Results for all specimens were concordant with Sangersequencing and MPP results. Thus, levels of necrosis up to 85% did not affecttest performance.

Results for the ten FFPET specimens tested under the 13 conditions using thecobas EGFR test matched the expected results for HER2/3/4 cross-reactivity. Onespecimen that was spiked with the HER4 exon 21 analog plasmid initially produceda result of “Mutation Not Detected”, but yielded the correct callupon retesting. The plasmid with the exon 19 L747S mutation yielded an exon 19deletion call in all specimens that did not already contain an exon 19 deletion,confirming cross-reactivity between the L747S mutation and the cobas EGFR test.The BLAST (Basic Local Alignment Search Tool) results demonstrated that theprimers and probes in the cobas EGFR test are unlikely to cross-hybridize withsequences other than the target sequence. Analogous sequences to the targetedEGFR exons from the HER2, HER3, and HER4 genes did not interfere with theperformance of the cobas EGFR test.

Results are presented in Additional file 1: Table S1.All of the assessed less common mutations except one (exon 19 deletion mutation2236_2248 > AGAC) were detected at a similar DNA input level asthat for the corresponding predominant mutation. The exon 19 deletion mutation2236_2248 > AGAC was not consistently detected at any DNA inputlevel.

Neither Haemophilus influenzae nor Streptococcus pneumoniae hadany effect on the performance of the cobas EGFR test (data not shown).