A comparison of Direct sequencing, Pyrosequencing, High resolution melting analysis, TheraScreen DxS, and the K-ras StripAssay for detecting KRAS mutations in non small cell lung carcinomas

At present, identifying targeted anticancer treatment suitable for a given patient requires the availability of accurate diagnostics. Diagnostic techniques therefore have a significant impact on patients’ survival and quality of life[1]. In recent years, it has become apparent that certain types of tumors undergo mutations that either originate from the aberrant physiology of the tumor or are induced/selected by mutagenic cancer therapies[2‐4]. Failure to detect mutations in important regulatory genes in tumor specimens may have serious consequences for the patients, because these alterations can significantly reduce the effectiveness of certain biological and cytotoxic therapies. Mutations in the KRAS oncogene are often found in human cancers. They are most common in pancreatic cancer, which can exhibit mutation rates of 80 - 90%. KRAS mutations are also observed in 40 – 50% of colorectal cancers and 10 - 30% of Non-Small Cell Lung Cancers (NSCLCs).

Recent studies have shown that some anticancer drugs are only effective against tumors in which the KRAS signaling pathway has not undergone oncogenic activation. These include the small-molecule epidermal growth factor receptor inhibitors erlotinib (Tarceva®) and gefitinib (Iressa®), which are used to treat NSCLC patients, and monoclonal antibody therapies such as cetuximab (Erbitux®) and panitumumab (Vectibix®), which are primarily used in the treatment of metastatic colorectal cancers (mCRC)[5‐7]. According to the U.S. National Comprehensive Cancer Network (NCCN) guidelines from November 2008 (http://​www.​nccn.​org/​about/​news/​newsinfo.​asp?​NewsID=​194) and recommendations of the American Society of Clinical Oncology (ASCO)[8], screening of the status of the KRAS gene is mandatory when deciding whether or not a patient with colorectal cancer should receive anti-EGFR drugs. Similar rules are being considered for NSCLC where KRAS mutations have prognostic value for progressive disease in adenocarcinoma[9, 10].

There are multiple methods for detecting KRAS mutations in patient tissues, with varying analytical parameters. Individual methods need to be evaluated in terms of their sensitivity, specificity, and cost per analysis before they can be considered to meet acceptable gold standards in clinical practice. A standardized European quality assurance program for tests to detect mutations in KRAS was proposed at the Third International Congress of Pathology, held by the European Society of Pathology (ESP) in Barcelona in May 2008. This program is focused on achieving optimal accuracy and proficiency across the European Union[11]. However, there are many methods in current use, some of which are only employed by individual laboratories and are not commercially available. These typically include sequencing assays[12] and gel-based DNA conformation assays[13, 14]. Some of the commercial assays for detecting mutations in the KRAS gene have not yet been validated for clinical use (i.e.: Allele-specific oligonucleotide hybridization - Invigene®, KRAS mutation test kit - EntroGen®). At the time of writing, only the TheraScreen® kit sold by QiaGen, the KRAS LightMix® kit sold by TIB MolBiol, and the K-ras StripAssay® sold by ViennaLab had received the Conformité Européenne (CE) mark certifying them as being suitable for diagnostic use in the clinic under the terms of the European IVD Directive 98/79/EC.

In order to assess the specificity, sensitivity, cost, and working time of five frequently used methods for detecting mutations in KRAS, we performed parallel tests using DNA extracted from 131 frozen NSCLC tissue samples. The methods examined were Sanger cycle sequencing, Pyrosequencing, High-resolution melting analysis (HRM), and the CE-marked TheraScreen DxS and K-ras StripAssay kits. Our data demonstrate that there are important differences between these methods, which should be considered in routine clinical testing for KRAS mutations.

Five genotyping methods for determining the status of mutation of KRAS were assessed using frozen tissue from primary NSCLC tumor specimens. 131 DNA samples were analyzed with 4 of the methods (Direct sequencing, K-ras StripAssay, TheraScreen DxS, and Pyrosequencing), and 116 of these were also analyzed using the High resolution melting (HRM) technique. In the absence of a gold standard, we adopted a consensus method for assigning each sample’s mutation status. The results obtained and methodology used are shown in Table1.

As expected, the percentage of the DNA samples in which mutations were detected varied (from 20% to 5%) depending on the method of detection used. The Kras-StripAssay had the highest likelihood of referring a mutation in the KRAS locus, followed by TheraScreen DxS, HRM, Pyrosequencing, and Direct sequencing (Table2).

However, on the basis of our evaluation criteria (Table1), the most sensitive tool was the TheraScreen DxS kit (95%), followed by the K-ras StripAssay (90%), HRM (70%), Pyrosequencing (48%), and Sequencing (29%). The most specific tools were the TheraScreen DxS kit, Sequencing, and Pyrosequencing (100%), followed by HRM (98%) and the K-ras StripAssay (95%) (Table3).

The number of false positives and false negatives obtained with each method would change if one were to change the interpretation criteria. For example, if the K-ras StripAssay is taken to be the gold standard, then the false positives detected by this method would become false negatives detected by the other methods. To eliminate this potential ambiguity, we performed more tests to assess and compare the sensitivity thresholds of the tested methods. We used three ATCC cell lines whose KRAS mutation statuses are known and recorded in the COSMIC database: A549 (p.Gly12Ser), NCI-H620 (p.Gly12Val), and NCI-H2009 (p.Gly12Ala). We extracted sample DNA from the cell lines, measured its concentration by spectrophotometry, and then made dilution series of the DNA from the KRAS mutant cell lines in DNA from the NCI-H1975 KRAS wild-type cell line such that the mutant DNA comprised 25%, 20%, 15%, 10%, 5%, 1%, 0.5%, 0.25%, or 0.125% of the total KRAS DNA (Figure6).

We have examined the ability of five different methods to detect mutations in the KRAS gene in 131 DNA samples. KRAS mutations were detected in 21 samples (16.0%), 107 samples were found to contain wild-type DNA (81.7%), and three yielded inconclusive results (2.2%) (Table1). Of the 21 samples in which mutation was detected by one or more methods, there were only four for which all five yielded a positive result (19.0%). Of the 95 wild-type samples analyzed by all five methods, concordance was observed in 87 (91.6%); overall, the five methods were in agreement with one-another for 78% of the samples examined. Excluding HRM, the four remaining analytical methods all generated positive results for 4 out of 21 samples found to be positive by one or more method (4.4%) and all generated negative results for 101 of 107 samples found to be negative by one or more method (94.4%), giving an overall agreement of 82%.

Our findings concerning the ability of these methods to detect mutations in KRAS are similar to those of Whitehall et al. (2009), who compared Dideoxy sequencing, HRM, the TIB Molbiol kit (Berlin, Germany), and the TheraScreen DxS (Manchester, UK) kit using DNA isolated from frozen colorectal cancer tissues. In their study, all five methods were found to be in concordance with regard to the KRAS mutation status of 66 of the 80 samples tested (83% agreement)[20].

Both our results and those obtained by Whitehall[20] show that a significant number of samples from colorectal tumor and NSCLC contain mixtures of KRAS wild-type and KRAS mutant cells, and that in many cases the percentage of mutant cells is below the threshold that can be detected by direct sequencing. This inherent heterogeneity of bioptic tumor tissues is an universal problem, albeit one that can be partially addressed by concentrating the tumor cells (e.g. by laser capture microdissection) before extracting their DNA. However, the fact that even a pure sample of tumor cells may contain large quantities of wild-type KRAS further complicates the selective identification of mutations in this gene. Consequently, it is desirable that methods for detecting KRAS mutations should be highly sensitive, and this point should be borne in mind when selecting a proper diagnostic method. Our study identified the TheraScreen DxS kit as having the best ability to detect KRAS mutations in clinical samples, followed by the K-ras StripAssay (Table4).

Our results also indicate that direct sequencing is only of limited utility when trying to detect mutations in the KRAS gene in cancer tissues, since this method only detected KRAS mutations in 6 of the 131 DNA samples tested, even though 21 were found to contain mutations by other methods. Though direct sequencing is still being advocated as KRAS genotyping method of choice[21], it missed 72% of all mutations in our cohort. Obviously, the sensitivity of the sequencing methods could be further improved by using laser microdissection[22], preferential preamplification[23], inclusion of both primary and metastatic tissues in the analysis, or by using clamping to suppress PCR amplification of the wild-type gene[24]. However, we agree with Pinto et al. that Sanger sequencing (without the first steps of COLD-PCR)[25] is currently outperformed by more sensitive techniques[26].

Pyrosequencing is easily capable of detecting PCR fragments that are 25–50 bp in length while longer fragments may pose a problem. However, this is not the case of detecting mutations in KRAS, because the most frequent mutations in this gene are adjacent, occurring in codons 12 and 13. It may even be advantageous to use short fragments when diagnosing mutations because DNA may be fragmented during the processing of clinical tissue samples. In accordance with results of others[27, 28], Pyrosequencing outperformed conventional sequencing for detecting KRAS mutations in samples with levels of mutant cells ranging from 5 to 25% (Table4) while quantification of mutated portion of DNA was not possible. This is probably due to preferential amplification of the mutated samples by the primers designed for the particular Biotage kit used. This shortcoming could be obviated by a better primer design or other modification of the kit and/or improvements in the interpretation algorithm[29, 30]. Promisingly, a massively parallel pyrosequencing system using nanoliter reaction volumes has yielded satisfying results in an interlaboratory comparison[28]. While this probably represents the future of testing in predictive oncology, such systems are prohibitively costly for most laboratories at the present.

HRM proved to be the least expensive and the most rapid method, as it requires only standard real-time PCR reagents and a slightly prolonged PCR protocol. Despite the optimistic references from other laboratories[31], the analysis of the melting profiles in our hands remains less reliable than other methods, and even repeated testing of our reference DNA did not always yield consistent results. Because of this, the typing of two samples by this method was inconclusive. We may speculate with Do[32] that treatment of DNA with uracil glycosylase or special step of DNA cleaning would help standardize the method and better its analytical parameters. Interestingly, HRM analysis identified mutations in the KRAS locus of two DNA samples (samples 31 and 32) for which none of the other methods detected any mutation (Table1). In keeping with the findings of other authors[33], we interpret these results as reflecting a tendency of HRM to generate false positives. However, it is possible that they reflect rare mutations outside codon 12 and 13 that destabilize heteroduplex DNA even in the presence of an excess of wild-type DNA. Although cost and time efficiency are important factors in clinical diagnosis, the reproducibility of the HRM method will need to be improved before it can be considered viable. This could potentially be done by changing primers[34], adding melting standards[35], spiking with oligonucleotides[36], or combination with SNaPshot[37].

The StripAssay was the most analytically sensitive test (Table2) of those we examined. On the basis of the results obtained with this method in the series of tests conducted with dilution series of mutant KRAS DNA (Figure6), one could even argue that samples 24 to 30 should be reassigned as mutants (Table2), thereby changing the false positive rate for the K-ras StripAssay to 0/128 and the false negative rate for TheraScreen DxS to 7/128. However, the interpretation of StripAssay results can be quite problematic for samples whose mutant DNA content is below 1% (see the result obtained with a mutant minority of 0.5% NCI-H620 in Figure6). Insofar, it was not tested in clinical studies what is a significance of fraction of mutated cells below 1%, regardless of the typing method used.

During time of submitting this article, company’s software was upgraded to follow more precisely the requirements of ISO15189 norm (scanner calibration standard was added and manual baseline correction feature was removed). It remains to be seen if such changes bring any improvement to diagnostic accuracy.

Of the methods examined in this study, the TheraScreen DxS kit was the fastest method and exhibited the highest sensitivity and specificity. However, it was also the most expensive method that is not free of false reactions. Specifically, the kit failed to detect the p.Gly13Cys mutation in sample 2 because it is not designed to detect this mutation. Although the frequency of the mutations that are not covered by the TheraScreen DxS test is very low and clinically not highly relevant, this nevertheless constitutes an inherent limitation of the kit. In addition, the precise allelic mutation detected by this kit in samples 3, 16, and 18 differed from the consensus result. While this could potentially be due to stochastic variation in the early events of PCR priming, there is no firm evidence to support this hypothesis. Although discrepancies in the precise identity of the mutation are not yet clinically relevant, and these results were not scored as errors in this study, this finding warrants caution when using the ARMS Scorpions assay in different diagnostic setups, where the type of mutation is important (e.g. when looking at the T790M and S768I activating mutations in EGFR genotyping). As discussed above, samples 24 to 30 gave positive results in the StripAssay but were negative when analyzed with the TheraScreen DxS kit, and they seem to have a mutant population in the clinically “grey area,” having less than 1% of the cells in the sample containing KRAS mutation. Ideally, their status should have been resolved by PCR amplicon cloning, followed by sequencing of the clones, digital PCR, or ultradeep sequencing. However, this approach is not practical for routine work and we did not have sufficient DNA to perform this experiment. Moreover, the low frequency of KRAS mutations in patient tumors have unknown clinical relevance, since all drug registration trials were performed using 1% of mutant KRAS cells as a low detection limit of the method.

The properties of the different methods examined in this work are summarized in Table5.

We agree with Tsiatis et al.[27] that for research purposes more than one genotyping platform is necessary to reveal double mutations and to provide complementary data. In clinical settings, the most readily accessible NSCLC sample type is needle or brush biopsy, which is examined cytologically while resected, or biopsied tumors processed by formaldehyde fixation and paraffin embedding (FFPE). Proportion of FFPE samples from all samples usually reflects the best local practice and experience. Unfortunately, the FFPE process alters significantly the quality of DNA, and in many cases the DNA isolation from cytology smears yields higher quality albeit lower quantity of DNA.Very low quantity of available DNA isolated from cytological preparations was a major limiting factor in our comparative study, which we tried to overcome using frozen tissue from biobank, since it provides both high quality and quantity of DNA. Moreover, due to recent biobanking initiatives[38], we are more frequently facing situations, where the tumor molecular diagnostics is performed from frozen tissues. Of the 11 FFPE samples genotyped using both the StripAssay and TheraScreen, 5 samples could not be typed by at least one method, 2 samples were wildtype by both methods, 3 samples were mutant by both methods, and one sample was p.Gly12Asp by TheraScreen and wildtype by StripAssay. From one point of view, it could be argued that our genotyping results obtained using frozen samples are transferable to genotyping of FFPE samples because the mechanisms by which the methods work are not dependent on the nature of the input sample. On the other hand, it should be noted that improperly-performed paraffin embedding damages DNA and can favor methods that are more robust to variation in the amount and quality of the starting material (this would arguably disfavor TheraScreen because it requires eight PCR reactions whereas the other methods require only one equivalent reaction). It has been suggested that the issue of limited material for testing can be largely circumvented by using whole genome amplification techniques[39, 40], although the potentially biasing impact of the genome amplification techniques on low frequency somatic mutation genotyping is still not fully addressed. However, we suppose that our tests of kit performance on frozen tissue samples provide useful insights into their general utility and will be valuable for orchestrating genotyping efforts across molecular pathology laboratories.

The performance of five methods (Direct sequencing, Pyrosequencing, High resolution melting analysis, the TheraScreen DxS kit, and the K-ras StripAssay) for detecting mutations in the KRAS gene was compared using DNA extracted from 131 frozen NSCLC samples. The TheraScreen DxS kit was found to be the most effective, followed by the StripAssay kit. However, because of the heterogeneity of typical cancer tissue samples and the differences in the two methods’ mechanisms of action, there are still unsatisfactory numbers of discrepancies between these two ‘best’ methods, which failed to agree on 8 of the 131 specimens examined in this work. Nevertheless, our findings should facilitate the rational selection of methods for detecting mutations at the KRAS locus using heterogeneous clinical samples obtained from biopsies of cancer patients.