Research paper
Internal validation of STRmix™ for the interpretation of single source and mixed DNA profiles

https://doi.org/10.1016/j.fsigen.2017.04.004Get rights and content

Abstract

The interpretation of DNA evidence can entail analysis of challenging STR typing results. Genotypes inferred from low quality or quantity specimens, or mixed DNA samples originating from multiple contributors, can result in weak or inconclusive match probabilities when a binary interpretation method and necessary thresholds (such as a stochastic threshold) are employed. Probabilistic genotyping approaches, such as fully continuous methods that incorporate empirically determined biological parameter models, enable usage of more of the profile information and reduce subjectivity in interpretation. As a result, software-based probabilistic analyses tend to produce more consistent and more informative results regarding potential contributors to DNA evidence. Studies to assess and internally validate the probabilistic genotyping software STRmix™ for casework usage at the Federal Bureau of Investigation Laboratory were conducted using lab-specific parameters and more than 300 single-source and mixed contributor profiles. Simulated forensic specimens, including constructed mixtures that included DNA from two to five donors across a broad range of template amounts and contributor proportions, were used to examine the sensitivity and specificity of the system via more than 60,000 tests comparing hundreds of known contributors and non-contributors to the specimens. Conditioned analyses, concurrent interpretation of amplification replicates, and application of an incorrect contributor number were also performed to further investigate software performance and probe the limitations of the system. In addition, the results from manual and probabilistic interpretation of both prepared and evidentiary mixtures were compared.

The findings support that STRmix™ is sufficiently robust for implementation in forensic laboratories, offering numerous advantages over historical methods of DNA profile analysis and greater statistical power for the estimation of evidentiary weight, and can be used reliably in human identification testing. With few exceptions, likelihood ratio results reflected intuitively correct estimates of the weight of the genotype possibilities and known contributor genotypes. This comprehensive evaluation provides a model in accordance with SWGDAM recommendations for internal validation of a probabilistic genotyping system for DNA evidence interpretation

Introduction

As the sensitivity of forensic DNA typing procedures has improved with the development of better DNA extraction and amplification chemistries and detection instrumentation, more DNA profiles originating from the DNA of two or more individuals are being encountered in forensic casework. The complexity of profile interpretation increases with each additional contributor to a mixture, particularly if the DNA contribution is low and therefore subject to stochastic effects (e.g., allele dropout and greater heterozygous peak height variance). Binary decision making can be applied to the interpretation of mixed profiles and has historically been used in many aspects of the analysis of DNA for human identification purposes. This approach has provided an easily applied means of addressing biological phenomena exhibited in PCR-based typing results at short tandem repeat (STR) loci [1], [2], [3]. Two primary outcomes are considered in a binary interpretation method. For example, (a) a peak observed in an electropherogram at an expected stutter position is interpreted as either stutter or an allelic peak based on relative height, (b) two allelic peaks are interpreted as having originated from the same or different individuals depending on whether they fall within height variance expectations for heterozygous alleles, and (c) an allele is either used or not used to estimate evidential weight based on whether its height meets an empirically determined stochastic threshold [4].

Such “either-or” determinations, however, can be difficult to make given the characteristics of STR mixture results. The primary criterion used in STR interpretation is peak amplitude, relative to the size and position of the peak in the electropherogram. Yet, the sharing of an allele with that of another contributor and/or with a stutter product renders peak height information less meaningful. Furthermore, locus-specific amplification efficiencies and DNA degradation, which can vary in degree among contributors in a mixture, impact relative peak heights. Also, an allelic component of peaks that qualify as stutter cannot be ruled out when alleles from a minor contributor(s) are in the same general height range as stutter peaks [2]. Together with the possibility of allele dropout, the intricacies of mixture analysis create scientific uncertainty in the determination of possible contributor genotypes and can complicate manual interpretation of mixed DNA profiles.

The use of safeguards (such as a stochastic threshold) was recommended by the Scientific Working Group on DNA Analysis Methods (SWGDAM) to mitigate the uncertainty inherent to binary interpretation of single source, mixed-source and low-level typing results [5]. These safeguards, if applied correctly, tend to limit the usage of profile information and thereby typically lead to more common profile probability estimates, as well as more inconclusive conclusions.

Statistical software programs that incorporate probabilistic interpretation models overcome these limitations and fully utilize the available DNA typing information [6], [7], [8], [9]. Probabilistic genotyping refers to the use of software and computer algorithms to apply biological modeling, statistical theory, and probability distributions to infer the probability of the profile from single source and mixed DNA typing results given different contributor genotypes [10]. The software weighs potential genotypic solutions for a mixture by utilizing more DNA typing information (e.g., peak height, allelic designation and molecular weight) and accounting for uncertainty in random variables within the model, such as peak heights (e.g., via peak height variance parameters and probabilities of allelic dropout and drop-in, rather than a stochastic or dropout threshold). Likelihood ratios (LRs) are generated to express the weight of the DNA evidence given two user-defined propositions. Probabilistic genotyping software has been demonstrated to reduce subjectivity in the interpretation of DNA typing results and, compared to binary interpretation methods, is a more powerful tool supporting the inclusion of contributors to a DNA sample and the exclusion of non-contributors [11]. Despite the effectual incorporation of higher level interpretation features, though, probabilistic software programs are not Expert Systems as defined under the National DNA Index System (NDIS) Procedures [12]. The DNA typing data and probabilistic genotyping results require human interpretation and review in accordance with the Quality Assurance Standards for Forensic DNA Testing Laboratories [13].

The fundamental onus on the forensic laboratory with regard to the analysis of DNA mixtures is to seek to remain current with technological developments and relevant issues and to ensure the reliability of its procedures and usage in casework by properly and thoroughly validating any new method prior to use. The interpretation of complex mixtures in particular requires that the laboratory design and execute thorough, targeted experimental studies as part of its internal validation, recognize limitations revealed through the results, and use the results of validation studies to develop detailed, reliable procedures that can be applied uniformly and consistently among analysts. SWGDAM provides guidelines and the Quality Assurance Standards for Forensic DNA Testing Laboratories provide quality assurance requirements for validation [13], [14].

We outline here the internal validation of STRmix™ [6], [15] at the FBI Laboratory in accordance with SWGDAM Guidelines for the Validation of Probabilistic Genotyping Systems [10]. STRmix™ is software that employs a continuous model for DNA profile interpretation and genotype determination based on a Markov Chain Monte Carlo (MCMC) sampling method. Using weights assigned to the resultant genotypes or genotype sets, STRmix™ calculates LRs, which are the probability of the DNA evidence under two opposing hypotheses referred to as H1 and H2. The terms H1 and H2 are used in lieu of “Prosecution hypothesis” (Hp) and “Defense hypothesis” (Hd), respectively, given that they are assigned by the scientist, usually without consultation with legal representatives.

A LR greater than 1 provides support for a specified person of interest as a contributor to the DNA evidence (H1), whereas an LR less than 1 provides support that the person of interest is not a contributor (H2). An LR of 1 provides no greater support for either proposition. We describe suitable experiments using single source samples and a breadth of mixed DNA samples to meet the recommendations and requirements for internal validation and detail additional testing conducted at the FBI Laboratory to aid in procedural and policy development.

Section snippets

Methods

All single source and mixed DNA profiles were generated in-house using DNA samples (collected and typed with informed consent) that were amplified for 27 cycles using the Applied Biosystems AmpFlSTR® Identifiler® Plus PCR Amplification Kit (Thermo Fisher Scientific, Waltham, MA), followed by detection on a 3130xl Genetic Analyzer (Thermo Fisher Scientific). 3130xl data were subsequently analyzed using Applied Biosystems GeneMapper® ID-X version 1.3 (Thermo Fisher Scientific). Protocols and

Verification of model performance, accuracy and precision

For a small subset of profiles, the LR is evident without calculation or can be estimated easily as described in Bright et al. [29]. These include single source profiles where the genotype at each locus is unambiguous, and hence the weight for the correct genotype combination is expected to be 1. As an initial verification of software performance, manually calculated LRs for individual loci in a single source sample were identical to the corresponding LRs produced by STRmix™, and STRmix™

Conclusions

The internal validation studies described herein involved the examination of more than 300 autosomal STR profiles, derived from one to five contributors and representing a wide range of contributor ratios and DNA template amounts. The probabilistic interpretations using laboratory-specific parameters totaled more than 800 known contributor propositions, nearly 60,000 non-contributor tests, and nearly 100 reference sample comparisons to mixed profiles developed from authentic forensic specimens.

Acknowledgements

The authors would like to thank the many individuals who provided scientific or technical support for this work, including: Jerrilyn Conway, Jade Gray, Jeremy Fletcher and Baxter Cohen of the DNA Casework Unit, FBI Laboratory; Jill Smerick and Jodi Irwin of the DNA Support Unit, FBI Laboratory; Laura Russell, Catherine McGovern and Stuart Cooper of the Institute of Environmental Science and Research; Jeffrey Monaghan of Robotech Science, Inc.; and Luigi Armogida of NicheVision. This work was

References (37)

  • J.-A. Bright et al.

    Searching mixed DNA profiles directly against profile databases

    Forensic Science International: Genetics

    (2014)
  • D. Taylor et al.

    The ‘factor of two’ issue in mixed DNA profiles

    Journal of Theoretical Biology

    (2014)
  • J.-A. Bright et al.

    A series of recommended tests when validating probabilistic DNA profile interpretation software

    Forensic Science International: Genetics

    (2015)
  • J.-A. Bright et al.

    Developmental validation of STRmix™, expert software for the interpretation of forensic DNA profiles

    Forensic Science International: Genetics

    (2016)
  • J.M. Curran et al.

    What is the magnitude of the subpopulation effect

    Forensic Science International

    (2003)
  • C.M. Triggs et al.

    The sensitivity of the Bayesian HPD method to the choice of prior

    Science & Justice

    (2006)
  • D. Taylor et al.

    An illustration of the effect of various sources of uncertainty on DNA likelihood ratio calculations

    Forensic Science International: Genetics

    (2014)
  • J.-A. Bright et al.

    The effect of the uncertainty in the number of contributors to mixed DNA profiles on profile interpretation

    Forensic Science International: Genetics

    (2014)
  • Cited by (0)

    View full text