Neurocognitive clinical outcome assessments for inborn errors of metabolism and other rare conditions

doi:10.1016/j.ymgme.2016.04.006

Molecular Genetics and Metabolism

Volume 118, Issue 2, June 2016, Pages 65-69

https://doi.org/10.1016/j.ymgme.2016.04.006 Get rights and content

Highlights

•
Evaluating rare disease treatments requires reliable clinical outcome assessments.
•
Considerations were identified for studies measuring behavior and neurocognition.
•
Natural history studies can help identify sensitive and specific clinical endpoints.
•
FDA offers multiple ways to work with the Agency to evaluate measurement tools.
•
New tools can aid research and treatment of inborn errors of metabolism.

Abstract

Well-defined and reliable clinical outcome assessments are essential for determining whether a drug provides clinically meaningful treatment benefit for patients. In 2015, FDA convened a workshop, “Assessing Neurocognitive Outcomes in Inborn Errors of Metabolism.” Topics covered included special challenges of clinical studies of inborn errors of metabolism (IEMs) and other rare diseases; complexities of identifying treatment effects in the context of the dynamic processes of child development and disease progression; and the importance of natural history studies. Clinicians, parents/caregivers, and participants from industry, academia, and government discussed factors to consider when developing measures to assess treatment outcomes, as well as tools and methods that may contribute to standardizing measures. Many issues examined are relevant to the broader field of rare diseases in addition to specifics of IEMs.

Introduction

This article summarizes key points discussed among participants at a workshop convened by the U.S. Food and Drug Administration (FDA) in April 2015 entitled, “Assessing Neurocognitive Outcomes in Inborn Errors of Metabolism.” The workshop brought together clinicians, parents/caregivers, and representatives from industry, academia, and government (FDA and National Institutes of Health). Participants presented their perspectives on factors to consider when developing measures to assess clinical outcomes of candidate and approved treatments for diseases resulting from inborn errors of metabolism (IEMs). [Points raised are meant as considerations and should not be interpreted as guidance for drug development. Similarly, discussion of particular scales does not constitute FDA endorsement of these scales for trial endpoints.] Full proceedings of the meeting are available online at http://www.fda.gov/downloads/Drugs/NewsEvents/UCM493766.pdf.

Section snippets

Challenges of clinical studies of rare diseases

Clinical studies of rare diseases that affect the brain and neurological systems are challenging by their nature. Developing reliable and valid study endpoints can be difficult due to many factors, including small numbers of patients who are often geographically dispersed; heterogeneity of deficits between patients and within individual patients over time; limited clinical data describing signs and symptoms of disease and its progression; and lack of knowledge about the natural history of many

Establishing clinical neurocognitive outcome assessments for IEMs and other rare diseases

Recent advances in diagnostics and enhanced newborn screening programs have made it possible to identify diseases earlier in life and begin treatment sooner, if treatments are available. Such is the case for many IEMs as well as other rare diseases. Increased understanding of the mechanisms of IEMs has led to development of a substantial number of new treatments. Evaluating outcomes of these treatments, however, requires that researchers distinguish brain changes resulting from treatment

Natural history studies of rare diseases

Ideally, natural history studies investigate the natural course of a disease from or before inception, through pre-symptomatic, symptomatic, and clinical stages to the point of cure, chronic disease, or death [6]. They are valuable tools for improving understanding of a disease, establishing clinical outcome assessments that aid in identifying treatment effects, and enhancing and accelerating drug development. Natural history studies may: (1) provide a clinical baseline; (2) quantify rate and

Defining clinically meaningful changes in neurocognition

Presenters noted that clinicians have traditionally been trained to focus on the pathophysiology of disease, the resulting impairments, and associated laboratory measures that can predict the course of disease. For patients with IEMs, however, such measures are often not available and, when they are, may not predict outcomes. While biochemical and other quantitative measures contribute to research on IEMs as well as clinical care, family members and health care providers at the workshop

Approaches for assessing cognition and behavior

Meeting participants with expertise in measuring cognition and behavior stated that such assessments for patients with IEMs and other conditions must be based on an understanding of the patient population and the natural history of the disease. When developing outcome measures, it is important to involve patients and caregivers and to pilot test instruments to ensure that they can be reliably and safely completed by patients and are sensitive enough to characterize impairments and detect

Conclusion

Emerging therapies for IEMs require novel and improved measures for neurocognitive and behavioral endpoints that are easily applied, reliable, and valid. This workshop addressed the significance of such measures for clinical research and the role of natural history studies in developing and evaluating such measures. Meeting participants suggested selecting measures that: (1) are suitable for particular developmental stages, brief, and appropriate to the disability and level of the child; (2)

References (17)

M.Z. Cohen et al.
Implementing common data elements across studies to advance research
Nurs. Outlook
(2015)
E.G. Shapiro et al.
Quantifying behaviors of children with Sanfilippo syndrome: the Sanfilippo Behavior Rating Scale
Mol. Genet. Metab.
(2015)
E.F. Augustine et al.
Clinical trials in rare disease: challenges and opportunities
J. Child Neurol.
(2013)
Section 314.126 - adequate and well-controlled studies
Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims
(December 2009)
Food and Drug Administration: Clinical Outcome Assessment Qualification Program
Guidance for Industry – Critical Path Innovation Meetings
(2015)
S.C. Groft et al.
Rare diseases - avoiding misperceptions and establishing realities: the need for reliable epidemiological data
Adv. Exp. Med. Biol.
(2010)

There are more references available in the full text version of this article.

Cited by (26)

Challenges and strategies for clinical trials in propionic and methylmalonic acidemias
2023, Molecular Genetics and Metabolism
Clinical trial development in rare diseases poses significant study design and methodology challenges, such as disease heterogeneity and appropriate patient selection, identification and selection of key endpoints, decisions on study duration, choice of control groups, selection of appropriate statistical analyses, and patient recruitment. Therapeutic development in organic acidemias (OAs) shares many challenges with other inborn errors of metabolism, such as incomplete understanding of natural history, heterogenous disease presentations, requirement for sensitive outcome measures and difficulties recruiting a small sample of participants. Here, we review strategies for the successful development of a clinical trial to evaluate treatment response in propionic and methylmalonic acidemias. Specifically, we discuss crucial decisions that may significantly impact success of the study, including patient selection, identification and selection of endpoints, determination of the study duration, consideration of control groups including natural history controls, and selection of appropriate statistical analyses. The significant challenges associated with designing a clinical trial in rare disease can sometimes be successfully met through strategic engagement with experts in the rare disease, seeking regulatory and biostatistical guidance, and early involvement of patients and families.
Brain proton MR spectroscopy measurements in CLN3 disease
2023, Molecular Genetics and Metabolism
CLN3 is an autosomal recessive lysosomal disorder with intracellular accumulation of ceroid-lipofuscins. CLN3 classically has onset around 4–6 years of age involving vision loss, followed by developmental regression and seizures. Symptoms are progressive and result in premature death. Because treatments are under development, here we explore magnetic resonance spectroscopy (MRS) measurements of metabolite levels in the brain as a potential objective outcome measures.
Individuals with genetically confirmed CLN3 were enrolled from October 2017–November 2021 in a prospective natural history study (NCT033007304). Baseline concentrations of brain metabolites measured by MRS were compared to concurrently collected dimensional assessment measures: Vineland-3 Adaptive Behavior Composite (ABC) score, verbal intelligence quotient (VIQ), and the Physical, Capability with actual vision, and Clinical global impression of change sub-domains of the Unified Batten Disease Rating Scale (UBDRS).
27 participants with typical CLN3 presentation (15F:12M; ages 6.0–20.7 years) completed MRS, ABC, and UBDRS; 20 (12F:8M; ages 6.5–20.7 years) also completed the VIQ assessment. N-acetyl aspartate [B(95% CI) = −0.61(−0.78;-0.45)] and glutamine/glutamate/GABA [B(95% CI) = −0.82(−1.04;-0.6)] in the parietal gray matter (PGM) decreased across the ages. The strongest correlations between MRS metabolite measurements and the clinical severity assessments were found with N-acetyl aspartate [VIQ (ρ = 0.58), Vineland-3 ABC (ρ = 0.59), UBDRS |ρ| range = (0.57;0.7)] and glutamine/glutamate/GABA [VIQ (ρ = 0.57), Vineland-3 ABC (ρ = 0.60), UBDRS |ρ| range = (0.59;0.77)] measured in the midline PGM. These correlations were accounted for when age was considered.
Based on their correlations to established assessments, NAA and glutamine/glutamate/GABA measured in the midline parietal gray matter may be useful indicators of CLN3 disease state. In a clinical trial, divergence of the MRS measurements and clinical severity markers from age may be useful as surrogate measures for treatment responses.
Role of Psychologists in Pediatric Metabolic Disorders
2022, Pediatric Clinics of North America
Increasing precision in the measurement of change in pediatric neurodegenerative disease
2022, Molecular Genetics and Metabolism
Citation Excerpt :
Blood tests, brain imaging, and sampling of the cerebrospinal fluid cannot currently reveal functional benefit of brain treatments. It is well recognized that the path toward therapy development requires meaningful clinical outcome assessments that can accurately characterize 1) disease natural history, 2) the safety of a therapy, and 3) whether treatment makes a difference in how patients feel and function (cognitive and adaptive skills) [7–9]. Cognitive and adaptive outcome assessments have therefore been used as inclusion/exclusion criteria for enrollment, and as primary and secondary outcome measures in clinical trials (see clinicaltrials.gov).
Due to the surge in new brain-directed treatments, metrics to detect the alteration in developmental trajectories in cognition and adaptive behavior have become increasingly important. We propose Growth Scale Values (GSVs) as a solution to monitoring children with severe neurologic/neurodegenerative conditions. This report stems from a panel of experts presenting at the Gorlin symposium (WORLD Symposium) and a subsequent open Webinar sponsored by the National MPS Society.
Because norm-referenced scores (Standard Scores or Intelligence Quotient, i.e., IQ) do not yield information about gain, stability, or loss of skills, they are not suitable for natural history studies or clinical trials. Age-equivalent (AE) scores have been the standard metric used in natural history studies. While AEs are familiar and interpretable to clinicians and parents, they are imprecise due to lack of standard deviations, standard errors of measurement, and equal intervals between scores. Raw scores also have unequal intervals and are not comparable between ages or ability levels.
The GSV, a nonlinear transformation of raw scores using item calibration to make an interval scale score, can be used for accurate measures of within-person change. GSVs have been identified as a useful metric for longitudinal measurement of other conditions involving neurodiversity. These growth scores circumvent inaccurate AEs in infants, are not limited by age and can be used for impaired patients who are chronologically above the normative age range. GSVs have interval properties (a given difference between GSV values represents the same difference in ability at all score levels) and each GSV value has a known standard error of measurement (SEM). GSVs are recommended to measure change in cognitive and adaptive behavior in natural history studies and in clinical trials for children with neurologic disease.
The nature and impact of neurobehavioral symptoms in neuronopathic Hunter syndrome
2020, Molecular Genetics and Metabolism Reports
Citation Excerpt :
Understanding direct and indirect causes of behavioral decline; the timing, sequence, and evolution of symptoms; and neurobehavioral response to therapies is critical in light of associated profound caregiver and family burden, as well as caregiver preferences that greater attention be given to neurobehavioral disease when assessing therapeutic outcomes. Evaluating whether interventions have an impact on disease requires first a clear measurement of the natural history, from which therapy-induced differences in course can be assessed [13,33–35]. Important work has been done to estimate age ranges that some categories of behaviors emerge [8,18], yet a current gap exists for systematic, prospective measurement and analysis of the full array of disease-specific symptoms [17,23,25,26,28].
In neuronopathic Hunter syndrome, neurobehavioral symptoms are known to be serious but have been incompletely described. While families face significant stress stemming from this complex and far-reaching array of symptoms, neither caregiver burden nor the neurobehavioral symptoms have been measured comprehensively. We delineated these neurobehavioral characteristics and their impact on the caregiver using multiple approaches. Methods: As part of the initial phase of developing a Hunter-specific behavioral assessment tool, we used multiple methods to obtain data on patient behaviors and caregiver burden, with the intention of drafting item sets for the tool. We utilized 1) caregiver descriptions from focus groups and individual interviews, 2) observations from video-recorded play of affected children, 3) descriptions from historic chart review, 4) consultation with patient advocacy groups and international experts, 5) reports from a caregiver advisory board, and 6) literature review. Results: Neurobehavioral symptoms were diverse and categorized as focus/attention, impulsivity/heightened activity, sensation seeking, emotional/behavioral function, social interaction, and sleep. A significant reported challenge was susceptibility to misinterpretation of some behaviors as defiant or aggressive, particularly if physical. Caregiver burden involved social isolation, exhaustion, stress, and financial and vocational strain. These new descriptions will aid in developing quantitative measures of change in neurobehavioral symptoms and family burden. These descriptions will be the foundation of a neurobehavioral rating scale, which is very much needed to aid in patient management and assess interventions for individuals with neuronopathic Hunter syndrome.
Assessments of neurocognitive and behavioral function in the mucopolysaccharidoses
2017, Molecular Genetics and Metabolism
Citation Excerpt :
An important prerequisite for selecting the most appropriate test is a good knowledge of the natural history of the disease, including the physical limitations and behavior that may interfere with test performance. Using the test in a natural history study allows an assessment of whether the test is feasible for the patient population and appropriate for the range of their cognitive functioning, and whether it sensitively measures change resulting from the disease process [12–14]. In addition to determining whether a test is appropriate for the population, the metric to report the child's performance on a test needs to be sensitive to change.
The mucopolysaccharidoses (MPS) are a group of rare, inherited lysosomal storage disorders in which accumulation of glycosaminoglycans (GAGs) leads to progressive tissue and organ dysfunction. In addition to a variety of somatic signs and symptoms, patients with rapidly progressing MPS I (Hurler), II, III, and VII can present with significant neurological manifestations, including impaired cognitive abilities, difficulties in language and speech, behavioral abnormalities, sleep problems, and/or seizures. Neurological symptoms have a substantial impact on the quality of life of MPS patients and their families. Due to the progressive nature of cognitive impairment in these MPS patients, neurocognitive function is a sensitive indicator of disease progression, and a relevant outcome when testing efficacy of therapies for these disorders. In order to effectively manage and develop therapies that address neurological manifestations of MPS, it is important to use appropriate neurocognitive assessment tools that are sensitive to changes in neurocognitive function in MPS patients.
This review discusses expert opinions on key issues and considerations for effective neurocognitive testing in MPS patients. In addition, it describes the neurocognitive assessment tools that have been used in clinical practice for these patients. The content of this review is based on existing literature and information from a meeting of international experts with extensive experience in managing and treating MPS disorders.

View all citing articles on Scopus

¹: Currently at Agios Pharmaceuticals, Cambridge, MA USA.

View full text

Published by Elsevier Inc.

CommentaryNeurocognitive clinical outcome assessments for inborn errors of metabolism and other rare conditions

Highlights

Abstract

Introduction

Section snippets

Challenges of clinical studies of rare diseases

Establishing clinical neurocognitive outcome assessments for IEMs and other rare diseases

Natural history studies of rare diseases

Defining clinically meaningful changes in neurocognition

Approaches for assessing cognition and behavior

Conclusion

Nurs. Outlook

Mol. Genet. Metab.

Clinical trials in rare disease: challenges and opportunities

J. Child Neurol.

Section 314.126 - adequate and well-controlled studies

Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims

Food and Drug Administration: Clinical Outcome Assessment Qualification Program

Guidance for Industry – Critical Path Innovation Meetings

Rare diseases - avoiding misperceptions and establishing realities: the need for reliable epidemiological data

Adv. Exp. Med. Biol.

Commentary
Neurocognitive clinical outcome assessments for inborn errors of metabolism and other rare conditions