Background
Niemann-Pick disease type C (NPC) is a rare, progressive, neurodegenerative disease arising from autosomal recessive mutations in the
NPC1 (≈95% of cases) or
NPC2 (≈5% of cases) genes [
1,
2], which encode essential lysosomal proteins associated with intracellular lipid transport and metabolism [
3,
4]. Mutated NPC proteins are often misfolded and degraded prematurely, leading to impaired lysosomal function, accumulation of multiple lipid species, and neurodegeneration accompanied by disease symptoms of the liver, spleen, and lungs [
4,
5]. The age of onset for NPC disease can vary greatly, from a neonatal, rapidly fatal disorder to an adult-onset, slowly progressing neurodegenerative disease. NPC has an estimated incidence of approximately 1:100,000 live births [
6]. There are limited treatment options for NPC and the only approved drug in Europe is miglustat.
The NPC Clinical Severity Scale (NPCCSS) is a disease-specific, clinician-reported outcome (ClinRO) measure that was developed to characterize and quantify disease progression. [
7] The original NPCCSS has 9 major domains and 8 minor domains. Four of the major domains (Ambulation, Fine Motor Skills, Swallow, and Speech) were modified from a disability scale developed by Iturriaga et al., [
8] (Seizures and Ocular movements were later added to this scale [
9]); additional clinical findings frequently observed in NPC (Cognition, Eye Movement, Hearing, Memory, and Seizures) were added to form the 9 major domains. These reflect the major neurological features of NPC. Other important but less frequent clinical aspects of NPC were also included as minor domains (Auditory Brainstem Response, Behavior, Gelastic Cataplexy, Hyperreflexia, Incontinence, Narcolepsy, Psychiatric, and Respiratory Status).
The NPCCSS has shown good inter-rater reliability [
7,
10], construct validity, and internal consistency [
7]. Responsiveness of the NPCCSS has also been demonstrated in 2 published studies [
11,
12].
The NPCCSS allows a comprehensive assessment of the symptom burden experienced by NPC patients. However, in clinical research, a short form version would be beneficial, allowing clinicians to focus on the core symptoms of the condition, evaluate outcome across those specific symptoms and reduce variability, as well as increase feasibility in studies. Additionally, a shortened version would be ideal for use in daily practice research as it is less time consuming for clinicians to complete. Therefore a 5-domain version has been developed to cover Cognition, Swallow, Fine Motor Skills, Speech, and Ambulation (Table
1). The domain choice was based on the original work by Iturriaga et al., [
8] which included 4 of the 5 domains, and cognition was added due to its importance as identified by Cortina-Borja et al. [
12] These core domains were also highlighted as the most important from a survey conducted at a recent Patient Focused Drug Development meeting in the US [
13]. Criterion validity of the 5-domain version in terms of its correlation with the 17-domain NPCCSS has been shown [
12] and reliability of the 5-domain measure has also been established, with both strong intra-rater reliability and inter-rater reliability being demonstrated [
14].
Ambulation | 0 = Normal 1 = Clumsy 2 = Ataxic unassisted gait or not walking by 18 months 4 = Assisted ambulation or not walking by 24 months 5 = Wheelchair dependent | 0–5 |
Fine Motor Skills | 0 = Normal 1 = Slight dysmetria/dystonia (independent manipulation) 2 = Mild dysmetria/Dystonia (requires little to no assistance, able to feed self without difficulty) 4 = Moderate dysmetria/dystonia (limited fine motor skills, difficulty feeding self) 5 = Severe dysmetria/Dystonia (gross motor limitation, requires assistance for selfcare activities) | 0–5 |
Swallow | 0 = Normal, no dysphagia 1 = Cough while eating Intermittent dysphagia* + 1 = w/Liquids + 1 = w/Solids Dysphagia* + 2 = w/Liquids + 2 = w/Solids 4 = Nasogastric tube or gastric tube for supplemental feeding 5 = Nasogastric tube or gastric tube feeding only | 0–5 |
Cognition | 0 = Normal 1 = Mild learning delay, grade appropriate for age 3 = Moderate learning delay, individualized curriculum or modified work setting 4 = Severe delay/plateau, no longer in school or no longer able to work, some loss of cognitive function^ 5 = Minimal cognitive function | 0–5 |
Speech | 0 = Normal 1 = Mild dysarthria (easily Understood 2 = Severe dysarthria (difficult to understand) 3 = Non-verbal/functional communication skills for needs 5 = Minimal communication | 0–5 |
5-domain NPCCSS score | Sum of all scores from the 5 domains above | 0–25 (higher score = more severe clinical impairment) |
To further investigate the 5-domain version of the NPCCSS (Table
1), 2 studies were undertaken to evaluate the relevance of these 5 core domains from a patient, caregiver, and clinician perspective. These studies also sought to explore how patients or caregivers and clinicians would define meaningful change across these 5 domains. Data from a clinical trial [
15] was used to assess the construct validity, sensitivity to change, and meaningful change threshold of the 5-domain NPCCSS using an anchor-based approach (Clinician Global Impression of Improvement [CGI-I]) as recommended by regulatory agencies.
Discussion
The original 17 domain NPCCSS was developed to characterize and quantify disease progression across all symptoms experienced by NPC patients. However, a smaller number of items in a measure can reduce variability and provide a more focused subset of symptoms. Focusing on the most relevant domains as assessed by patients, caregivers, and clinicians (Ambulation, Fine Motor Skills, Swallow, Cognition, and Speech), will help inform treatment effects where there are such effects. An abbreviated tool is also helpful in daily clinical practice. From the mixed methods approach used in the current research, involving both patients and caregivers, and the qualitative clinician interviews, the 5-domain NPCCSS was found to be content valid and suitable for use across ages, age of onset, and severity subgroups. This is supported by prior research from Iturriaga et al. [
8] and Cortina-Borja et al., [
12] and is consistent with the report from a recent Patient Focused Drug Development workshop. [
13].
Difficulties with ambulation, swallowing, and speech were highly salient NPC symptoms to the participants interviewed. After these 3 symptoms, Fine Motor Skills, Cognition, and Memory were those most often discussed by participants as being important because of the impacts on patients and caregivers. Cognition and Memory were seen by many participants as being highly interrelated if not the same thing, thus use of one or other in a grouping of the most important symptoms appears to be acceptable.
The 5-domain NPCCSS was shown to have construct validity and be sensitive to change using the data from a recently conducted clinical trial [
15], and has demonstrated reliability in a separate study [
14], reflective of the construct validity and responsiveness of the full NPCCSS measure [
7,
11,
12]. The high correlation reported by Cortina-Borja et al. 2018 [
8] between the 5-domain NPCCSS and the 17-domain NPCCSS was also confirmed (correlation = 0.97), providing data to support the criterion validity of the 5-domain NPCCSS.
Interpretation of any outcome measure score is essential to allow understanding of whether observed changes in a condition are meaningful or not. There has been no prior research looking at this specific element of the NPCCSS measure. Data gathered in both the qualitative studies and the clinical trial provided evidence that a 1-category change in any domain, corresponding to a 1–2-point change in total score of the 5-domain NPCCSS scale, is meaningful. This indicates that worsening by 1 category is a reflection of loss of complex function and increased disability.
Among caregivers and patients across all levels of severity in each domain, all but one participant reported a 1- or 2-category worsening as being meaningful, with most stating a 1-category worsening is meaningful. Clinicians agreed that the measure captures clinically important and relevant NPC symptoms and domain changes, and that delaying progression was important.
The qualitative data provide support and contextualization to the quantitative approaches to determine the minimal clinically important difference for the 5-domain NPCCSS. The anchor-based analyses suggested that progressing beyond a 1-point worsening on the 5-domain NPCCSS would be clinically meaningful and, therefore, preventing a 2-point worsening would be a viable treatment goal.
A potential limitation to the mixed methods study is the small sample size for the web-based survey (n = 49) and interviews (n = 28), especially when considering the broad age range of patients (1–65 years). While there was a good proportion of adult and paediatric patients represented, there was a limited number of adults reporting for themselves (web-based survey n = 6; interviews n = 5) (which could reflect the nature of the condition), and also a small number of very young patients included (aged 6 months to 3 years; web-based survey n = 5 and interviews n = 1). Although the key domains appeared to be generally consistent across age groups in the current survey, there may be some differences in the very young children where Fine Motor Skills were not always ranked as most important. In addition, swallowing impairment tends to begin later and occur when the disease has progressed in severity. While the 5 domains are relevant for all age groups, it is acknowledged that the assessments have some limitations for the very young (0–2 years) age group. For example, ambulation impairments may be difficult to assess in the very young who would not yet be expected to have reached this developmental milestone. An alternative age appropriate measure would need to be utilized for this age group. Furthermore, the interviews focused on those domains highlighted as important or within the 5-domain NPCCSS which means that some of the original 9 domains were not discussed in depth, however the findings suggest that the domains focused on were those that were most important. A further potential limitation is that the patient or caregiver interviews and survey were only conducted in 2 countries (US and UK). Research involving participants from other countries would enable confirmation of cross-cultural validity. Finally, although the qualitative and quantitative data support each other regarding what constitutes a meaningful worsening on the 5-domain NPCSS (ie, 1-point or greater) the gold-standard, anchor-based approach, was based on a small group of subjects (n = 13) due to the late introduction of CGI into the study. Furthermore because of the late introduction, there was increased risk of recall bias as the ratings for the majority of patients were based upon retrospective evaluations of disease status at baseline. Therefore, it is recommended that future researchers continue to explore this property of the measure.
Conclusions
The results of the mixed methods, qualitative studies and clinical trial data are supportive of the 5-domain NPCCSS as a valid measure of NPC. Importantly, through this research, a meaningful change threshold has also been proposed. Nearly all participants expressed in interviews that slowing the progression within any domain on the 5-domain NPCCSS scale would be meaningful to patients and/or caregivers, and this was echoed by clinicians. A 1-category change on a domain equivalent to 1-point or greater change in the 5 domain NPCCSS total score supports a clinically meaningful transition. This supports that each category reflects loss of complex function and increased disability. The anchor-based analyses suggest that progressing beyond a 1-point worsening on the 5-domain NPCCSS would be clinically meaningful and, therefore, preventing a 2-point worsening would be a viable treatment goal.
The reliability has also been demonstrated both within clinician (intra-rater) and between clinicians (inter-rater) [
14]. This evidence is supportive of the 5-domain NPCCSS as a useful measure for assessing disease progression in NPC patients.
Methods
NPC patient survey (OR-SRV-NPC-01)
This study involved adult patients (aged ≥ 18 years) and caregivers of either paediatric (< 18 years old) or adult patients from the US and UK with a confirmed diagnosis of NPC. There were 2 parts to the study: Part 1 was a web-based survey; and Part 2 was a follow-up telephone interview with a subset of survey participants.
In Part 1, participants were asked to rate the severity of their NPC (or the NPC of the person under their care) using a series of questions based upon the 9 major domains of the NPCCSS (Ambulation, Speech, Swallow, Fine Motor Skills, Cognition, Memory, Seizures, Hearing, Eye Movements). The NPCCSS questions and response options had been reworded to be appropriate completion by patients and caregivers while still reflecting the clinician-reported version of the NPCCSS. Definitions for some of the domains were provided. For example, for Cognition, participants were directed to think of cognitive ability as the ability to learn new skills, make decisions, follow instructions, or focus attention; within the measure this is considered distinct from Memory. The 5-domain NPCCSS scores were then used to select a range of severities for the interviews. Participants were also asked to identify and rank the 5 most important domains in NPC from 1 = “the very most important symptom” to 5 = “the least important symptom.” They were asked to identify these from the 9 major domains or given the option to add others.
In Part 2, semi-structured interviews (approximately 90 min long) were conducted by trained interviewers following a discussion guide. All were conducted in English. Although the survey asked about the 9 major domains of the NPCCSS, the interviews focused on exploring the 5-domain NPCCSS symptoms and any other symptoms that participants had indicated as being most important in the web survey. Participants were asked why these symptoms were important, how they impacted activities of daily living (ADL) and health related quality of life (HRQoL), what category decline in each domain would be meaningful (from the point of their current level, a 1-category decline would be a movement to the next more severe response description on an item), and what such a change would mean in terms of impact on ADLs and HRQoL. The response option range of the 5 domains are as follows: Ambulation (normal—wheelchair dependent); Speech (normal—minimal communication); Swallow (normal – nasogastric tube feeding only); Fine Motor Skills (normal—severe dysmetria/dystonia [gross motor limitations, requires assistance for self-care activities]); and Cognition (normal—minimal cognitive function). This information was used to infer what change in total score (range 0–25) on the 5-domain NPCCSS would be meaningful, since it is cognitively challenging to qualitatively discuss the meaning of change at the overall scale level with patients/caregivers.
NPC expert clinician interviews
In-depth, semi-structured interviews lasting up to 1 h were conducted with clinician experts over the telephone. The interviews explored the importance and relevance of the 5 domains, the level of change clinicians consider to be meaningful in the 5-domain NPCCSS score, and each of the separate domains.
NPC clinical trial (CT-ORZY-NPC-002)
The construct validity and meaningful change threshold for the NPCCSS was derived from a 12-month, prospective, randomized, double-blind, placebo controlled therapeutic study in paediatric patients with a confirmed diagnosis of NPC (NCT02612129) [
15]. A total of 50 subjects were randomized and received either placebo (n = 16) or arimoclomol (n = 34). The age range was 2–19 years (mean [SD] 11.1 years [5.0]) and there were 26 females and 24 males.
Convergent validity (construct validity) was assessed using Spearman’s correlation coefficient due to the categorical nature of the response options. A treatment effect was evaluated using mean change (SD) from baseline across both treatment arms (placebo and arimoclomol). Analysis of meaningful change through the anchor-based approach used the CGI-I as the anchor [
18]. The CGI-I is a 1-item measure of change: “Rate total improvement whether or not, in your judgement, it is due entirely to drug treatment” with a 7 category response scale ranging from “very much improved” to “very much worse.”
The CGI-I was collected throughout the trial by the same investigator for each participant. To determine the anchor-based meaningful change threshold, patients were grouped by CGI-I levels. Mean and median change, 95% confidence interval, and standard deviation of the NPCCSS were calculated by CGI-I category.
The analysis included all patients, independent of the assigned treatment group, and was performed on blinded data.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.