Time to See the Difference: Video Capture for Patient-Centered Clinical Trials

Developing drugs for rare disease is a complex challenge. Two main factors that contribute to this challenge are limited information on the natural history of the disease and lack of validated outcome measures. There is often little or no information about the pathophysiological mechanism, disease progression and symptoms, or the impact on quality of life and patient burden. Moreover, both aspects are often further compounded by the heterogeneity of disease presentation [1]. For established diseases such as cancer, while endpoints that can capture meaningful aspects of the disease are often readily available, in orphan diseases the creation of novel ‘fit-for-purpose’ assessment tools and clinical trial endpoints are frequently required.

Especially in diseases where mobility is impacted, the use of video technology to capture the impact of the disease and the assessment of specific parameters, such as gait and stride length, can help design sensitive endpoints. Despite the fact that the collection of video images can be challenging due to confidentiality and privacy issues, videos are one of the most powerful tools to demonstrate in a meaningful way how the patient functions and feels, as opposed to traditional observations. This dynamic measurement allows the assessment of certain parameters of movement, such as gait, that a static picture does not easily allow [2]. Moreover, use of video in clinical trial settings helps to address more recent challenges associated with the coronavirus disease 2019 (COVID-19) pandemic, where the capacity of clinical study sites has been severely limited and has decreased the willingness of vulnerable patients to travel and/or spend time in clinic waiting rooms [3].

This article first details the existing US and EU regulations on how to collect valuable information for the selection of appropriate clinical endpoints and how to generate such data leading to a regulatory approval, with a discussion on the opportunities and challenges. The article then considers examples of orphan drugs approved, with video scales as key clinical endpoints, outlining the difficulties in developing new targeted therapies in orphan disease without standardized clinical assessment tools. Two case studies are finally presented. The first study included patients with juvenile-onset hypophosphatasia (HPP), who suffer, in particular, from impaired mobility. Mobility is a complex function requiring integration across multiple domains, including strength, balance, and coordination. As part of the US FDA review of Strensiq^® (asfotase alfa), a clinical functional scale with video footage was developed to meet the regulatory requirements [4]. The second case study describes how a home-setting video tool was developed and validated (SARA^home) from a commonly used clinical scale (Scale for the Assessment and Rating of Ataxia [SARA]) that allows patients to record their own severity of ataxia [5].

Clinical trials within a drug development program generally build upon the knowledge gained in earlier studies to guide the design and endpoint selection for later stages of development. The value and importance of starting this work early cannot be overstated, as timely input from patients, physicians, sponsors, and regulators can prevent issues when it comes to operationalizing a clinical study.

For many rare diseases, well-characterized efficacy endpoints appropriate for the disease and age are not available. Therefore, developers must often identify new pathways to measure efficacy based on endpoints or biomarkers that have never been used or have not been fully validated. A systemic review and understanding of the clinical manifestations associated with the disease, as well as the aspects of the disease that are meaningful to the patient, are key to identifying a suitable endpoint to evaluate the effectiveness of a drug in a clinical trial [6]. The involvement of patients, caregivers, and patient advocates in this selection of endpoints are encouraged by the European Medicines Agency (EMA) and the FDA as it allows better understanding of patient experiences, perspectives, needs, and priorities related to potential endpoints and what constitutes meaningful changes [7]. There are numerous ways in which patients can engage and provide input, including providing solicited consultation on scientific issues (e.g., clinically meaningful outcome measures) or participating in natural history studies. In 2019, the FDA published a guidance on rare diseases [8] supporting the importance of natural history studies by stating that an in-depth study can help to “select clinical endpoints and develop sensitive and specific outcome measures”. In order to ensure feasibility, usability, and eventual acceptability of the data by the regulatory agencies, it is critical to carefully evaluate the way in which patient engagements are conducted, including the underlying methodology.

In the US and EU, the regulatory approval of any new drug, be it for a common or a rare disease, has to adhere to the same statutory standard of evidence and is based on its positive benefit–risk ratio. Generally, the principle of benefit for regulators is to demonstrate improvement in clinical function, as measured using a clinical endpoint that demonstrates how the patient functions, feels, and survives. This can be very challenging in rare diseases where it may not be possible to meet these standards due to confounding factors such as the small size and heterogeneity of the patient population, the lack of available historical data, the nature of the disease itself, and the absence of regulatory precedents for clinically relevant endpoints for the specific disease [9]. Often, without the regulatory flexibility and access to regulatory pathways such as Accelerated Approval (US) or Conditional Marketing Authorization (EU), emerging therapies would not reach patients in a timely manner, as gathering sufficient data to conclude on risk–benefit would take too long.

In parallel to the involvement of patients and their caregivers, the use and importance of real-world evidence (RWE) is rapidly evolving in registration procedures. A survey conducted among executives of global life science companies by Deloitte highlights that technological and analytical capabilities to utilize RWE have advanced from 2017 to 2018 since their last survey was conducted. Moreover, the use of electronic and digital health care data, including patient-generated data from health applications (apps) and wearables as sources of real-world data strongly increased [10]. Based on the findings of that survey, the authors also expect that the future data landscape will evolve strongly to include new innovative ways of procuring RWE data using non-traditional data sources, and they see a need for external partnerships to access and assess those data sets. The 2018 EMA extrapolation reflection paper [11] supports situations where data sources other than clinical trials can be used and may prove to be complementary to well-designed, prospective studies that track the standard of care. The importance of routine data generated in clinical practice is increasing in value as it gives a better perspective of the real-world reality.

In this context, remote digital health data are increasingly valuable and the rapidly evolving technology and innovation enables new means for reporting disease burden and drug value from a patient’s perspective and experience. Similarly, the landscape of video capture is rapidly evolving to accommodate for the impact of COVID-19 and designing patient-centric studies. For example, the company experience at Aparito is that we are seeing more clinical trials move towards home-based standardized video capture to address the impact of travel fatigue, limited clinical/hospital fatigue, and patient reluctance to expose themselves to unnecessary risk [12].

These advances are also supported by the regulators, as stated by former FDA Commissioner Scott Gottlieb in November 2018: “Across our medical product centers we’re exploring how we can harness real world evidence through electronic health records, apps and other mobile health technologies to collect more reliable continuous data on products’ benefit and risk profiles in the pre and post-market” [13].

There has been a change in the drug development paradigm, accelerated by the restrictions associated with COVID-19. Decentralized clinical trials digital technologies and telemedicine offer new ways for patients to participate in clinical trials. This approach allows the investigator to take the clinical trial to the patient, rather than requiring them to travel to an assessment center. Where the route of administration allows and where there are no acute safety concerns, decentralized trials may also allow for home administration of medicines.

The advantages of such an approach is that a diverse cohort of participants can be recruited more quickly, and attrition rates are reduced because the burden on participants is much lower, overall making the experience of participating in a clinical trial more patient centric and also more representative of real-world administration/use post-approval.

The key challenge with regulatory approvals, in particular in rare diseases, is the selection of a clinical endpoint demonstrating a meaningful way of how the patient functions and feels. In diseases where mobility is impacted, the use of video to capture the impact of the disease and the assessment of specific parameters, such as gait and stride length, can help design sensitive endpoints. Video as an assessment tool also allows the use of historical videos or videos filmed by non-experts outside of clinical settings. Moreover, as a dynamic measure, videos allow for the assessment for certain parameters of movement, such as gait, that a static picture does not easily allow.

Originally, video scales, especially the 6-min walk test (6MWT), were primarily used in cardiac and pulmonary diseases. The 6MWT is widely used to measure functional exercise capacity and reflects an ability to perform everyday physical activities [14].

Analysis of video recordings of patients performing the 6MWT via a video-based assessments tool—Performance Oriented Mobility Assessment (POMA-G)—was instrumental for the approval of Strensiq^® [15] in 2015 by the FDA and will be discussed in more detail below.

Since the approval of Strensiq, video footage has become more frequently used, and, to date, several drugs that treat rare disease have used this approach and resultant clinical outcomes are included in the labels. A review of orphan medicines that received FDA and/or EMA approval between 2017 and 2021 was conducted in the online FDA and EMA Orphan Drug Designations and Approved Drugs Registers to identify indications where video evidence could have been used to support approval (see Table 1). The orphan drug approvals searched for were disease therapy areas with a clear physical component that could potentially use video to review patient changes. As such, oncology or other areas with clear clinical endpoints/biomarkers were excluded. Of those potential examples identified, the review documents were then manually assessed for relevance.

For example, two FDA-approved orphan drugs for the treatment of spinal muscular atrophy—Zolgensma (onasemnogene abeparvovec) [16] and Evrysdi (risdiplam) [17]—used videos to evaluate a patient’s improvement and were reviewed by independent experts in addition to the site clinical evaluator assessment. Video assessments also supported the FDA approval of other orphan drugs, such as Ruzurgi (amifampridine), to treat Lambert–Eaton myasthenic syndrome in pediatric patients, [18] and Brineura (cerliponase alfa), an enzyme replacement therapy for ceroid lipofuscinosis neuronal type 2, also known as Batten disease [19]. Table 1 provides an overview of drug approvals that were supported by video technology (2017–2021).

HPP is a rare, inherited, systemic, metabolic disease caused by deficient activity of the tissue non-specific alkaline phosphatase TNSALP, typically due to mutation(s) in the alkaline phosphatase ALPL gene. HPP is characterized by defective bone mineralization and impaired phosphate and calcium regulation as a result of decreased TNSALP activity, which can lead to potentially serious clinical sequelae, including deformity of bones, fractures/pseudofractures, craniosynostosis, respiratory failure, seizures, impaired renal function, pain, profound muscle weakness, joint damage, chondrocalcinosis, impaired mobility, and dental abnormalities [25‐28].

Classically, HPP has been classified broadly along age groups, and the specific manifestations of HPP in childhood lent itself to gait analysis. HPP in childhood (juvenile-onset HPP) has variable clinical expression with growth retardation. Loss of deciduous teeth before the age of 5 years occurs in association with characteristic dental x-ray findings. Patients complain of stiffness and pain along with delayed walking and/or a waddling gait. X-rays show rachitic deformities and osteopenia [29].

At the time of asfotase alfa development, there were no approved treatments for HPP and management of HPP had been essentially symptomatic or orthopedic [30]. Previous attempts at restoring bone mineralization as a treatment for HPP had very limited to no success [29].

Preliminary data from the asfotase alfa pivotal single-arm study (NCT00952484) [29] and its then ongoing extension study (NCT01203826) [31] supported asfotase alfa’s ability to produce rapid remineralization of bone and sustained healing of rickets in HPP patients between 5 and 12 years of age. Improvements in bone health resulted in sustained gains in growth, as well as in ambulation, strength, and speed and agility, as measured by two performance measures, the 6MWT and the BOT-2 [32].

The original purpose of the 6MWT was to test exercise tolerance in patients with moderate to severe heart or lung disease; however, the use of the test has since been expanded to detect changes in functional exercise capacity in other populations, including people with rheumatic conditions and in healthy older adults or children [33], and has broad regulatory approval.

Since HPP is a musculoskeletal disease, 6MWT was selected as a natural clinical endpoint; however, limitations were identified linked to the population of pediatric patients with juvenile-onset HPP, and the FDA requested the company identify an endpoint that could adequately demonstrate how the patient functions and feels and support the demonstration of relevant improvements [4].

In the analysis of the natural history study conducted to support asfotase alfa registration, the predominant clinical feature of juvenile-onset disease was bone disease, which manifests as impaired gait, bone pain, and muscle weakness [4]. The challenge was to find an assessment tool in juvenile-onset HPP that would accurately quantify medically relevant motor deficiencies and would allow a comparison of data from patients enrolled in the pivotal study and in the historical control group. Patients with HPP have functional deficits in mobility, including ambulation difficulties, weakness, shortened stature, and an inability to carry out activities of daily living. Gait analysis provides an efficient and direct global assessment of a patient’s capabilities across these functional domains. During a second meeting between the FDA and the company, change in gait as a clinical endpoint in the population of patients with juvenile-onset HPP was considered by the FDA as clinically meaningful and was accepted by the FDA to assess the efficacy of asfotase alfa in this population [4].

Gait analysis and the 6MWT are complementary assessments. While the 6MWT is a unidimensional assessment of mobility, gait analysis is multidimensional and reveals details of the individual functional components that contribute to overall ambulation. Video footage of the patients were either collected as part of the 6MWT for patients in the pivotal clinical study with asfotase alfa or as part of the routine visit for patients in the historical control group. Patient confidentiality was maintained throughout the duration of the study, in particular by assignment of a unique masking code, and videos were physically modified to blur identifying features prior to submission for assessment, in order to mask the physical therapist raters and to protect patient identities. Ideally, the endpoint would be both accurate and reliable in the setting of retrospectively obtained data.

Previous studies have demonstrated that measures of gait using accelerometry can be used as a diagnostic tool to distinguish between active healthy adults and individuals with dementia [34], and have shown that inertial sensors can extract more subtle clinical changes from standardized tests such as the Timed Up and Go test to diagnose early-stage Parkinson’s disease [35]. In naturalistic conditions, video footage has also been used to capture the circumstances of falls in elderly people in residential care [36], and sensors embedded in shoes could be potential clinical endpoints in Alzheimer’s disease [37]. These examples illustrate the opportunity but combining new technology with established measures is a way to provide additional construct validity for a novel endpoint.

After a systematic assessment of available gait tools and consultation with physical therapists and gait experts, a specific analysis of the video footage (video footage of juvenile-onset HPP patients walking under controlled conditions collected as part of the pivotal clinical trial) was found to be best suited to meet these needs. Consequently, a scale that could adequately record the findings from the video was required. The Tinetti POMA is an assessment tool originally developed to measure functional performance in older adults and has since been expanded to other populations, including children [38, 39]. The POMA test includes both a balance (POMA-B) and gait (POMA-G) score. For the gait assessment, the patients have to walk a few meters at their usual speed, followed by turning and walking back at a ‘rapid, but safe’ speed using their normal walking aids. The tasks will be evaluated using a prespecified set of items, including the length and height of the steps, step symmetry and continuity, straightness of the trunk, and stance, all of which may be abnormal in HPP patients. These items are scored by a 3-point ordinal grading scale (0, 1, and 2), with 0 indicating the highest level of impairment, and will be summarized to a total score.

For the asfotase alfa trials, the POMA-G was designed to be usable to assess gait from videos without specialized equipment or training, to be reliable and reproducible, and to measure clinically meaningful attributes of gait performance. This aspect was of particular importance, as the video in both trials (pivotal and natural history) was taken without any specific equipment. Moreover, video recordings, compared with immediate visual assessment, largely reduce analysis bias as multiple assessors can evaluate the videos and evaluations can be repeated over time. In addition, static pictures are clearly inferior in capturing dynamic movements such as gait.

Physical therapy experts with extensive experience in evaluating HPP patients assessed the suitability of the POMA-G for use in HPP patients who had pre-existing video footage. Based on this assessment, modifications were made to the POMA-G to improve its sensitivity and discriminative ability for use in children and adolescent patients with HPP while preserving its original clinometric properties, resulting in the mPOMA-G. These modifications were, in particular, the removal of initiation of gait (typically not a difficulty observed with HPP). Path was also removed due to a possible lack of focused and purposeful gait in young children and difficulty in assessing path deviation from historical videos. Furthermore, step length and continuity scale were changed from a 2-point rating scale to a 3-point rating scale to provide greater sensitivity and precision [40].

The videos were then watched by three trained physical therapists who did not participate in the clinical care of enrolled patients and who were blinded to patient identifiers and dates of video recording.

Despite limiting factors identified by the FDA due to the post hoc nature of the analysis, this innovative approach was accepted by the FDA, concluding that improvements noted in both gait, as measured by the mPOMA-G, and mobility, as measured by the 6MWT, were supportive of the efficacy of asfotase alfa in the treatment of patients with juvenile-onset HPP. Along with the totality of evidence, including exploratory analyses of growth and radiographic assessments in the juvenile-onset population and survival analyses in the perinatal/infantile-onset population, this approach of developing a gait scale allowed the demonstration of significant improvement in the mobility of HPP juvenile-onset patients [15]. The mPOMA-G analysis revealed a difference in the mean rate of change in mPOMA-G score for treated patients (2.25 per year) compared with the control group (0.37 per year). This was primarily due to improvements in step length, where 6/8 (75%) treated patients compared with 1/6 (17%) historical controls showed at least a 1-point improvement in step length in either foot [41].

Overall, the approval of Strensiq illustrates the challenges associated with developing drugs for rare diseases that lack validated outcome measures and how they were successfully overcome, including the innovative use of video technology.

Many scales integrating video recording for data collection are under development or have been validated [42‐44].

Most recently, a partnership approach to combine technical and scientific knowledge was used by DZNE (German Centre for Neurodegenerative Diseases) and a med-tech company (Aparito) to co-develop a means for patients to record their own severity of ataxia by using a simplified and shorter Scale for the Assessment and Rating of Ataxia (SARA) version in a video-recordable manner at home (SARA^home).

The original SARA is a validated clinical scale developed to measure disease severity by assessing a range of different impairments in cerebellar ataxia. The assessment includes eight different domains that measure gait, stance, nose–finger coordination, finger chase, hand movement, and speech, each of which are evaluated by clinicians within a hospital setting [45]. Each component is scored against standardized criteria, along a scale ranging from 0 to between 4 and 8 increments depending on the domain, e.g., gait (0–8 points), sitting (0–4 points). The total score from each domain (up to a maximum of 40) indicates the severity of ataxia. The SARA instrument has been used successfully to gauge disease severity in a range of conditions featuring cerebellar degeneration, such as spinocerebellar ataxia [46], Friedrich ataxia [47], and autosomal recessive spastic ataxia of Charlevoix–Saguenay [48]. Although the assessment itself is relatively quick, the need to travel to clinical sites can be stressful and time-consuming for patients and may also impact a patient’s performance while being assessed. This can potentially lead to missed insights into the true nature of disease variability. To adapt the SARA scale into a disease score used by patients outside of hospital settings (‘SARA^home’), a video instruction guide for patients was developed by DZNE with the integration of a reduced number of domains to follow in a standardized order (selection of five SARA items: gait, stance, speech, nose–finger test, and fast alternating hand movements). This was configured into an app within the software platform Atom5™ and translated to German, Dutch and Portuguese. A main benefit of the SARA^home app is that it enables close monitoring of ataxia symptoms within a significantly brief period of time, given that ataxia symptoms can be highly variable within a single day, from day to day and week to week. Travel to clinical sites can be stressful and time-consuming for patients and can distort a patient’s performance while being assessed, potentially leading to missed insights into disease variability. Crude domain scoring performed by the clinician or healthcare professional with the naked eye limits the granular insight that can be obtained from a movement in terms of alignment, fluidity, and symmetry.

Pose estimation software, such as OpenPose [49] and DeepLabCut [50], help to overcome the limitations associated with human manual assessment. The software can identify specific points on the body in each frame of a video, generating time-series data that follows patient movement during the completion of a task. With consideration given to standardization of capture, this data can be analysed to look at features of interest for each study, such as the pattern of trajectory, or smoothness, of a limb; the range of movement, either desired or involuntary (e.g., compensatory or supportive); and symmetry of movement or ability.

In addition to the raw, positional data, this computer vision approach allows the extraction of objective, numerical measures from videos, beyond what is possible by visual assessment alone.

Machine learning (ML) techniques can further enhance the level of information that may be extracted from videos and provides an assessment that is stable over time and not prone to the issues of inter-rater reliability [51].

Examples to date illustrate the potential of widely available technologies such as video recordings in rare disease drug development. Assessment tools using videos have been considered as valuable by the authorities to illustrate the importance of how the patients function as clinically significant endpoints that supported various drug approvals.

Moreover, the evolution of digital tools, connected devices, and other capabilities will allow exploration of new ways of measuring patients’ functioning and feelings more accurately in their daily lives beyond the traditional video assessment tools. New endpoints that capitalize on digital tools, and software and hardware that can more precisely measure function, among others, can not only support modern solutions to existing rare diseases but might also be essential to design and develop new scales that are tailored to rare diseases that are multi-symptomatic and have different expressions, from physical aspects (e.g., gait, as outlined above) to other patient-reported symptoms (e.g., that might impact quality of life).

Early planning of these endpoints with video recordings in the clinical development plan should also be considered. This will facilitate the finalization of the protocol design, ensure a strong analysis plan, and allow certain limitations to be taken into account. In particular, both the video recordings and the analysis should occur in a standardized manner where possible. Training as well as remote support will be essential in this regard to support home assessments in decentralized clinical trials. Overall, this will allow for more patient-centric clinical trials, which will be a major drive to support recruitment and retention, especially in rare and neurodegenerative diseases.