Background
Process required for pro development: a real-world example
Omecamtiv mecarbil experience: A literature review and gap analysis [12] was conducted after completion of phase I and phase IIa studies of omecamtiv mecarbil [15, 16]. The results of the review identified multiple PRO measures for HF symptoms and impacts, but none was considered to meet fully the criteria put forward by the FDA in the PRO guidance document for use in a clinical trial to support a label claim, particularly due to gaps in documentation of content validity [3]. The FDA is most favorable of direct measures of symptoms and physical limitations as opposed to more abstract concepts of social limitations and patients’ perceptions of their quality of life [3]. Therefore, a decision was made to develop two new measures: one to assess the symptoms of CHF and one to assess the impact of symptoms on functioning. A PRO development committee of clinical and PRO experts was formed.Lessons learned: The number of existing PRO measures and the extensive use of existing PRO measures do not guarantee that any existing measure meets current FDA PRO guidance criteria [12].
Omecamtiv mecarbil experience: An institutional review board (IRB)-approved qualitative study was conducted; patients provided written informed consent before initiation of study-related activities. One-on-one concept elicitation interviews and focus groups were conducted by trained interviewers and moderators using semi-structured interview guides. Each session was recorded and subsequently transcribed for analysis using qualitative analysis software. An open coding approach was used in the analysis. Additional concept elicitation interviews were conducted until saturation was achieved.Three concept elicitation focus groups and six individual interviews were conducted with 19 US adults with CHF (mean age 73.8 [standard deviation (SD) = 13] years; 21 % female; 79 % white; 11 % New York Heart Association [NYHA] class IV disease). The most frequently reported symptoms due to HF in concept elicitation focus groups and interviews were shortness of breath (reported by 74 % of patients), tiredness (58 %), fluid retention (58 %), fatigue (47 %), dizziness/light-headedness (32 %) swelling (32 %), weight fluctuation (26 %), and trouble sleeping (21 %). Other symptoms mentioned by less than 20 % of participants included difficulty concentrating, muscle problems (pain, cramps, numbness), loss of appetite, and bloating. Concept saturation of symptoms was reached by the end of the interviews.HF had the greatest negative impact on physical (difficulty walking [58 %], needing frequent rests [54 %], reduced ability to do household chores [37 %]), social (reduced social/family interactions [37 %], participation in recreational activities/hobbies [26 %]), and emotional (frustration stemming from diminished ability or inability to complete activities [26 %]) domains. Other impacts on functioning mentioned by less than 20 % of participants included difficulty carrying/lifting objects, running, feeling dependent on others, dressing, impacts on intimate relationships, depression, and mood swings. Concept saturation of impacts was reached by the end of the interviews.Initial item pools of two HF measures, the Heart Failure Symptom Diary (HF-SD) and Heart Failure Impact Scale (HFIS), were created based on the results of the concept elicitation focus groups and interviews, along with experts, and input from PRO, translation, and clinical experts, and a review of available literature. All items were programmed in an electronic device for further testing.Lessons learned: A recruitment strategy to ensure inclusion of patient groups that are difficult to recruit (e.g., female patients with CHF and patients with severe [NYHA class IV] disease) is an important consideration at the early stages of development, but can present practical challenges by slowing the development process. Use of a recruitment tracker and continuous contact with study sites to understand which patients are available are helpful strategies, as is securing back-up sites. Close relationships with hospital settings may enhance recruitment of patients with acute and severe disease.
Omecamtiv mecarbil experience: Cognitive interviews using the HF-SD and HFIS electronic PRO (ePRO) measures were conducted with 18 adult US patients with CHF (mean age 68.8 [SD = 12] years, 33 % female, 83 % white, 22 % NYHA class IV disease). Findings from the cognitive interviews suggested that items of the HF-SD and HFIS were relevant and well understood by patients with CHF. Response options were also well understood and appropriate for the items. The recall periods for the HF-SD and HFIS were adequate for recollection of symptom experience and impacts due to HF. Results from cognitive interviews also suggested the need for two additional items of impacted activities: indoor and outdoor activities were changed to separate items, and lifting and carrying were changed to separate items. Some items were reworded for clarity.Lessons learned: Achieving saturation of concepts in the concept elicitation focus groups and interviews ensured that no new concepts were found in the cognitive interviews, and only the separation of related items was needed for the HF-SD and HFIS following the cognitive interviews.
Omecamtiv mecarbil experience: Results of concept elicitation interviews and cognitive interviews were presented to the FDA with request for feedback on the presented evidence in support of the content validity of the two measures. A second round of concept elicitation interviews and cognitive interviews was suggested by the FDA to solicit input from patients with specific demographic (women, African Americans) and clinical (NYHA class IV disease) characteristics. Additional concept elicitation interviews were therefore conducted with 15 US adults with CHF (mean age 59.7 [SD = 10.8] years; 73 % female; 47 % white; 13 % NYHA class IV disease). No participants were included in both rounds of qualitative interviews. The second round of concept elicitation interviews supported the concepts endorsed in the first round of interviews, no new concepts were identified, and no changes were made to the instruments. Additional cognitive interviews were conducted with ten additional US adults with CHF (mean age 68.6 [SD = 9] years; 30 % female; 40 % white; 20 % NYHA class IV disease). The findings from the additional cognitive interviews demonstrated that items of the HF-SD and the HFIS were well understood and relevant to CHF patients, supporting the findings of the initial cognitive interviews.Lessons learned: A challenge was the recruitment of patients with NYHA class IV disease outside of large, specialized academic centers, as current treatment regimens are effective and prevent development of HF to this stage in many patients. While theoretically important, the successful saturation of concepts in the initial round of qualitative studies meant that no new concepts were identified in the second round.
Omecamtiv mecarbil experience: The HF-SD and HFIS measures were piloted in a phase 2 trial to assess item performance in order to identify and select the final instrument items. This trial also provided the opportunity for an initial assessment of the psychometric properties of the final measures. The measures were administered as ePRO assessments, which were initially tested as part of the cognitive interviews, and enabled an assessment of the usability of electronic data capture in HF patients in advance of proceeding with the electronic mode of administration in the phase 3 clinical trial. In addition to providing initial evidence of the measurement properties of the HF-SD and the HFIS, this step provides additional evidence for the measure’s content validity.Lessons learned: Including ePRO administration in the cognitive interview phase provided information on usability and avoided a study to assess concordance between paper and ePRO modes of administration, which is time- and cost-consuming. Testing for feasibility and ease of use of ePRO administration early in development is useful before embarking on its use in clinical trials.
Practical and logistical issues: key learnings from the omecamtiv mecarbil clinical program
Timelines
Omecamtiv mecarbil experience: The expected and actual timelines of HF PRO development stages are presented in Fig. 2 . Early planning in our program led to successful completion of the first round of qualitative interviews and drafting of instruments before the start of phase 2. Work on the PRO strategy for the omecamtiv mecarbil clinical development program started in 2012 and was expected to be completed by 2013. Several factors caused delays in the original timeline, including the need for a second round of concept elicitation interviews and cognitive interviews, difficulties with recruitment of patients with NYHA class IV disease for qualitative interviews, and the urgency and priority status of the drug. Another factor was related to potential difficulty in identifying clinical sites that could help recruit the population of interest within the desired timeframe. Sites using local IRBs usually require lengthier contracting and review processes than sites with central IRBs.Lessons learned: An early start is critical as delays in the original timeline are likely. Sponsors should anticipate and plan for a second round of qualitative studies, including strategies for recruitment of specific types of patients.
Cost
Factor | Considerations |
---|---|
Patient and site recruitment | Use of vendors to identify and recruit patients, payment for participation in qualitative studies, ease of access to target patient groups |
Multiple stakeholder involvement | PRO experts, IRBs, clinicians, patients, researchers, PRO and ePRO vendors |
Number of iterations | Multiple rounds of concept elicitation interviews and cognitive interviews to achieve saturation of concepts and satisfactory evidence of content validity |
Delays in clinical program unrelated to PRO development | Need to update reviews prior to regulatory approval, increased risk of staff changes reducing efficiency of process |
Approach selected for quantitative validation | Stand-alone vs regular clinical trials |
Mode of instrument administration | ePRO vs pencil and paper vs mixed mode; need to demonstrate instrument equivalence |
Clinical trial design | Number of languages and cultural adaptations needed |
Dissemination strategies | Dossier preparation, conference presentations, manuscripts, etc. |
Omecamtiv mecarbil experience: The planning of the PRO strategy for the omecamtiv mecarbil clinical development program started in 2011 with a literature review and the initial budget for the PRO development was within the cost estimate published in 2011. The need to update the literature review prior to regulatory submission, and to conduct a second round of concept elicitation interviews and cognitive interviews and secondary collection of clinical data from the first round of interviews based on FDA feedback adversely affected the timeline and increased the cost. With careful planning we have managed to save costs by including the ePRO measure in the phase 2 clinical trial, thus eliminating the need for a stand-alone study.Lessons learned: The iterative nature of qualitative studies can easily increase costs, but this can be offset by initiating ePRO development early in the planning process.
Engagement with various stakeholders
Omecamtiv mecarbil experience: A PRO committee was formed to oversee development of the new instruments. Patients with CHF were involved in concept elicitation and cognitive interview studies. Within the sponsor organization, representatives from Health Economics and Outcomes Research, Biostatistics, Commercial, Clinical Development, and Regulatory Affairs departments contributed to the PRO development program. Qualitative studies were designed and implemented by a vendor. FDA review and feedback was solicited. Clinician and payer feedback were solicited through market/payer research advisory boards and 1:1 interviews.Lessons learned: Involvement of multiple stakeholders in PRO development ensures and improves the quality of the final measure and contributes to better understanding of the role a PRO can play in clinical trials for involved parties. Careful planning and management of the practical aspects of stakeholder involvement is critical for the success of the program.