Background
Methods | ||
---|---|---|
Preference exploration methods | Individual methods | In-depth individual interviews |
(Semi) structured - individual interviews | ||
Complaints procedures | ||
Concept mappinga | ||
Group methods | Delphi method | |
Dyadic interview | ||
Citizens’ juries | ||
Focus groups | ||
Nominal group technique | ||
Public meetings | ||
Preference elicitation methods | Discrete choice based methods | Adaptive conjoint analysis |
Discrete choice experiment / Best- worst scaling (type 3) | ||
Indifference methods | Contingent valuation | |
Person-trade off | ||
(Probabilistic) threshold technique | ||
Standard gamble | ||
Starting known efficacy | ||
Test trade-off | ||
Time trade-off | ||
Rating methods | Allocation of points | |
Analytic hierarchy process | ||
Constant sum scaling | ||
Measure of value | ||
Outcome prioritization tool | ||
Repertory grid method | ||
Swing weighting | ||
Visual analogue scale | ||
Ranking methods | Best-worst scaling (type 1)b | |
Best- worst scaling (type 2)b | ||
Control preference scale | ||
Q-methodology | ||
Qualitative discriminant process | ||
Self-explicated conjoint |
Methods
Step 1: Q-methodology
Most important criteria | A: Early development | B: Early development | C: Late phase III | D: Post-marketing |
---|---|---|---|---|
A typical survey can be conducted at relatively low costs | ✓ | ✓ | ||
Data can be collected during quick sessions with participants | ✓ | ✓ | ||
Low frequency of sessions required by patients | ✓ | ✓ | ✓ | |
Relatively quick delivery of preparation, data collection, and analysis | ✓ | ✓ | ✓ | ✓ |
A large number of attributes can be explored | ✓ | |||
Suitable to study preferences in a small sample size | ✓ | ✓ | ✓ | |
A low cognitive load on patients | ✓ | ✓ | ✓ | ✓ |
Does not need an education tool or preparatory instructions in order to enhance participant comprehension | ✓ | ✓ | ||
Publically acknowledged by your organisation’s guidelines as an acceptable method to study preferences | ✓ | ✓ | ||
New attributes can be added without making prior results invalid | ✓ | ✓ | ✓ | |
Can be used to collect data from more than one participant in a single session | ✓ | |||
The analysis can calculate risk attitudes, like risk tolerance, and calculate how value functions bend due to the presence of uncertainty in the participant | ✓ | ✓ | ✓ | ✓ |
Explores the reasons behind a preference in detail | ✓ | ✓ | ✓ | ✓ |
Can estimate weights for attributes | ✓ | ✓ | ✓ | ✓ |
Estimates trade-offs that patients are willing to make among attributes | ✓ | ✓ | ✓ | ✓ |
Can quantify heterogeneity in preferences | ✓ | ✓ | ✓ | ✓ |
Internal validity can be established | ✓ | ✓ | ✓ | ✓ |
External validity can be established | ✓ | ✓ | ✓ | ✓ |
Outcomes can refer to a course of health over time (as opposed to a constant health state) | ✘ | ✘ | ||
Sensitivity analysis is possible | ✘ | ✘ | ✘ | ✘ |
Can combine quantitative and qualitative methods | ✘ | ✘ | ✘ | |
Applies validation tests | ✘ | ✘ | ✘ | |
Results can be reproduced by an (independent) researcher for reproducibility | ✘ | ✘ | ✘ | ✘ |
Applies tests for consistency | ✘ | ✘ | ||
Can be conducted without the need for specialized software (beyond Excel) | ||||
Can be conducted without programming skills | ||||
Researcher does not need to supervise the data collection | ||||
Does not require hypothetical scenarios | ||||
Attributes and attribute levels can be determined as part of the method itself (internal identification) | ||||
Data saturation can be achieved relatively quickly | ||||
Does not require model estimations | ||||
Outcomes can be expressed in a particular format (e.g. probability scores, marginal rates of substitution, monetary values) | ||||
Outcomes can refer to a constant health state (as opposed to a course of health over time) | ||||
Uses respondent validation by asking participants to check their data or responses | ||||
Validates through triangulation |
MPLC Scenario | COLUMN A Q-Methodology Description | COLUMN B AHP Description |
---|---|---|
A: Early Development (mechanism of action well understood) | Phase 2a results are complete and phase 2b is being designed. The indication and population are well-defined. The clinical and commercial teams are discussing the criteria and requirements for a target product profile (TPP), including which benefits, risks and tolerability issues to include and what levels of each are the target. The TPP decision is an in-house activity for now, with information being shared with commercial and clinical development teams. The mechanism of action is well-understood. This is a novel indication of a treatment that has been on the market for years. | A drug is being developed for a certain population. The mechanism of action, meaning the specific biochemical interaction by which a drug produces an effect, is well-understood. The drug has been on the market for years for a different condition and its benefit-risk profile is well-understood in that population. However, this is a novel indication of the treatment, and the benefits, risks, and dosing strategy are still being identified in the new population and condition. Phase 2a studies have been conducted to demonstrate clinical efficacy. Phase 2b studies are being designed to find the optimum dose that has the greatest efficacy with minimal side-effects. The internal clinical and commercial teams are discussing the criteria and requirements for a successful treatment. The preference study would be conducted for internal decision-making on whether or not the medication should advance further in development. |
B: Early Development (mechanism of action is not well understood) | Phase 2a results are complete and phase 2b is being designed. The indication and population are well-defined. The clinical and commercial teams are discussing the criteria and requirements for a target product profile (TPP), including which benefits, risks and tolerability issues to include and what levels of each are the target. The TPP decision is an in-house activity for now, with information being shared with commercial and clinical development teams. The mechanism of action is not understood. This is novel indication. | A drug is being developed for a certain population. The mechanism of action, meaning the specific biochemical interaction by which a drug produces an effect, is not understood. This is a novel indication of the treatment, and the benefits, risks, and dosing strategy are still being identified. Phase 2a studies have been conducted to demonstrate clinical efficacy. Phase 2b studies are being designed to find the optimum dose that has the greatest efficacy with minimal side-effects. The clinical and commercial teams are discussing the criteria and requirements for a target product profile (TPP), including which benefits, risks and tolerability issues to include and what levels of each are the target. The TPP decision is an in-house activity for now, with information being shared with commercial and clinical development teams. The preference study would be conducted for internal decision-making on whether or not the medication should advance further in development. |
C: Late Phase III | Clinical data available for pivotal trials. Mechanism of action is understood. Advisory committee/scientific advisory group meeting is scheduled. The goal is to provide data to support benefit-risk assessment to health authorities for regulatory dossier submission. | The benefits and risks dosing strategy of a medical product are reasonably well-characterized, as clinical trials in patients have been completed to assess efficacy, effectiveness, and safety. Mechanism of action is understood, (meaning the specific biochemical interaction by which a drug produces an effect). There is an advisory committee/scientific advisory group meeting scheduled. The goal is to provide patient preference data to support benefit-risk assessment when submitting dossiers to regulators and HTA bodies. |
D: Post-Marketing | The treatment approved a year ago is now discovered from a registry or observational data to have a clinical significant side effect. Currently, the discussion is all in-house, but the signal is likely to lead to a discussion with health authorities. | A medical product approved a year ago is now discovered from a registry or observational data to have a clinical significant side effect. Currently, the discussion is all in-house, but the signal is likely to lead to a discussion with health authorities. The preference study would be used to complement the clinical data by providing the patient’s perspective on benefit-risks. |
Step 2: Analytical Hierarchy Process
Step 3: Method Performance
Step 4: Method Comparison
Results
Step 1: Q-methodology
Step 2: Analytical Hierarchy Process
Criteria | A: Early development | B: Early development | C: Late phase III | D: Post-marketing |
---|---|---|---|---|
Cost | 12.38 | 10.36 | ||
Sample Size | 11.76 | 12.91 | 14.01 | |
Study duration (time needed) | 12.10 | 13.18 | 14.36 | |
Low frequency of sessions | 5.45 | 4.21 | – | – |
A low cognitive load on patients | 8.21 | 4.35 | – | – |
Quick sessions with participants | – | 2.04 | – | – |
Complexity of instructions to participants | – | 3.78 | – | |
Group dynamic with participants | – | – | 1.95 | – |
No interaction between participants (Solitarily exercise) | – | – | 3.80 | – |
Ease to which new attributes can be added without making prior results invalid | 2.91 | 2.75 | 2.92 | – |
Estimating weights for attributes | 4.60 | 3.59 | 6.45 | 4.04 |
Estimating trade-offs between attributes | 5.48 | 6.18 | 9.31 | 5.98 |
8 or more attributes can be explored | – | – | – | 1.89 |
Degree to which internal validation methods can be incorporated | 7.16 | 8.87 | 12.89 | 7.57 |
Degree to which external validity is established | 10.15 | 8.00 | 11.72 | 11.62 |
Exploring the reasons behind a preference in qualitative detail | 8.00 | 9.01 | 6.09 | 4.91 |
Public acknowledgement by your organisation as an acceptable method to study preferences | – | – | 6.15 | 4.27 |
Quantifying heterogeneity in preferences | 6.94 | 6.62 | 13.2 | 9.02 |
Calculating of risk attitudes (like risk tolerance vs. risk aversion) due to uncertainty in the value of an attribute | 4.87 | 4.18 | 8.36 | 6.85 |