A Comparison of Analytic Hierarchy Process and Conjoint Analysis Methods in Assessing Treatment Alternatives for Stroke Rehabilitation

Background: With growing emphasis on patient involvement in health technology assessment, there is a need for scientific methods that formally elicit patient preferences. Analytic hierarchy process (AHP) and conjoint analysis (CA) are two established scientific methods — albeit with very different objectives.

Objective: The objective of this study was to compare the performance of AHP and CA in eliciting patient preferences for treatment alternatives for stroke rehabilitation.

Methods: Five competing treatments for drop-foot impairment in stroke were identified. One survey, including the AHP and CA questions, was sent to 142 patients, resulting in 89 patients for final analysis (response rate 63%). Standard software was used to calculate attribute weights from both AHP and CA. Performance weights for the treatments were obtained from an expert panel using AHP. Subsequently, the mean predicted preference for each of the five treatments was calculated using the AHP and CA weights. Differences were tested using non-parametric tests. Furthermore, all treatments were rank ordered for each individual patient, using the AHP and CA weights.

Results: Important attributes in both AHP and CA were the clinical outcome (0.3 in AHP and 0.33 in CA) and risk of complications (about 0.2 in both AHP and CA). Main differences between the methods were found for the attributes ‘impact of treatment’ (0.06 for AHP and 0.28 for two combined attributes in CA) and ‘cosmetics and comfort’ (0.28 for two combined attributes in AHP and 0.05 for CA). On a group level, the most preferred treatments were soft tissue surgery (STS) and orthopedic shoes (OS). However, STS was most preferred using AHP weights versus OS using CA weights p< 0.001). This difference was even more obvious when interpreting the individual treatment ranks. Nearly all patients preferred STS according to the AHP predictions, while >50% of the patients chose OS instead of STS, as most preferred treatment using CA weights.

Conclusion: While we found differences between AHP and CA, these differences were most likely caused by the labeling of the attributes and the elicitation of performance judgments. CA scenarios are built using the level descriptions, and hence provide realistic treatment scenarios. In AHP, patients only compared less concrete attributes such as ‘impact of treatment.’ This led to less realistic choices, and thus overestimation of the preference for the surgical scenarios. Several recommendations are given on how to use AHP and CA in assessing patient preferences.