Alternative Approaches to Indirect Treatment Comparison
Alternative approaches to indirect treatment comparison include simulated treatment comparison and matching-adjusted indirect comparison [
11,
12]. Simulated treatment comparison is an approach in which detailed predictive equations are constructed to characterize a single index trial for which individual patient-level data are available. Equations include enrollment, randomization, and follow-up. External baseline data from other studies can then be used to simulate those patients’ experience and outcomes according to the index trial. In matching-adjusted indirect comparison, the index trial for which individual patient-level data are available is reweighted using propensity score-type approaches, so that it matches the characteristics of another study.
Neither of these approaches were considered feasible for this analysis due to the complexity of the prognostic information, combined with the heterogeneity in patient and trial characteristics, including need to consider disease stage, age, gender, visceral disease, brain metastases, lactate dehydrogenase (LDH) levels, in addition to any other patient or study characteristics. In the case of simulated treatment comparison there were not sufficient data for the required equations; for matching-adjusted indirect comparison, there was also a limitation in the matching across many prognostic factors, and the need to match to several studies.
For this analysis, a treatment-specific meta-analysis of absolute treatment effect was undertaken, which involved analysis of independent data on OS for talimogene laherparepvec, ipilimumab, and vemurafenib in each published study, but separate analyses of each drug at a time. No attempt was made at network meta-analysis, following the assessment using the Cope framework. However, the outcomes of each relevant treatment arm in the studies used were adjusted for heterogeneity in prognostic factors (i.e., external data were adjusted accordingly to their baseline characteristics), to be comparable to the OPTiM trial. Adjustments were made using a published algorithm [
13,
14].
Compared with the OPTiM trial, trials including ipilimumab and vemurafenib had higher percentages of patients with stage IV M1b/c melanoma, who have a greater mortality risk than patients with stage III melanoma (Table
2). Patients also varied in terms of other baseline characteristics, including gender, Eastern Cooperative Oncology Group (ECOG) performance status, presence of visceral metastases, presence of brain metastases, and LDH levels. Therefore, adjustment was needed to permit comparability of these factors with those of the OPTiM trial.
Table 2
Summary of randomized controlled Phase 3 trials included in the indirect treatment comparison, and patient characteristics used for adjustment of survival
Comparator (dose) | Talimogene laherparepvec | Talimogene laherparepvec | Ipilimumab (3 mg/kg) | Ipilimumab (10 mg/kg) | Vemurafenib (960 mg orally twice daily) |
Patients | Previously untreated and previously treated | Previously untreated and previously treated | Previously treated | Previously untreated | Previously untreated |
Disease stage | Unresectable, stage IIIB/C or IV | Unresectable, stage IIIB/C or IV M1a | Unresectable, stage III or IV | Unresectable, stage III or IV | Unresectable, stage IIIC or IV, positive for the BRAF V600E mutation |
Female (%) | 41 | 44 | 41 | 39 | 41 |
ECOG 0 (%) | 71 | 74 | 53 | 71 | 68 |
Normal LDH (%) | 90 | 94 | 61 | 63 | 58 |
No visceral disease (%) | 55 | 100 | 11 | 17 | 16 |
No brain metastases (%) | 99 | 100 | 89 | 99 | 100 |
The adjustment of survival for differences in baseline characteristics was based upon a predictive model for survival that was developed by Korn et al. using pooled data from 2100 patients with metastatic melanoma treated with variety of regimens from 42 trials conducted between 1975 and 2005 [
13]. This is valuable in this instance as the Korn model is founded on a larger data set than the OPTiM trial would represent, and broader in terms of the baseline characteristics, so that it should be less prone to bias. The Korn model demonstrated that four factors are associated with OS: gender, ECOG performance status, presence of visceral metastases, and presence of brain metastases. In 2014, a five-factor model was used in the National Institute for Health and Care Excellence (NICE) technology appraisal of ipilimumab for previously untreated advanced melanoma (NICE TA 319; [
14]), in which the original Korn model was modified to include LDH level as the fifth factor. The modified Korn model was accepted by NICE and was used in this study.
Survival was adjusted using a hazard ratio (HR) as the modifier; that is, an HR was used that reflected the impact of the difference in patient characteristics between a given trial and the OPTiM trial. For example, a trial including more patients with better ECOG performance status, and more patients without visceral disease, would exhibit higher rates of survival even without treatment; therefore, survival in this trial would have to be adjusted downwards so that each trial’s baseline survival better matched baseline survival in the OPTiM trial, and it is this effect that the Korn algorithm achieves.
In the adjustment used, the trial-specific HR was estimated by applying the modified Korn model from NICE TA 319 [
14], where
\( \bar{X} \) is the proportion of each sample satisfying the condition (e.g.,
\( \bar{X}_{\text{Gender = Female}} \) is the proportion of females).
$$ { \log }\left( {\text{HR}} \right) = - 0.154\bar{X}_{\text{Gender = Female}} - 0.400\bar{X}_{\text{ECOG = 0}} - 0.285\bar{X}_{\text{Visceral = NO}} - 0.306\bar{X}_{\text{Brain = NO}} - 0.782\bar{X}_{\text{LDH = Normal}} $$
In the equation, all variables represent a better prognosis: if more patients in a trial are female, more patients have ECOG status 0, more patients have non-visceral melanoma, more patients do not have brain metastases, and/or more patients have normal LDH levels, prognosis (i.e., survival) improves, and the HR is lower. The ratio of the HR for a given trial and the HR for the OPTiM trial becomes the adjustment factor: \( {\text{HR}}\left( {\frac{{T_{\text{TVEC}} }}{{T_{\text{TRIAL}} }}} \right) = \frac{{{\text{HR}}_{{T_{\text{TVEC}} }} }}{{{\text{HR}}_{{T_{\text{TRIAL}} }} }} \).
Implicit in this is that each of \( T_{\text{TRIAL}} \) and \( T_{\text{TVEC}} \) are relative to the worst prognosis, when all of the factors in the equation equal zero and the adjusted HR equals 1.
Kaplan–Meier (KM) data were simulated at each time point for \( T_{\text{TRIAL}} \), assuming it had the patient population of \( T_{\text{TVEC}} \), which was calculated as \( S\left( t \right)_{{T_{\text{TRIAL}} |T_{\text{TVEC}} }} = S\left( t \right)_{{T_{\text{TRIAL}} }}^{{{\text{HR}}\left( {\frac{{T_{\text{TVEC}} }}{{T_{\text{TRIAL}} }}} \right)}} \).
If a drug was studied in more than one trial included in the analysis, the data from each trial were combined so that all survival data on that drug were included in the comparison. To do this, OS data were adjusted using the modified Korn model and were then pooled across studies using the Mantel–Haenszel method [
15,
16], a fixed-effect model primarily for dichotomous outcomes that can be implemented in modeling survival counts by transformation of the survival data into hazards, or risks, period by period.
The procedure for this involves two stages: first, producing data containing events and non-events such that odds can be calculated; these data were then combined across studies to produce a pooled survival estimate. The data were not combined automatically on the basis of the single curve for survival; rather, the Mantel–Haenszel method combines the rates of death and censoring, across all studies, at each time point, and the Mantel–Haenszel survival curve is calculated from the resultant data.
Detailed procedures/steps involved are as follows:
1.
Each study’s KM data (unadjusted and adjusted) were broken out using the Parmar algorithm [
17,
18], to produce estimates, for each time period (in our analysis this was 1 month), of the number of patients at risk, the number of events (i.e., death or progression, depending upon whether OS data were being analyzed) and the number of censored data points.
2.
In each time interval, the data were pooled using the Mantel–Haenszel method, which is as follows:
(a)
Pooled proportion of deaths in time interval (sum of proportions across included studies for each time point).
(b)
Pooled proportion of patients alive through time interval (sum of proportions across included studies for each time point).
(c)
Mantel–Haenszel odds of dying in time interval (a/b).
(d)
Estimated probability of death in the time interval (c/1 + c).
(e)
Estimated cumulative probability of surviving to the end of that time interval [probability of surviving to end of previous time interval × (1 − d)].
3.
Finally, the pooled survival curve S(t) was created from E. In this method, confidence intervals also can be constructed around S(t).