The MAGEC System for Spinal Lengthening in Children with Scoliosis: A NICE Medical Technology Guidance

The sponsor undertook a search for published clinical evidence relating to the decision problem, which identified four studies suitable for inclusion [7, 13‐15]. A further four unpublished studies were identified and were included by the sponsor [16‐19]. One of these four studies has subsequently been published [18]. Two studies were judged as being out of scope—one being an animal study [15] and the other being a cost analysis [17] (which was subsequently considered as economic evidence).

Four of the six studies were prospective case series [7, 13, 14, 18], with one being a retrospective case review [16] and one being a retrospective case-matched series of 12 patients with MAGEC rods and 12 with conventional growth rods [19]. Both retrospective reviews drew on patients from several centres; details on processes such as follow-up and measurement were not provided. The mean follow-up period ranged from 10 months to 2.5 years, with a weighted mean, using patient numbers as weights, of 19 months. All studies recruited patients with childhood scoliosis, except for one study [16], which included a wider patient group—children with any early-onset spinal deformity.

There was some overlap of patients within the included studies, with one study [14] including all of the patients from another [18]; the latter is not discussed further in this article. Twelve patients receiving a MAGEC rod in the case-matched series [19] were included in the case review [16]. The results of the case-matched study [19] are reported, despite the overlap of patients, because it is judged important to report both arms of the only study that included a comparator arm.

The comparator study selected by the sponsor evaluated the clinical and radiographic complications associated with growth rod treatment and was not an efficacy study. It included data from 140 patients, with a mean age of 6 years (range 1.7–10 years), recorded on the multicentre Growing Spine Study Group database. The inclusion criteria were growth rod treatment for early-onset scoliosis and a minimum of 2 years of follow-up [20].

Table 2 reports the results from the five included MAGEC rod studies [7, 13, 14, 16, 19], as well as the comparator conventional rod study used by the sponsor [20].

The sponsor undertook either meta-analysis or quantitative synthesis on all key outcomes, which were compared with those of the conventional growth rod study [20] or the conventional rod arm of the case-matched study [19]. The reported change in the Cobb angle¹ in the MAGEC rod studies [7, 13, 14, 16] was estimated to be 27° (95 % confidence interval [CI] 23–31), similar to the 28° (CI not reported) reported for the comparator [20]. The change in the total spine height¹ for the MAGEC system [7, 13, 14, 16] was estimated to be 46 mm (CI 40.0–51.1)—a substantially lower value than the 77 mm (CI not reported) achieved in the conventional rods arm of the study by Akbarnia et al. [19]. The comparator study did not report this outcome [20].

The change in the thoracic spine height¹ of 31 mm in the MAGEC system studies [7, 13, 14, 16] was not compared with the changes reported in conventional rod studies, because of data limitations. Compared with the findings in the conventional rod study [20], the infection rates (15 % versus 6.4 %), numbers of surgical procedures (6.4 versus 1.1) and numbers of device failures per patient (0.37 versus 0.07) were lower in the MAGEC rod studies [7, 13, 14, 16].

The sponsor concluded that the clinical evidence demonstrated that MAGEC rods are equally as effective as conventional rods and provide a reduced risk of adverse events.

The EAC critiqued the literature search, evidence selection and quality assessment of the included studies completed by the sponsor, and consulted clinical experts, identified using NICE’s published processes, for advice on relevant comparators and patient groups. Where possible, the EAC repeated the sponsor’s searches and obtained a broadly similar yield of search results. The study selection criteria employed by the sponsor limited the intervention to the MAGEC system and placed no constraints on outcomes, study design, language or search dates. It was unclear how the sponsor had selected the comparator study [20].

The EAC reviewed each paper for potential bias, limitations, inconsistencies in results and uncertainty around generalizability. Each study was graded; all were case series, which scored 3 on the Scottish Intercollegiate Guidelines Network (SIGN) grading [21] and ‘very low’ on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) [22], thus any estimate of the effect is very uncertain. Meta-analysis could not mitigate fully against this underlying uncertainty. Case series are prone to selection and observer bias when patients are selected for inclusion in the study and in reporting outcomes. The reporting of inclusion and exclusion criteria within the included studies was generally poor, with two studies not reporting inclusion criteria [7, 18] and only one study reporting any exclusion criteria [14]. The sponsor failed to provide discussion around limitations relating to the included studies.

The technique used by the sponsor for meta-analysis was not reported; only the results were presented. It was thus not possible to critique the methodology and hence to have confidence in the results. The largest weaknesses identified following the critique were the absence of a rationale for the choice of the comparative study, and the lack of discussion of differences in follow-up durations between the conventional and MAGEC rod studies and their potential implications for comparison of outcomes. Given these limitations within the sponsor’s submission, the EAC undertook additional work.

The EAC undertook a literature search to identify any MAGEC rod studies that had not been identified from the sponsor’s search and to identify studies of conventional growth rods to inform the comparator. A number of databases (including Medline, the Cochrane Library and Embase) and the grey literature were searched (the strategies that were adopted are available from the EAC assessment report [2]).

The included studies of conventional rods were limited to those that included at least 20 patients and were published after 1 January 2008. The choice of 20 patients was informed by a scoping search: a cut-off was required pragmatically to enable the EAC to manage the studies within the evaluation timescale. The 2008 filter was applied to capture studies of current growth rod technologies and hence appropriate comparators for MAGEC rods.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram [23] (Fig. 1) shows the numbers of papers that were retrieved and excluded following dual selection by two independent reviewers, together with the reasons for exclusion. Of the 23 included studies, 15 [20, 24‐37] were studies of conventional rods and eight were studies of MAGEC rods [7, 13, 14, 16, 18, 19, 38, 39]. Of the eight studies of MAGEC rods, two were additional to those found from running the sponsor’s search [38, 39]; however, the sponsor advised that patients in these studies were already included in another study [14]. The conventional rod arm of the case-matched study [19] included by the sponsor was also included within the evidence base, totalling 16 studies. Of the 16 conventional rod studies, 15 were retrospective case series and the remaining one was a prospective case series [28].

Data were extracted from the included studies, and the weighted mean follow-up duration was calculated, using the number of patients within each study as a weight. The extraction was undertaken by one reviewer and checked by a second reviewer. There were notable differences between the conventional rod studies and the MAGEC rod studies. The latter included a greater mean Cobb angle at baseline (72.4° versus 65.7°); a lesser mean total spine height at baseline (258 versus 288 mm) and a younger mean age at surgery (6.4 versus 8.0 years). These differences were likely to influence the potential changes in each outcome over time. For example, a greater height at baseline may limit the potential for growth during follow-up periods; similarly, a less severe Cobb angle may limit the potential improvement in that outcome. The mean duration of follow-up in the conventional rod studies was 4.3 years (range 2.5–6.6 years). This was materially longer than the follow-up in any of the MAGEC system studies (range 10 months to 2.5 years). The number of distractions is a function of time, and these are related to other outcomes of interest, including the change in the Cobb angle, the change in spine height and the change in infection rates. Hence the difference in the follow-up durations was a major source of confounding and limited the ability to use the evidence to inform recommendations.

Meta-analyses were undertaken by the EAC for the MAGEC rod and conventional rod studies, following data extraction into a predefined template. The data items that were required included changes in mean differences in the Cobb angle and spine height, and a measure of variance. Not all studies provided usable data. Given the variation in the duration of follow-up between the two sets of studies, the studies were grouped into those with less than, or greater than, 38 months of follow-up. Those conventional rod studies with a follow-up period of less than 38 months were considered to be more comparable to the MAGEC rod studies. The few (n = 3 to n = 8) studies of conventional rods that provided sufficient statistical data to enable them to be included in the longer-term follow-up analyses is an indication of how poor the reporting of many of these studies was. Variance, ranges and CIs were particularly poorly reported. The MAGEC rod studies reported results better than the majority of the conventional rod studies. This variation in reporting could be explained in part by the prospective nature of four of the six MAGEC rod studies [7, 13, 14, 18], compared with the retrospective nature of all but one [28] of the conventional rod studies.

The included studies and their results are shown in Table 3. Fixed- and random-effects models were fitted. Heterogeneity was assessed by using the I ² statistic, which measures the percentage of variation across studies that is due to heterogeneity rather than sampling error. Benchmarks were suggested for I ²: values of the order of 25 % were considered low, 50 % moderate and 75 % high [40]. Where studies were heterogeneous, the random-effects model was more appropriate because (unlike the fixed-effects model) it incorporates the heterogeneity among studies. The results for meta-analysis with low heterogeneity scores tend to be similar for the fixed- and random-effects models.

The mean differences in the Cobb angle between baseline and follow-up were 27° for the MAGEC rod group and 32° for the longer follow-up conventional rod group (CI reported in Table 3). However because of the differences in follow-up periods, the meta-analysis results for these two groups were unlikely to be comparable. Only one study had sufficient data to be included in the shorter follow-up group [37], and there was considerable variation between this and the conventional growth rod studies with longer follow-up. For the total spine height, the estimated mean difference between baseline and follow-up was 4.55 cm with MAGEC rods and 10.76 cm with conventional rods (in the shorter follow-up group). The very high heterogeneity between the conventional rod studies greatly limits the confidence of those results. Pfandlsteiner et al. [29] reported the highest value for changes in the total spine height of all of the studies. The patients in their study tended to be older at the time of surgery than those in the other studies. If that study is excluded, the results for this group are based on the study by Wang et al. [37] only, for which a mean change in the total spine height of 7.96 cm was reported. The average number of infection episodes per patient with MAGEC rods was 0.03–0.04. For the shorter follow-up conventional rod group, the estimated average number of infection episodes per patient was about 0.03, and heterogeneity was quite high. Only two studies had enough data to be included in the meta-analysis. Wang et al. [37] only reported on deep wounds and not both deep and superficial wounds, as was the case for the rest of the studies. If that study is excluded, the results for this group are based on the study by Zhao et al. [33], who reported an average number of infection episodes per patient of 0.12. The average number of infection episodes per patient in the longer follow-up group was estimated to be around 0.14–0.15.

Quantitative analysis was carried out around device failure (limited to rod breakage, rather than either failure to distract or screw failure), the rate of surgical procedures and the rate of distraction. The mean number of distractions per year was greater for the MAGEC rod studies (4.5) than for all conventional rod studies (1.1). The mean number of surgeries per patient per year was smaller in the MAGEC studies (0.9) than in the conventional rod studies (1.5 for shorter follow-up and 1.1 for longer follow-up). The mean rates of device failure per child were lower in the MAGEC group than in either conventional growth rod group (MAGEC studies 6 %, shorter follow-up conventional rod studies 13 %, longer follow-up conventional rod studies 31 %); however, the annualized rates were similar at around 4.5 % in the MAGEC rod studies and in the shorter follow-up conventional rod studies, with the longer follow-up studies having an annualized rate of 7.2 %.

Information on the impact of the procedure on quality of life was reported in only one study [7]. No evidence was reported on ability to conduct daily activities, school attendance or psychological trauma. The sponsor claimed that use of the MAGEC system would improve these aspects, compared with conventional growth rods. Clinical experts advised the EAC and MTAC of the psychological trauma that some children with conventional growth rods experience as a result of the need for repeated surgeries. In some instances, children have developed ‘hospital phobia’ and refused any further distractions. As the use of the MAGEC system overcomes the need for repeated surgery, limiting benefit to the clinical endpoints may understate the value that patients place on the improvement they experience.

The results from the meta-analysis were subject to limitations arising from the high heterogeneity existing across the conventional rod studies; the confounding factor of differences in the baseline characteristics of patients in the MAGEC and conventional rod studies (relating to the total spine height, age and Cobb angle), fundamental differences in the study designs and the substantially shorter follow-up duration reported in the MAGEC rod studies. The EAC judged that there was neither the quality nor the quantity of evidence to draw firm conclusions around the clinical efficacy of MAGEC rods compared with conventional rods and, as such, MAGEC rod studies with longer follow-up durations are required. From a patient perspective, the absolute number of surgeries is the key driver. It was apparent from the clinical evidence that this number was reduced with MAGEC rods compared with conventional rods.

The sponsor undertook a limited search for economic evidence, which was based upon the strategy reported by Richards and Nnadi in their cost study [17]. No other economic studies were identified, hence that study [17] was the only economic study to be included. Richards and Nnadi collected data from 14 children undergoing growth rod surgery for scoliosis. The included patients either underwent primary surgery (MAGEC insertion) or conversion surgery (replacement of conventional rods with MAGEC rods). A unit cost analysis was undertaken to compare the costs of conventional growth rod treatment (n = 8 children) and MAGEC rod treatment (n = 14 children). The authors used the data that were collected and the costs that were calculated to forecast cost savings over a 10-year time horizon [17].

A de novo temporal cost minimization model was submitted by the sponsor and compared the MAGEC system and conventional growth rods over a 6-year time horizon from a NHS and Personal Social Services perspective, with the year of valuation being 2012. The model was created in Statistical Analysis Software^®, and the results were copied into Microsoft Excel^® for submission. The structure of the model was identical for both the treatment and the comparator. In month 0, the cost of insertion of the rods, plus the cost of complete device failure multiplied by the incidence of this failure, were included. In subsequent months, the cost of each lengthening was added, and the total cumulative cost after 6 years was calculated for both the MAGEC and conventional rod arms of the model.

The costs of rod insertion, rod lengthening and device failure for MAGEC and conventional rods were obtained from Richards and Nnadi [17]. Other model inputs were the frequency of rod lengthening and the rate of device failure. Within the sponsor’s model, the cost of distraction of MAGEC rods was applied every 3 months, and that of conventional rod lengthening was applied every 6 months. In the base case, a device failure rate of 0 % was assumed.

The results were reported as the month in which breakeven occurred between the devices, absolute costs (analysed by cost category) and cost differences at 6 years. The base case results from the model showed that MAGEC rods generated cost savings of £9,946 per patient after 6 years, compared with conventional rods. At 6 years, the cumulative per-patient cost of MAGEC rods was £36,094 versus £46,040 for conventional rods. Breakeven between the two types of rod occurred in the 39th month after insertion.

The only sensitivity analysis that was conducted varied the rate of device failure (from 0 % used in the base case). First, an 8.8 % failure rate was considered for both arms of the model, taken from the mean device failure for MAGEC rods in the clinical evidence [7, 13, 14, 16, 19]. Second, a device failure rate of 17.2 % (the highest rate for MAGEC rods in the clinical evidence) was used [16]. Finally, an 8.8 % failure rate in month 0 was considered for both arms of the model, and an additional device failure rate of 1.76 per patient was included at month 13 for conventional rods. The additional device failure at month 13 was misquoted from Yang et al. [41].

The results of the sponsor’s univariate sensitivity analysis on the rate of device failure found that the MAGEC system was always cost saving after 6 years. These savings ranged from £8,109 (breakeven at month 45) where device failure occurred 17.2 % of the time in both arms of the model, and £12,984 (breakeven at month 28) where additional device failure, specific to conventional rods, occurred only at month 13. The results of the sponsor’s de novo model led the sponsor to conclude that even with the most conservative assumptions, the MAGEC system is cost saving, compared with conventional growth rods, and cost savings are usually realised during the third year following insertion.

A critique of the sponsor’s submission was undertaken, which included the literature search for economic evidence, selection of the included studies, the included evidence and the de novo economic model. The EAC attempted to replicate the sponsor’s literature search of PubMed for published economic evidence, using the limited description provided, and also undertook its own search in specialist databases indexing economic research (the databases that were searched and the strategies that were used are available from the EAC assessment report [2]). No economic studies additional to the one included by the sponsor [17] were identified.

Given that the economic evidence base consisted of only one study, it was appropriate for the sponsor to build a de novo economic model. The EAC judged that the sponsor’s de novo costing model captured the key cost considerations relating to the decision problem. There were, however, limitations relating to the model structure and inputs, which the EAC explored.

The modelled cost inputs were taken from the study by Richards and Nnadi [17], which included 14 patients. The EAC acknowledged that there was a lack of published economic evidence available. It sought to verify all inputs used in the model, using data from other sources and expert advice. Other clinical inputs were also validated where possible. After consideration of the components of the model cost inputs, the EAC judged that the sponsor had overestimated the cost of MAGEC rod insertion, MAGEC rod lengthening and conventional rod insertion, and had underestimated the cost of conventional growth rod lengthening and the cost of device failure for both devices.

The EAC contacted clinical experts for advice on the suitability of the structural assumptions that had been made by the sponsor. The key assumption of the sponsor’s de novo model was that the clinical efficacy of all outcomes was equivalent between MAGEC and conventional rods. Given the lack of good-quality clinical evidence, it was not possible to prove or disprove this assumption.

Four further structural assumptions were made. First, no adverse events other than device failure were incorporated into the model; however, the EAC deemed that infection rates were well reported within the clinical evidence and should be included within the model. Second, device failure occurred immediately or not at all (except in the final sensitivity analysis in the conventional growth rod arm of the model). Clinical experts advised that device failure occurring only in the same month as rod placement is unlikely. Third, all parameters in the model were identical for both single and dual rods for both the treatment and the comparator, despite large cost variations between single and dual rods. Finally, no discounting was applied within the model. The NICE methods guide states that “A discount rate of 3.5 % (per year), as recommended by HM Treasury, is used to reflect the time value of costs and benefits” [42].

Because of these structural limitations, the EAC replicated the sponsor’s model in Microsoft Excel^® (making it fully executable) and made four adaptations to the model structure. First, surgical site infections, as reported in the clinical evidence, were included within the model. For conventional growth rods, a monthly infection rate was applied, as these patients undergo both initial insertion surgery and ongoing surgery for distractions. The monthly rate was calculated from the nine studies that reported infection rates in patients with conventional growth rod [20, 24, 25, 27, 28, 31‐33, 37]. For the MAGEC system, infections were assumed to occur in month 0—the same month as the initial rod insertion—with the infection rate being calculated from four studies reporting this information [7, 13, 14, 16].

Second, the EAC calculated and applied a monthly rate of device failure over the 6-year time horizon of the model for both the MAGEC system and conventional rods. This was consistent with the advice received from clinical experts, who, on the whole, said that rod breakage could occur at any time. The monthly rate of complete rod breakage was established using the clinical evidence for both the MAGEC system [7, 13, 14, 16] and conventional growth rods [20, 24, 25, 27‐33, 37].

Third, the EAC weighted the cost of the device in both arms, according to the proportion of patients receiving single and dual rods within the clinical evidence (65 % dual MAGEC rods and 64 % dual conventional rods). Finally, the EAC discounted costs at a rate of 3.5 % per year, as recommended by HM Treasury [42]. The model input parameters were also updated by the EAC, following validation by clinical experts and against other sources where possible.

Following the amendments made by the EAC to the model structure and inputs, the EAC found that MAGEC rods generated cost savings of £12,077 per person over the 6-year time horizon. The cumulative discounted cost over 6 years was £38,242 for MAGEC rods and £50,319 for conventional growth rods, with breakeven at month 35.

Extensive deterministic sensitivity analysis was undertaken around the model inputs. Each model input was varied within a range deemed plausible following consideration of the published evidence and advice from clinical experts. One-way sensitivity analysis showed that the model was robust to changes in most parameters, but it highlighted that the cost and the frequency of distractions of conventional growth rods were the key drivers of the model. This was confirmed by the two-way sensitivity analysis. MAGEC rods would be the more expensive technology if conventional growth rods are distracted less frequently than every 10.2 months if and the majority of distractions are undertaken as outpatient procedures with no complications. Advice from clinical experts suggested that this is unlikely; hence MAGEC rods are likely to be cost saving, compared with conventional rods, when used in the context of the NHS.