A systematic review with attempted network meta-analysis of asthma therapy recommended for five to eighteen year olds in GINA steps three and four

A trained clinical librarian performed a comprehensive literature search for relevant RCTs in the Cochrane Central Register of Controlled Trials (Central), Medline (Pubmed), Embase, CINAHL and ongoing trial registers registered on WHO Search Portal [15], published until 4 February 2010 (For search details, see Additional file 1). In addition, two reviewers (LvdM, PhEL) scrutinized reference lists of included articles, the GINA-guideline and relevant systematic reviews.

We included RCTs conducted in participants aged 5 to 18 years with persistent-moderate asthma and comparing any GINA step 3&4 medication options (see Additional file 2) to each other or against placebo, with a follow-up duration of at least four weeks after start of the intervention. There were no language restrictions. Acceptable outcome measurements were: spirometry (forced expiratory volume in 1 second (FEV₁)_, forced vital capacity (FVC), FEV₁/FVC ratio, forced expiratory flow 25%-75% (FEF_25-75), peak expiratory flow (PEF)), methacholine challenge test (PC₂₀-FEV₁), fractional exhaled Nitric Oxide (FeNO), asthma symptom score, use of ß₂-agonists as breakthrough medication, and quality of life.

If results did not pertain to the 5 to 18 years age category, the trial was excluded with one exception: RCTs including 4-year olds were included if mean or median age was between 5 to 18 years. Studies were excluded if they compared add-on medication to a non-standardised dose of ICS. Cross-over studies not reporting on treatment effects for each separate treatment period were also excluded since carry-over effects cannot be excluded and are extremely difficult to handle [16].

Two reviewers (LvdM, PhEL) independently assessed titles and abstracts of all identified citations against the inclusion criteria. Any disagreements were resolved by consensus; in case of doubt references were included. LvdM and PhEL evaluated in full text all papers thus selected against the inclusion criteria.

LvdM and PhEL extracted, not in duplicate, data on author, source and year of publication, language, study design, interventions (medication, way of administration, dose, and frequency), population summary characteristics (me(di)an age, asthma severity) and study outcomes. If possible, we calculated the mean change from baseline to endpoint for each trial arm. To facilitate meta-analysis, we contacted authors and sponsors of included studies for additional information such as outcomes expressed on other scales, (mean) patient characteristics such as height, and statistics such as standard errors if needed. We asked for separate data (summaries) of participants in the 5 to 18 years range if the trial had combined this group with younger or older participants.

Methodological quality of all included trials was assessed on 9 items [17, 18] (see Table 2). The risk of bias scale was developed using the Cochrane Collaboration’s tool for assessing risk of bias [17]. All items were scored as “yes” for low, “no” for high, and “?” for uncertain risk of bias, respectively.

Many network meta-analyses were based on dichotomous outcomes for each trial. In our study, outcomes were mostly continuous. To take lung function as an example, meta-analysis had been possible if, for each treatment arm, every publication had reported change in mean FEV₁%_pred and its standard error after a suitable period of follow-up. Unfortunately, several trials only reported FEV₁(l) therefore we did some efforts to salvage the problem by converting the FEV₁(l)-value in FEV₁%_pred. Ideally, we would have had access to individual patient data (IPD) for each trial in the review. In our case we simulated IPD using the summary statistics reported. We simulated 1000 virtual children from a general population with age, height and sex distribution based on the available data on mean age, height and sex per trial arm. Next, we calculated a corresponding FEV₁(l)-value per virtual child, using existing formula’s. In a final step, for each trial-arm, we tried to calculate a mean FEV₁%_pred and a corresponding SD from the simulated data, to be used for meta-analysis. (For details on the statistical analysis see Additional file 3) [37‐41].

We also considered the use of Z-scores. However, the SD was frequently missing and not provided after request. Furthermore, Z-scores can only be compared if the average of both outcomes (FEV₁(l) and FEV₁%_pred) differ by a multiplicative factor, equal to the quotient of the standard errors. Since there were no studies that provided both FEV₁(l) and FEV₁%_pred and their respective standard errors, we could not check whether this property was approximately correct in our data and refrained from using this method.

The comprehensive literature search yielded 8,175 references (see Figure 1). We retrieved 200 as full text articles, representing 160 unique studies. Reference tracking of the GINA guideline, systematic reviews and included references did not yield additional references. Twenty-three trials, conducted between 1984 and 2010, met the inclusion criteria and were included [6‐10, 19‐36]. Additional file 4 shows the study characteristics of the 23 trials with 4,129 patients ranging in age from 4 to 18 years; we included 6 trials with a lower age range of 4 years, but with a mean or median between 5 and 18 years [6, 9, 29, 31, 32, 36]. Figure 2 shows the network of direct and indirect comparisons. There are 28 theoretically possible pair-wise comparisons: all 7 GINA options versus placebo and 21 head-to-head comparisons, 3A versus 3B, …, 3A versus 4C, and, taking the other options as a starting point, all the way to 4B versus 4C. The arrows represent the ten actually published direct comparisons. The white boxes show the number of RCTs and total number of participants for each comparison. In total, we found seven different head-to-head comparisons (with between 1 and 7 studies per comparison) and 3 different comparisons with placebo (with between 1 and 8 studies). All indirect comparisons were possible, except for comparisons with GINA 4c (medium dose ICS+theophylline as add-on to step 3), which is not connected to the network. An example of a possible indirect comparison replacing a non-existent direct comparison is step 3A versus 4A via 3B. A more complicated example is 3D versus 4A via 4B and 3B. An example where both direct and indirect comparisons exist would be 3B vs 3C, namely, direct via a N=63 trial, and indirect via N=955(899(3B vs Placebo)+56(Placebo vs 3C)) participants. The latter example illustrates how NMA may add strength to scarcely investigated direct comparisons. Figure 2 shows that 3A versus 3B (N=776), 3B versus placebo (N=899) and 3B versus 4A (N=1977) are relatively well researched, while most other comparisons depend on weak statistical evidence. However, there are some comparisons that benefit from the relatively strong 3B versus placebo connection, for example, 3A versus 3B, and 3B versus 3C.

The high number of question marks (138/207, or 67%) in Table 2 indicates that incomplete, unclear or non-reporting hampered thorough quality assessment. Eleven out of 23 trials reported on compliance, while only 4 reported on blinding of the physician or how missing values were dealt with.

We found enormous variation in choice of outcome measures and how they were reported. Twenty-one studies reported FEV₁, but variation in methods of reporting was quite extreme (Table 3). None of the studies reported IPD. One study reported the method of converting “liters” to “percentage of predicted” (e.g. Quanjer, Zapletal, Polgar or Hankinson) [32]. Thus, although FEV₁ is an outcome that 21/23 studies reported in some form, the results could not be compared straightforwardly, nor pooled. Pooling outcomes on asthma symptoms, the second best, was also not possible (see Additional file 5).

FEV₁₋values depend on sex, age, and height. FEV₁-values are usually not normally distributed and extreme values occur, skewing the mean [31]. Besides differences in reporting of litres and percentages of predicted, the mix of outcome measures and statistical details was often reported unsystematically and awkwardly. Intra-arm differences instead of between-arm differences were often reported, while descriptive statistics (standard deviation, range) were used where inferential statistics (standard errors, confidence intervals) were needed. These inconsistencies or mistakes thwarted our attempts at pooling of results and made a sensible summary difficult altogether. We contacted authors or sponsors for more details (e.g. summaries of patient characteristics for height, IPD, alternative outcome measurements such as the mean difference between the groups with corresponding standard errors, different time point of follow-up) to allow expressing the results on identical scales. Unfortunately, only in four instances we received additional information through these personal communications [7, 8, 22, 34].

We used those trials that reported both FEV₁(l) and FEV₁%_pred to directly compare our simulation results with those empirically measured in these trials, and found that, regrettably, they were very different. In particular, the ranges of the results were much narrower than the empirically measured percentages of predicted. In some cases, results from our conversion method were opposite to the true results (simulated result of FEV₁>100% of predicted versus an observed result of FEV₁<100% of predicted) [6, 30, 31]. Because of these considerable and irresolvable discrepancies, we decided that formal meta-analysis seemed irresponsible. This decision was made easier by the deficient reporting and potentially low methodological quality of many trials.

Since we are not able to pool the data and establish an evidence based ranking of effectiveness of drug treatments in GINA steps 3 and 4, we describe the main findings from the trials of the most frequently compared interventions (>2 trials/comparison), including over 100 patients per intervention group and having a follow up of at least 8 weeks. Many different outcomes are reported. However, in the trial descriptions below we restrict our focus to the following clinically most relevant ones: number of exacerbations, level of control, reliever medication use, symptom score, frequency of night-time awakening, quality of life, FEV_1, hyperresponsiveness (PC₂₀-FEV₁) and PEF. The only interventions compared more than two times were steps 3A versus 3B, that is, adding a LABA to a low dose of ICS or increasing the dose of ICS, and 3B versus 4A, that is, adding a LABA to medium or high dose ICS.