Using structural equation modeling for network meta-analysis

The fixed effect network meta-analysis for multiple treatment comparisons based on Lu and Ades’s approach can be specified as:where treatments are coded as A, B, C,..., K, and K is the number of treatments to be compared within the network. The \( {\hat{y}}_{k\cdot j} \) is the expected value for y _k . _j , which is the observed outcome for treatment k in study j. The g() is a link function for the model to transform the \( {\hat{y}}_{k\cdot j} \) to η _k . _j , which is the expected value given by the model for arm k in study j, and μ _b . _j is the baseline treatment effect in trial j. The difference between the other treatment k and treatment b in the same trial will be estimated by expressing them in terms of effects relative to the treatment A, which is the global baseline treatment within the whole network. Due to identification reason and its interpretation as the effect of treatment A compared to itself, d _AA is fixed at 0, and Lu and Ades called d _AB to d _Ak the basic parameters. The advantage of expressing all treatment comparisons as the relations between basic parameters is that the number of pairwise comparisons to be estimated for a network meta-analysis involving k treatments is reduced to k – 1 for the fixed effect [34].

For the random effect network meta-analysis, d _bk in Eq. (1) is replaced by δ _kb . _j , the trial-specific effect of treatment k relative to trial-specific baseline treatment b, and the equation is given as:

These trial-specific effects are then drawn from a normal distribution: \( {\delta}_{bk\cdot j} \sim N\left({d}_{bk},{\tau}_{bk}^2\right) \). Then, d _bk is expressed in terms of the basic parameters: d _bk  = d _Ak  − d _Ab , with d _AA being fixed at 0 [9, 12, 35]. Note that although the model in Eq. (2) uses data from each treatment arm of a study, it selects one treatment within each study as the trial-specific baseline treatment to estimate the treatment contrast between this baseline treatment and other treatments within the same study. When a study consists of more than two treatment arms, it will contribute more than one treatment contrast, and these treatment contrasts are not independent. Therefore, δ _bk . _j will follow a multivariate normal distribution. For instance, suppose study 1 compares treatment B, C and D, and δ _{BC .1} and δ _{BD .1} in Eq. (2) for this study will then follow bivariate normal distribution:

Where cv is the covariance between \( {\tau}_{BC}^2 \) and \( {\tau}_{BD}^2 \). In the Lu & Ades approach, all the random effect variances are constrained to be equal, i.e. \( {\tau}_{BC}^2={\tau}_{BD}^2={\tau}^2 \), and cv is \( \frac{1}{2}{\tau}^2 \), i.e. the correlation between random effects is 0.5 [11].

To implement the treatment contrasts model in Eqs. (1) and (2) into general or generalized linear mixed model, we can either use the contrast-based approach [36], where treatment contrasts are derived from each study before undertaking network analysis, or use the arm-based approach [25, 37], where data from each arm is used directly. For the contrast-based approach, the dependency of treatment contrasts within a multi-arm trial needs to be taken into account in the model. As taking into account this dependency is not straightforward in most software packages, data transformation using some matrix algebra techniques can be used to create an independent dataset [25, 28, 31]. In the contrast-based fixed effect model shown in Eq. (1), effect size summary odds ratio or risk ratio, needs to be transformed into natural log odds ratio or risk ratio, which behaves approximately as a normal, and the model can now be written as:

where Δ _i . _j is the effect size summary of the i ^th treatment contrast in study j such as difference in means or log odds ratio, t _Ak is the contrast coding dummy variable for treatment contrast A versus k for k = B to K, b _AB to b _AK are regression coefficients for treatment contrasts A versus B to A versus K in the network, and \( {\sigma}_{i\cdot j}^2 \) is the known variance of Δ _i . _j . The vector b for regression coefficients can be obtained by [38]:where the matrix X contains all the covariates t _AB, t _AC,…, and t _AK, X ^T is the transposed X, Δ is the vector of Δ _i . _j , and V ⁻¹is the inverse of the block-diagonal matrix V:

The diagonal elements in V are V _j , j = 1 to J, the variance-covariance matrix of v _i . _j in Eq. (3). V _j is a scalar if study j is a two-arm study and a matrix if study j is a multi-arm study. Cheung proposed to use Cholesky decomposition to decompose V ⁻¹ = LL ^T , where L is a lower triangular matrix and L ^T is the transpose of L [28]. We can pre-multiply X and Δ by L ^T to obtain the transformed matrix \( \overset{\sim }{\mathbf{X}}={\mathbf{L}}^{\mathbf{T}}\mathbf{X} \) and the transformed vector \( \overset{\sim }{\boldsymbol{\Delta}}={\mathbf{L}}^{\mathbf{T}}\boldsymbol{\Delta} \). So Eq. (4) can be re-written as:

Under this transformation, the impact of v _i . _j in Eq. (3) has been absorbed into \( \overset{\sim }{\mathbf{X}} \) and \( \overset{\sim }{\boldsymbol{\Delta}} \), so Eq. (3) can be re-written as an ordinary least squares model:where \( {\overset{\sim }{\varDelta}}_{i\cdot j} \) is the transformed Δ _i . _j , and x _Ak is the transformed t _Ak in Eq. (3).

The example dataset contains results of 26 studies that directly compared three treatment groups A, B and C for prevention of first bleeding in patients with liver cirrhosis [39]: A was the control group, B was sclerotherapy, and C was the use of beta-blocker. The whole dataset can be found in the Additional file 1. Among the 26 study, two are three-arm trials, and seven compared A to C and 17 compared A to B. Throughout the analysis in this article, treatment A was chosen as the global baseline treatment.

As the outcome is a binary variable, the difference in the outcome between any two treatments may be expressed as odds ratio or risk ratio, but to undertake a trial-based approach, we need to take a natural log transformation of odds ratio or risk ratio. Here, we used log odds ratio as the effect size measure. For the three-arm trials, we calculated two treatment contrasts, A vs B and A vs C, and the covariance between the two correlated treatment contrasts is the variance of log odds ratio for treatment A. The regression model for the fixed effect network meta-analysis is therefore written as:where lnOR _i . _j is the i ^th log odds ratio for study j, \( {\sigma}_{i\cdot j}^2 \) is the variance of lnOR _i . _j , t _AB is a dummy variable where treatment contrast for A versus B is denoted 1 and contrast for A versus C denoted 0, and t _AC a dummy variable where treatment contrast for A versus C is denoted 1 and contrast for A versus B denoted 0. Note that if there are trials that compared B to C, t _AB would coded −1 and t _AC coded 1 for those trials [25]. The regression coefficient b _AB and b _AC in Eq. (6) cannot be directly estimated in SEM, because lnOR _i . _j are not independent in the three-arm trials; but b _AB and b _AC can be obtained by transforming lnOR _i . _j , t _AB and t _AC using the procedure described in the previous section. For the random effect model, b _AB . j and b _AC . j in Eq. (6) are replaced with \( {b}_{AB. j}^{\ast } \) and \( {b}_{AC. j}^{\ast } \), which are assumed to follow a bivariate normal distribution:

where the β _AB and β _AC are the average treatment effect difference between A and B and between A and C, respectively; and τ ² is the treatment effect variability across studies.

SEM is a multivariate statistical analysis technique that is a combination of factor analysis and multiple regression analysis [40]. Many traditional statistical methods such as analysis of variance, regression analysis, and factor analysis can therefore be considered as special models of SEM. Traditional SEM requires that the outcome variables and the latent constructs have to be continuous, but with new development of SEM theory and software packages, these are no longer limitations of SEM. As a result, generalized linear mixed models and SEM can now be considered generalized latent variable models [41]. The main difference between SEM and generalized linear mixed models is that random effects are explicitly specified as latent variables in SEM and relationships between observed/latent variable are explicitly specified as causal or non-causal. A comprehensive overview of SEM is beyond the scope of this article, and readers can find an in-depth discussion of applications of SEM to univariate and multivariate meta-analyses in a series of articles and a textbook [28‐30, 42‐45].

Although network meta-analysis can now be undertaken within the statistical framework of generalized linear mixed models, we feel integrating network meta-analysis into SEM framework has several advantages: first, network meta-analysis can be visualized in SEM, and this can be useful for understanding the complexity of the model, especially when analysts wish to look into the role of potential effect modifiers or moderators in the comparisons of multiple treatments by undertaking meta-regression [46]. Secondly, SEM software packages are more flexible in making constraints on model parameters such as regression coefficients, variances and covariances, because random effects are explicitly modelled as latent variables. Thirdly, SEM is a primary research tool for social scientists, but they are less familiar with network meta-analysis, which is becoming more and more popular in biological and medical research. Therefore, integrating network meta-analysis into SEM framework will bring network meta-analysis to attentions of greater audiences [45].

Recently, a new approach has been proposed for meta-analysis, which differs from the standard fixed or random effect models [47, 48]. The standard fixed effect meta-analysis for pairwise comparisons is just weight least squares regression and can be written as:where Δ _j may be the log odds ratio or difference in means between two treatments, v _j is the standard error of Δ _j and \( {v}_j \sim N\left(0,{\sigma}_j^2\right) \), where \( {\sigma}_j^2 \) is the variance of Δ _j . In the (UWLS approach, the variance of v _j is in proportion to the variance of Δ _j , i.e. \( {v}_j \sim N\left(0,\phi {\sigma}_j^2\right) \). The introduction of variance adjustment factor ϕ to Eq. (8) will not affect the point estimate for μ, but its standard error will be affected: when ϕ is larger than 1, the confidence interval for μ will be greater than that given by the standard fixed effect model, but in contrast, when ϕ is smaller than 1, the confidence interval for μ will become smaller. According to recent studies [47, 49], UWLS approach provides satisfactory estimates and confidence intervals that are comparable to random effects when there is no publication bias and identical to fixed-effect meta-analysis when there is no heterogeneity.

In network meta-analysis, the numbers of studies involved in pairwise comparisons are usually quite different, and the degree of heterogeneity within each pairwise comparison also varies. Therefore, network meta-analysis usually uses random effect model to take into account the heterogeneity across the whole network. Currently, the Bayesian or non-Bayesian network meta-analysis usually assumes a common variance for the random effect estimation; for instance, our analysis of the example data in the previous section assumed that the random effect variances for the comparisons between treatment A and B and between A and C are identical. This assumption effectively reduces the number of parameters to be estimated in the model, rendering it more likely to converge, and saves the computation time. However, it also makes a strong assumption about the distribution of heterogeneity within the network meta-analysis and sometimes may yield ambiguous results. For instance, suppose in a network meta-analysis involving treatment A, B, C, D and E, only one trial that compares A and E was found. If the heterogeneity is large in other parts of the network, the estimated common variance for random effect is likely to be large but the estimated confidence interval for A-E comparison would become greater than that reported by the single trial, even if the evidence within the network is consistent. This is because the confidence interval for A-E comparison reported by the random effect network meta-analysis is the one given under the assumption that A-E comparison has the same degree of heterogeneity as other pairwise comparisons in the network.

The standard random effect network meta-analysis therefore gives rise to a few issues with regard to the assessment of inconsistency between direct and indirect evidence. For treatment contrasts with few head-to-head trials, their confidence interval estimated by traditional pairwise meta-analysis is very likely to be smaller than that given by the random effect network meta-analysis assuming a common random effect variance. Consequently, methods for evaluation of inconsistency between direct and indirect evidence may yield different results under different assumptions with regard to the random effect variance [50, 51].

The UWLS approach provides an alternative way to address the heterogeneity. The parameter ϕ in UWLS approach can be interpreted from two perspectives: one is to view ϕ as the dispersion parameter to provide a correction to the known with-study standard error \( {\sigma}_j^2 \). For a common ϕ, this can be implemented in most statistical packages. However, if ϕ is unique to different treatment contrasts, it will be far more straightforward to fit this type of models in SEM. The other way to interpret ϕ is to consider UWLS as a multiplicative random effect model, while the traditional random effect model is additive in the structure of random effect components. In other words, ϕ can be viewed as the random effect τ ² in Eqs. (6) and (7), where the total variance is ϕ + σ ², but in UWLS the total variance is ϕσ ². Consequently, UWLS is to add ϕ into a fixed effect model, making it behave similarly to a random effect model, and a large \( \widehat{\phi} \) indicates large treatment effect heterogeneity.

For different pairwise comparisons within the network meta-analysis, we may estimate different ϕ in Eq. (8) for different pairwise comparisons. We now extend the UWLS approach to network meta-analysis involving treatment A, B, C, …, K with p treatment pairs:

The variable Δ _c . j is the treatment contrast c, c = 1 to p, reported by study j, d _Ak , k = B to K, are the basic parameters for the comparison between A and k, \( {\sigma}_{c. j}^2 \) is the variance of Δ _c . j and ϕ _c is the variance adjustment factor for treatment contrast c within the network meta-analysis.

To implement such a model in SEM requires re-arrangement of data. Using the example data for illustration, its UWLS model can be written as:where Δ _1 . j is the log odds ratio reported by study j that compared treatment A to B and Δ _2 . j the log odds ratio for study j that compared treatment A to C; d _AB is the average treatment difference between A and B; d _AC is the average treatment difference between A and C; \( {\sigma}_{1. j}^2 \) is the variance of Δ _1 . j ; \( {\sigma}_{2. j}^2 \) is the variance of Δ _2 . j ; and ϕ ₁ and ϕ ₂ are the variance adjustment factors for treatment contrasts A-B and A-C, respectively.

We used SEM software package Mplus (version 7.11, Muthen & Muthen, Los Angeles, USA) to undertake all the analyses throughout our study, as Mplus is very flexible in making constraints on parameters estimation. All the data and Mplus codes in this article can be found in the Additional file 1.

SEM fits simultaneously a group of regression equations, which specify the relationships between observed and latent variables. Latent variables in SEM represent some hidden constructs that cannot be observed or measured directly but have to be estimated from a group of observed (also known as manifest) variables. One special feature of SEM is that the statistical model can be visualized by using a path diagram, and most SEM software packages allow users to draw their path diagrams and undertake the analysis directly. In a path diagram, observed variables are in squares, while latent variables are in circles. A single arrow represents a prediction or causal relationship, e.g. X → Y dipicts that X predicts Y or X causes Y. A double arrow represents a correlation or covariance, e.g. X ↔ Y depicts that X and Y are correlated. Results show that the log odds ratio for treatment A and B is −0.485 (95% Confidence Interval [CI]: −0.717 to −0.254) and for A and C is −0.600 (95% CI: -0.932 to −0.268).

Figures 1 and 2 show the path diagrams for Eqs. (6) and (7) with fixed and random effects, respectively, demonstrating how to use the multilevel SEM to undertake the random effect network meta-analysis for example data. In the level-1 model (the Within-level in Fig. 2), y is the transformed log odds ratio, and x _AB and x _AC are the transformed t _AB and t _AC, respectively. The filled circle on the arrow from x _AB to y represents random slope that is referred to as s ₁ in the level-2 model (the Between-level in Fig. 2). The filled circle on the arrow from x _AC to y represents random slope that is referred to as s ₂ in the level-2 model. The variance of s ₁ and s ₂ is τ ² in Eq. (7), and their covariance is constrained to be \( \frac{1}{2}{\tau}^2 \). Note that in Fig. 2 there is no random intercept, and the intercept of y is fixed at 0. The arrows from the variable in triangle to s ₁ and s ₂ indicate that the means of s ₁ and s ₂ are estimated, which give rise to β _AB and β _AC in Eq. (7). The variance of the residual error term e _y is fixed at unity. Results show that the log odds ratio for treatment A and B is −0.585 (95% CI: -1.087 to −0.082) and for A and C is −0.711 (95% CI: -1.438 to 0.016).

To estimate UWLS model with common variance adjustment factors ϕ ₁ = ϕ ₂ in Eq. (10), we only need to remove the constraint on the variance of e _y in the fixed effect network meta-analysis model shown in Fig. 1. Table 1 showed results from Mplus for the fixed effect, random effect, and the two UWLS models. Results from Mplus show that ϕ is 3.563, and the log odds ratio for treatment A and B is −0.485 (95% CI: -0.922 to −0.049) and for A and C is −0.600 (95% CI: -1.227 to 0.027). The point estimates are identical to those in the fixed effect model but the confidence intervals are greater. To estimate the UWLS model with unique variance adjustment factors in Eq. (10), we need to create two residual error terms for y: one for studies reporting treatment contrasts A-B and the other for those reporting treatment contrast A-C. Figure 3 shows the path diagram, where y is the transformed log odds ratios and is regressed on x _AB and x _AC, which are the transformed variables t _AB and t _AC, respectively. Variables g _AB and g _AC are dummy variables for studies reporting treatment contrasts A-B and A-C, respectively. The filled circle on the arrow from g _AB to y represents random slope that is labelled as s ₁, and the filled circle on the arrow from g _AC to y represents random slope that is labelled as s ₂. The means of s ₁ and s ₂ are fixed at zero, and the variances of s ₁ and s ₂ are ϕ ₁ and ϕ ₂ in Eq. 8, respectively, with their covariance being fixed at zero. In this model, the residual error for y is split into two independent random variables s ₁ and s ₂, and their variances are estimated separately. Results from Mplus show that the log odds ratio for treatment A and B is −0.484 (95% CI: -0.958 to −0.010) and for A and C is −0.600 (95% CI: -1.075 to −0.125). The point estimates are almost identical to those given by the fixed effect model, but the confidence intervals are greater. Compared to the confidence intervals reported by the UWLS model with common ϕ, the confidence interval for x _AB is greater but that for x _AC is smaller. This is because the variance adjustment factors ϕ ₁ and ϕ ₂ are 4.288 and 2.031, respectively, indicating a greater degree of heterogeneity within A-B head-to-head trials. This is consistent with results from the traditional pairwise meta-analyses in which the degree of heterogeneity in studies reporting treatment contrast A-B is greater than that of studies reporting contrast A-C.