Background
Motivating examples
Example 1: Teacher expectancy on pupil IQ
Example 2: Antidepressants for reducing pain in fibromyalgia syndrome
Methods
Normal random effects model
Bayesian estimation of model parameters
Quantification of heterogeneity
Prediction interval
Random effects model with Box-Cox transformation
Box-Cox transformation
Proposed meta-analysis model and its estimation
Definition of variance of the Box-Cox transformed treatment effect estimate
Frequentist estimation of λ and α
Bayesian estimation of model parameters
Interpretation of results
A median overall treatment effect
Quantification of heterogeneity using the ratio of IQR squares
Prediction interval
Another transformation for dealing with the negative skewness
Implementation of our proposed model
Results
Simulation study
Design
Step 1 | Choose a random effects distribution f(ψ) from candidates including normal distributions, skew-normal distributions, shifted exponential distributions and shifted log-normal distributions, where ψ represents a true parameter vector of the random effects distribution. |
Step 2 | Choose the number of studies (k), mean of the distribution for the within-study variance (σ
2) and true parameters of the random effects distribution (ψ). |
Step 3 | Draw a within-study variance of the treatment effect estimate for the ith study (i=1,…,k); \(\tilde {\sigma }_{i}^{2}\sim N(\sigma ^{2},0.040)\) conditioned on \(0.010<\tilde {\sigma }_{i}^{2}<(2\sigma ^{2}-0.010)\). |
Step 4 | Draw a sampling error of the treatment effect estimate for the ith study (i=1,…,k); \(\tilde {\epsilon }_{i}\sim N(0,\tilde {\sigma }_{i}^{2})\), where \(\tilde {\sigma }_{i}^{2}\) is obtained in Step 3. |
Step 5 | Draw a true treatment effect for the ith study (i=1,…,k); \(\tilde {\theta }_{i}\sim f(\psi)\), where f(ψ) is the specified random effects distribution with the true parameter ψ. |
Step 6 | Obtain a treatment effect estimate for the ith study (i=1,…,k); \(\tilde {y}_{i}=\tilde {\theta }_{i}+\tilde {\epsilon }_{i}\), where \(\tilde {\epsilon }_{i}\) and \(\tilde {\theta }_{i}\) are obtained in step 4 and step 5 respectively. |
Step 7 | |
Step 8 | |
Step 9 | Obtain a posterior median and a 95 percent credible interval of the I
2 from the normal random effects model (1), and those of the ratio of IQR squares from the proposed model (7). Check whether their credible intervals contain the true ratio of IQR squares given by one of either (20.0%, 40.0%, 80.0%). |
Step 10 | Repeat Steps 1 to Step 9 10,000 times. |
Step 11 | Using the posterior medians of the overall mean or the overall median obtained in Step 8, compute a bias and a root mean square error around the true overall median of 0.000. |
Step 12 | Obtain a coverage probability of the overall mean or the overall median by computing the proportion of the time that the 95 percent credible intervals contained the true overall median of 0.000. |
Step 13 | Using the posterior medians of the I
2 or the ratio of IQR squares obtained in Step 9, compute a bias and a root mean square error around the true ratio of IQR squares given by one of either (20.0%, 40.0%, 80.0%). |
Step 14 | Obtain a coverage probability of the I
2 or the ratio of IQR squares by computing the proportion of the time that the 95 percent credible intervals contained the true ratio of IQR squares given by one of either (20.0%, 40.0%, 80.0%). |
Random effects | Normalised | Ratio of | |||
---|---|---|---|---|---|
distribution | Scenario | True parameter | Median | IQR | IQR squares |
Scenario 1-3: Normal distribution (N) | |||||
N(mean,variance) | 1 |
N(0,0.15812) | 0.000 | 0.025 | 20.0% |
2 |
N(0,0.25822) | 0.000 | 0.067 | 40.0% | |
3 |
N(0,0.63252) | 0.000 | 0.400 | 80.0% | |
Scenario 4-6: Skew-normal distribution with moderate positive skewness (pSN1) | |||||
SN(location,scale,slant) | 4 |
S
N(−0.1724,0.2547,5) | 0.000 | 0.025 | 20.0% |
5 |
S
N(−0.2812,0.4159,5) | 0.000 | 0.067 | 40.0% | |
6 |
S
N(−0.6883,1.0198,5) | 0.000 | 0.400 | 80.0% | |
Scenario 7-9: Skew-normal distribution with large positive skewness (pSN2) | |||||
SN(location,scale,slant) | 7 |
S
N(−0.1734,0.2557,20) | 0.000 | 0.025 | 20.0% |
8 |
S
N(−0.2829,0.4192,20) | 0.000 | 0.067 | 40.0% | |
9 |
S
N(−0.6923,1.0258,20) | 0.000 | 0.400 | 80.0% | |
Scenario 10-12: Skew-normal distribution with moderate negative skewness (nSN1) | |||||
SN(location,scale,slant) | 10 |
S
N(0.1715,0.2546,−5) | 0.000 | 0.025 | 20.0% |
11 |
S
N(0.2803,0.4154,−5) | 0.000 | 0.067 | 40.0% | |
12 |
S
N(0.6874,1.0195,−5) | 0.000 | 0.400 | 80.0% | |
Scenario 13-15: Skew-normal distribution with large negative skewness (nSN2) | |||||
SN(location,scale,slant) | 13 |
S
N(0.1725,0.2556,−20) | 0.000 | 0.025 | 20.0% |
14 |
S
N(0.2820,0.4191,−20) | 0.000 | 0.067 | 40.0% | |
15 |
S
N(0.6914,1.0255,−20) | 0.000 | 0.400 | 80.0% | |
Scenario 16-18: Shifted exponential distribution (EXP) | |||||
EXP(rate,shift) | 16 |
E
X
P(5.1507,−0.1348) | 0.000 | 0.025 | 20.0% |
17 |
E
X
P(3.1542,−0.2202) | 0.000 | 0.067 | 40.0% | |
18 |
E
X
P(1.2877,−0.5377) | 0.000 | 0.400 | 80.0% | |
Scenario 19-21: Shifted log-normal distribution (LN) | |||||
LN(mean,variance,shift) | 19 |
L
N(0,0.15782,−1) | 0.000 | 0.025 | 20.0% |
20 |
L
N(0,0.25692,−1) | 0.000 | 0.067 | 40.0% | |
21 |
L
N(0,0.61472,−1) | 0.000 | 0.400 | 80.0% |
-
Bias around the true overall median: (mean of the posterior medians of the overall mean/the overall median) −(true overall median of 0.000)
-
Root mean square error (RMSE) around the true overall median: ((standard deviation of the posterior medians of the overall mean/the overall median) 2+(bias around the true overall median)2) 1/2
-
Coverage probability of the true overall median (%): the proportion of the time that the 95 percent credible intervals of the overall mean/the overall median contained the true overall median of 0.000
-
Bias around the true ratio of IQR squares: (mean of the posterior medians of the I 2/the ratio of IQR squares) −(true ratio of IQR squares given by one of either (20.0%, 40.0%, 80.0%))
-
RMSE around the true ratio of IQR squares: ((mean of the posterior medians of the I 2/the ratio of IQR squares) 2+(bias around the true ratio of IQR squares)2) 1/2
-
Coverage probability of the true ratio of IQR squares (%): the proportion of the time that the 95 percent credible intervals of the I 2/the ratio of IQR squares contained the true ratio of IQR squares given by one of either (20.0%, 40.0%, 80.0%)
Estimation
Results
-
Figure 2 plots the results for the overall mean or the overall median, with the between-study variation (the true ratio of IQR squares: Small = 20.0%, Moderate = 40.0%, Large = 80.0%) on the horizontal axis. The number of studies was fixed as k = 20.
-
Figure 3 plots the results for the I 2 or the ratio of IQR squares, with the between-study variation (the true ratio of IQR squares: Small = 20.0%, Moderate = 40.0%, Large = 80.0%) on the horizontal axis. The number of studies was fixed as k = 20.
-
Figure 4 plots the results for overall mean or the overall median, with the number of studies (k = 10, 20 and 40) on the horizontal axis. The true ratio of IQR squares was fixed as 80.0% (i.e. the scenario of large between-study variation).
-
Figure 5 plots the results for the I 2 or the ratio of IQR squares, with the number of studies (k = 10, 20 and 40) on the horizontal axis. The true ratio of IQR squares was fixed as 80.0% (i.e. the scenario of large between-study variation).
Overall treatment effect
Quantification of heterogeneity
Performance when the number of studies is small
Application
Estimation
Overall treatment effect and quantification of heterogeneity
NRE | BC | BC-SI | |||
---|---|---|---|---|---|
Square root of | |||||
between-study | Normalised | Normalised | |||
Overall mean | variance | Overall median | IQRa
| Overall median | IQRa
|
Post. (s.d.) | Post. (s.d.) | Post. (s.d.) | Post. (s.d.) | Post. (s.d.) | Post. (s.d.) |
(95% CI) | (95% CI) | (95% CI) | (95% CI) | (95% CI) | (95% CI) |
Example 1: Teacher expectancy on pupil IQ | |||||
0.083 (0.061) | 0.146 (0.087) | 0.030 (0.051) | 0.084 (0.074) | n/a | n/a |
(−0.021,0.222) | (0.011,0.344) | (−0.058,0.144) | (0.004,0.278) | ||
Example 2: Antidepressants for reducing pain in fibromyalgia syndrome | |||||
−0.418 (0.067) | 0.164 (0.097) | −0.369 (0.056) | 0.094 (0.077) | −0.361 (0.057) | 0.098 (0.081) |
(−0.567,−0.298) | (0.013,0.384) | (−0.489,−0.267) | (0.005,0.291) | (−0.484,−0.259) | (0.005,0.306) |
NRE | BC | BC-SI | |||
---|---|---|---|---|---|
Ratio of IQR | Ratio of IQR | ||||
I
2 (%) | 95% | squaresa (%) | 95% | squaresa (%) | 95% |
Post. (s.d.) | prediction | Post. (s.d.) | prediction | Post. (s.d.) | prediction |
(95% CI) | interval | (95% CI) | interval | (95% CI) | interval |
Example 1: Teacher expectancy on pupil IQ | |||||
44.9 (24.0) | (−0.284,0.500) | 20.9 (21.9) | (−0.179,0.393) | n/a | n/a |
(0.5,81.9) | (0.1,73.6) | ||||
Example 2: Antidepressants for reducing pain in fibromyalgia syndrome | |||||
39.1 (22.7) | (−0.879,−0.001) | 17.3 (19.1) | (−0.732,−0.118) | 18.4 (19.8) | (−0.753,−0.112) |
(0.4,78.0) | (0.1,65.7) | (0.1,67.9) |