Background
Methods
The bivariate random effects (BREM) approach
Between-studies variability
Parameter | Type of regression line | ||||
---|---|---|---|---|---|
ηon ξ
|
ξon η
|
Don S
| R & G | Major Axis | |
β
| |||||
α
|
-βξ
|
Within-study variability
Multivariate random effects meta-analysis (MREM)
FNAC outcome | Malignant | Suspect | Benign | Total |
---|---|---|---|---|
Final diagnosis | ||||
Malignant |
x
1i1
|
x
1i2
|
x
1i3
|
n
1i
|
Benign |
x
0i1
|
x
0i2
|
x
0i3
|
n
0i
|
The model
Between-studies model
Within-study model
-
The mean logit sensitivity and logit specificity along with their standard errors for any known threshold, j, are estimated by the model. One can then derive sensitivity and specificity as follows: sensitivty j = and specificity j = , and the corresponding standard errors can be calculated using delta method [25]. SAS NLMIXED users avoid hand calculation by using the
'Estimate'
statement (see the SAS syntax for example). -
The diagnostic odds ratio (DOR), that can be derived from sensitivity and specificity, for a given threshold, j, is given by DOR j =
-
The estimated parameters from model (5 - 7) are used to derive the overall median SROC curve that is given by sensitivity(specificity) = . Besides, study specific ROC curves can be generated from the empirical Bayes estimates of the random effects model. In SAS NLMIXED, the empirical Bayes estimates are generated automatically and collected from the output file (
'out'
) specified in the'random'
statement (see SAS syntax). -
Uncertainty around the SROC can be characterized by calculating the confidence interval at each point along the curve.
-
A prediction band for the true ROC curve of a new study can be calculated by adding and subtracting 1.96 times the estimated standard deviation of α i in the equation above.
-
The model accommodates study level covariates, for example corresponding to two different diagnostic tests, that enables to test the hypothesis for the differences between groups, e.g. diagnostic tests. A thorough discussion of comparison between groups or test results using the bivariate model is given in Hamza et al[26]. In the second data example of this paper we show how to test for the significant difference between groups.
Fitting the model
Results
Example 1: Fine-needle aspiration cytologic examination
Type of SROC |
α
|
β
| AUC |
---|---|---|---|
BREM | |||
η on ξ
| 2.110(0.321) | 0.107(0.118) | 0.882 |
ξ on η
| 7.636(6.307) | 2.276(2.463) | 0.955 |
D on S
| 2.643(0.371) | 0.316(0.137) | 0.918 |
Rutter and Gatsonis | 3.094(0.319) | 0.493(0.112) | 0.935 |
Major axis | 2.191(0.406) | 0.138(0.153) | 0.889 |
MREM | 2.368(0.135) | 0.224(0.016) | 0.902 |
Example 2: CAGE in screening for alcoholism
Type of SROC |
α
|
β
|
γ
|
AUC
non-PC
|
AUC
PC
|
---|---|---|---|---|---|
BREM | |||||
η on ξ
| 2.775(0.712) | 0.840(0.316) | 0.050(0.389) | 0.886 | 0.890 |
ξ on η
| 3.618(0.827) | 1.235(0.364) | 0.113(0.479) | 0.902 | 0.908 |
D on S
| 3.160(0.710) | 1.020(0.314) | 0.079(0.419) | 0.895 | 0.900 |
R & G | 3.156(0.657) | 1.019(0.286) | 0.079(0.417) | 0.895 | 0.900 |
Major Axis | 3.165(0.776) | 1.023(0.347) | 0.079(0.420) | 0.895 | 0.900 |
MREM | 2.537(0.312) | 0.795(0.047) | 0.207(0.382) | 0.849 | 0.888 |
Discussion
Model Extension and Limitation
Conclusion
Appendix
*PROC NLMIXED = call SAS procedure NLMIXED, and the options included represents:
df = degree of freedom, we specified 1000 to get Wald test in stead of t-test
qpoints = number of quadrature points, if not specified SAS automatically does
miniter = is the minimum number of iteration;
PROC NLMIXED DATA = cage DF = 1000 MINITER = 30 QPOINTS = 5;
*Specifies initial values for the maximum likelihood estimates
if not specified, all the initial values automatically assigned to be one;
PARMS ma = 2.6 b = 0.8 b2 = 0 mxi1 = -5.2 mxi2 = -3.7 mxi3 = -2.2 mxi4 = -1.0
va = 0.4 cavdi = -0.2 vdi = 0.5 vdij = 0.02;
*Constraint to make sure that xi's are in the right order;
BOUNDS mxi1-mxi2< = 0, mxi2-mxi3< = 0, mxi3-mxi4< = 0;
eta1 = a + b*xi1 + b2*z1;
eta2 = a + b*xi2 + b2*z1;
eta3 = a + b*xi3 + b2*z1;
eta4 = a + b*xi4 + b2*z1;
p01 = 1/(1+exp(-(xi1)));
p02 = 1/(1+exp(-(xi2))) - 1/(1+exp(-(xi1)));
p03 = 1/(1+exp(-(xi3))) - 1/(1+exp(-(xi2)));
p04 = 1/(1+exp(-(xi4))) - 1/(1+exp(-(xi3)));
p05 = 1 - 1/(1+exp(-(xi4)));
p11 = 1/(1+exp(-(eta1)));
p12 = 1/(1+exp(-(eta2))) - 1/(1+exp(-(eta1)));
p13 = 1/(1+exp(-(eta3))) - 1/(1+exp(-(eta2)));
p14 = 1/(1+exp(-(eta4))) - 1/(1+exp(-(eta3)));
p15 = 1 - 1/(1+exp(-(eta4)));
if (p01^ = 0 and p02^ = 0 and p03^ = 0 and p04^ = 0 and p05^ = 0 and
p11^ = 0 and p12^ = 0 and p13^ = 0 and p14^ = 0 and p15^ = 0) then
ll = n01*log(p01)+n02*log(p02)+n03*log(p03)+n04*log(p04)+n05*log(p05)+
n11*log(p11)+n12*log(p12)+n13*log(p13)+n14*log(p14)+n15*log(p15);
else ll = -1**100;
*SAS maximized the likelihood using the general distribution;
MODEL n11 ~general(ll);
*Random effects parameters alpha_i and xi_ij's are normally distributed around their mean and between-studies variance covariance matrix (equation 7);
RANDOM a xi1 xi2 xi3 xi4 ~normal([ma, mxi1, mxi2, mxi3, mxi4],
[va,
cavdi, vdi+vdij,
cavdi, vdi, vdi+vdij,
cavdi, vdi, vdi, vdi+vdij,
cavdi, vdi, vdi, vdi, vdi+vdij])
SUBJECT = study OUT = study_specific; *Empirical Bayes estimate;
*Estimate sensitivities and specificities at each of known thresholds (1,2,3,4);
ESTIMATE 'sensitivity_1' exp(ma + mxi1)/(1 + exp(ma + mxi1));
ESTIMATE 'sensitivity_2' exp(ma + mxi2)/(1 + exp(ma + mxi2));
ESTIMATE 'sensitivity_3' exp(ma + mxi3)/(1 + exp(ma + mxi3));
ESTIMATE 'sensitivity_4' exp(ma + mxi4)/(1 + exp(ma + mxi4));
ESTIMATE 'specificity_1' exp(1)/(1 + exp(mxi1));
ESTIMATE 'specificity_2' exp(1)/(1 + exp(mxi2));
ESTIMATE 'specificity_3' exp(1)/(1 + exp(mxi3));
ESTIMATE 'specificity_4' exp(1)/(1 + exp(mxi4));
run;