Estimating the concordance probability in a survival analysis with a discrete number of risk groups

Abstract

A clinical risk classification system is an important component of a treatment decision algorithm. A measure used to assess the strength of a risk classification system is discrimination, and when the outcome is survival time, the most commonly applied global measure of discrimination is the concordance probability. The concordance probability represents the pairwise probability of lower patient risk given longer survival time. The c-index and the concordance probability estimate have been used to estimate the concordance probability when patient-specific risk scores are continuous. In the current paper, the concordance probability estimate and an inverse probability censoring weighted c-index are modified to account for discrete risk scores. Simulations are generated to assess the finite sample properties of the concordance probability estimate and the weighted c-index. An application of these measures of discriminatory power to a metastatic prostate cancer risk classification system is examined.

Appendix

1.1 The asymptotic distribution for the CPE excluding ties

It is assumed that the proportional hazards model $h(t|\varvec{X}) = h_0(t) \exp (\varvec{\beta }_0^T \varvec{X})$ is the correct specification for the data and that the standard conditions for the asymptotic normality of $n^{1/2} (\hat{\varvec{\beta }} - \varvec{\beta }_0)$ apply (Andersen and Gill 1982).

Consistency of the CPE excluding ties: $\ \ K_{n,E}(\hat{\varvec{\beta }}) \ \mathop {\rightarrow }\limits ^{p} \ {\mathcal {C}}_E$ where

$$\begin{aligned} K_{n,E}\big (\hat{\varvec{\beta }}\big )= & {} \frac{ \sum _i \sum _j I\Big (\hat{\varvec{\beta }}^T \varvec{X}_i > \hat{\varvec{\beta }}^T \varvec{X}_j\Big )\bigg [1+\exp \Big (\hat{\varvec{\beta }}^T\varvec{X}_{ji}\Big )\bigg ]^{-1}}{\sum _i \sum _j I\Big (\hat{\varvec{\beta }}^T \varvec{X}_i > \hat{\varvec{\beta }}^T \varvec{X}_j\Big )}\\ \text{ and } {\mathcal {C}}_E= & {} \text{ Pr }\bigg [ \varvec{\beta }_0^\text {T}\varvec{X}_1 > \varvec{\beta }_0^\text {T}\varvec{X}_2 | T_2 > T_1, \varvec{\beta }_0^\text {T}\varvec{X}_1 \ne \varvec{\beta }_0^\text {T}\varvec{X}_2 \bigg ] . \end{aligned}$$

Using the proportional hazards specification provided in Sect. 2 and the law of large numbers,

$$\begin{aligned} K_{n,E}\big (\hat{\varvec{\beta }}\big ) \ \mathop {\rightarrow }\limits ^{p} \text{ Pr }\bigg [ T_2 > T_1 | \varvec{\beta }_0^\text {T}\varvec{X}_1 > \varvec{\beta }_0^\text {T}\varvec{X}_2\bigg ] . \end{aligned}$$

Note that the concordance probability excluding ties may be written as

$$\begin{aligned} {\mathcal {C}}_E = \text{ Pr }\Big [ T_2 > T_1 | \varvec{\beta }_0^\text {T}\varvec{X}_1 > \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ] \times \frac{\text{ Pr }\Big [\varvec{\beta }_0^\text {T}\varvec{X}_1 > \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ]}{\text{ Pr }\Big [ T_2 > T_1, \varvec{\beta }_0^\text {T}\varvec{X}_1 \ne \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ]} . \end{aligned}$$

(1)

Now using the independence between subjects, the following relations hold

$$\begin{aligned} \text{ Pr }\Big [\varvec{\beta }_0^\text {T}\varvec{X}_1 > \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ]= & {} \frac{\text{ Pr }\Big [\varvec{\beta }_0^\text {T}\varvec{X}_1 \ne \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ]}{2} \end{aligned}$$

(2)

$$\begin{aligned} \frac{1}{2}= & {} \text{ Pr }\Big [T_2 > T_1 | \varvec{\beta }_0^\text {T}\varvec{X}_1 \ne \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ] \text{ Pr }\Big [\varvec{\beta }_0^\text {T}\varvec{X}_1 \ne \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ]\nonumber \\&+ \,\text{ Pr }\Big [T_2 > T_1 | \varvec{\beta }_0^\text {T}\varvec{X}_1 = \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ] \text{ Pr }\Big [\varvec{\beta }_0^\text {T}\varvec{X}_1 = \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ] \end{aligned}$$

(3)

Substituting (2) into (3), it follows that

$$\begin{aligned} \text{ Pr }\Big [T_2 > T_1 , \varvec{\beta }_0^\text {T}\varvec{X}_1 \ne \varvec{\beta }_0^\text {T}\varvec{X}_2 \Big ] = \text{ Pr }\Big [\varvec{\beta }_0^\text {T}\varvec{X}_1 > \varvec{\beta }_0^\text {T}\varvec{X}_2\Big ] \end{aligned}$$

(4)

and substitution of (4) into the denominator of (1) proves the consistency of the CPE.

Asymptotic distribution of $\ n^{1/2} \left[ K_{n,E}\big (\hat{\varvec{\beta }}\big ) - {\mathcal {C}}_E \right]$

To prove the asymptotic normality of the CPE, it is first shown that if

$$\begin{aligned} \Big |\varvec{\beta }_0^T \varvec{X}_{ij}\Big | > \epsilon > 0 , \end{aligned}$$

(5)

then $n^{1/2} K_{n,E}(\hat{\varvec{\beta }})$ is asymptotically equal to

$$\begin{aligned} n^{1/2} \tilde{K}_{n,E}\big (\hat{\varvec{\beta }}\big ) = \frac{ n^{-3/2} \sum _i \sum _j I\Big (\varvec{\beta }_0^T \varvec{X}_{ij} > 0\Big )\bigg [1+\exp \Big (\hat{\varvec{\beta }}^\text {T}\varvec{X}_{ji}\Big )\bigg ]^{-1}}{ n^{-2} \sum _i \sum _j I\Big (\varvec{\beta }_0^T \varvec{X}_{ij} > 0\Big )} \end{aligned}$$

where within the indicator functions the estimate $\hat{\varvec{\beta }}$ is replaced with the true regression coefficient $\varvec{\beta }_0$ and $\varvec{X}_{ij} = \varvec{X}_i-\varvec{X}_j$. Note that in the discrete covariate case there are a finite number of covariate values $\varvec{X}_{ij}$. As a result, (5) is not a strong assumption.

The asymptotic equality is demonstrated by considering the cases $\varvec{X}_{ij}=0$ and $\varvec{X}_{ij} \ne 0$ separately.

If $\varvec{X}_{ij} = 0$, then clearly $I(\hat{\varvec{\beta }}^T \varvec{X}_{ij} > 0) = I(\varvec{\beta }_0^T \varvec{X}_{ij} > 0)$.

If $\varvec{X}_{ij} \ne 0$, then using the consistency of $\hat{\varvec{\beta }}$ and assumption (A.5), for n sufficiently large, $|\hat{\varvec{\beta }}^T \varvec{X}_{ij}| > \nu > 0$ and $I(\hat{\varvec{\beta }}^T \varvec{X}_{ij} > 0) = I(\varvec{\beta }_0^T \varvec{X}_{ij} > 0)$ for all $\varvec{X}_{ij}$ . Therefore, for n large,

$$\begin{aligned} n^{1/2} K_{n,E}\big (\hat{\varvec{\beta }}\big ) = n^{1/2} \tilde{K}_{n,E}\big (\hat{\varvec{\beta }}\big ). \end{aligned}$$

A Taylor expansion for the asymptotically equivalent CPE produces

$$\begin{aligned} n^{1/2} \left[ \tilde{K}_{n,E}\big (\hat{\varvec{\beta }}\big ) - {\mathcal {C}}_E \right]= & {} n^{1/2} \left[ \tilde{K}_{n,E}\big (\varvec{\beta }_0\big ) - {\mathcal {C}}_E \right] \\&+ \left( \frac{\partial \tilde{K}_{n,E}}{\partial \varvec{\beta }}\right) ^\text {T} \left[ n^{1/2}\big (\hat{\varvec{\beta }} - \varvec{\beta }_0\big )\right] + o_p(1) . \end{aligned}$$

The partial derivative $\partial \tilde{K}_{n,E}/\partial \varvec{\beta }$ is asymptotically constant. Since $n^{1/2}(\hat{\varvec{\beta }} - \varvec{\beta }_0)$ has asymptotic mean zero conditional on $\varvec{X}$, it is asymptotically independent of $n^{1/2} \left[ \tilde{K}_{n,E}(\varvec{\beta }_0) - {\mathcal {C}}_E \right]$. Therefore the asymptotic variance of $\tilde{K}_{n,E}(\hat{\varvec{\beta }})$ is

$$\begin{aligned} \text{ Var }\bigg \{ n^{1/2} \left[ \tilde{K}_{n,E}\big (\varvec{\beta }_0\big ) - {\mathcal {C}}_E \right] \bigg \} + \left( \frac{\partial \tilde{K}_{n,E}}{\partial \varvec{\beta }}\right) ^\text {T} \text{ Var }[\hat{\varvec{\beta }}] \left( \frac{\partial \tilde{K}_{n,E}}{\partial \varvec{\beta }} \right) . \end{aligned}$$

The individual components of the asymptotic variance can be estimated as follows. In each case, the substitution of $\hat{\varvec{\beta }}$ for $\varvec{\beta }_0$ provides a consistent estimate.

The $\text{ Var }(\hat{\varvec{\beta }})$ is estimated from the inverse of the second derivative of the partial likelihood.

The partial derivative evaluated at $\varvec{\beta }_0$ is equal to

$$\begin{aligned} \frac{\partial \tilde{K}_{n,E}(\varvec{\beta })}{\partial \varvec{\beta }} = - \frac{ \sum _i \sum _j I\Big (\varvec{\beta }_0^T \varvec{X}_{ij} > 0\Big ) \varvec{X}_{ji} \exp \Big (\varvec{\beta }_0^T \varvec{X}_{ji}\Big ) \left[ 1+\exp \Big (\varvec{\beta }_0^T \varvec{X}_{ji}\Big )\right] ^{-2} }{\sum _i \sum _j I\Big (\varvec{\beta }_0^T \varvec{X}_{ij} > 0\Big )}. \end{aligned}$$

The first term, $n^{1/2} \left[ \tilde{K}_{n,E}(\varvec{\beta }_0) - {\mathcal {C}}_E \right]$, may be approximated by the U-statistic

$$\begin{aligned} \pi ^{-1} n^{-3/2} \sum _i \sum _j I\Big (\varvec{\beta }_0^T \varvec{X}_{ij} > 0\Big ) \left\{ \left[ 1+\exp \Big (\varvec{\beta }_0^T \varvec{X}_{ji}\Big )\right] ^{-1} - {\mathcal {C}}_E \right\} \end{aligned}$$

where $\pi = \lim _{n \rightarrow \infty } n^{-2} \sum _i \sum _j I(\varvec{\beta }_0^T \varvec{X}_{ij} > 0)$.

The asymptotic variance of this U-statistic is

$$\begin{aligned} \pi ^{-2} n^{-3} \sum _i \sum _j \sum _{k \ne j} \left[ u_{ij} + u_{ji} \right] \left[ u_{ik} + u_{ki}\right] . \end{aligned}$$

where

$$\begin{aligned} u_{ij} = I\Big (\varvec{\beta }_0^T \varvec{X}_{ij} > 0\Big ) \left\{ \left[ 1+\exp (\varvec{\beta }_0^T \varvec{X}_{ji})\right] ^{-1} - {\mathcal {C}}_E \right\} . \end{aligned}$$

Combining these results provides the estimated asymptotic variance of the CPE excluding ties.

Asymptotic distribution of $\ n^{1/2} \left[ C_{n,E}\big (\hat{\varvec{\beta }}\big ) - {\mathcal {C}}_E \right]$

From Sect. 3.2, this expression may be written as

$$\begin{aligned}&\frac{ n^{-3/2} \sum _i \sum _j \delta _i I\big (y_i \!<\! y_j\big ) I\Big (\hat{\varvec{\beta }}^T\varvec{X}_i \!\ne \! \hat{\varvec{\beta }}^T\varvec{X}_j\Big ) \left\{ \hat{G}\Big (y_i|\hat{\varvec{\beta }}^T\varvec{X}_i\Big ) \hat{G}\Big (y_i|\hat{\varvec{\beta }}^T\varvec{X}_j\Big )\right\} ^{-1} \left[ I\Big (\hat{\varvec{\beta }}^T\varvec{X}_i \!>\! \hat{\varvec{\beta }}^T\varvec{X}_j\Big ) - {\mathcal {C}}_E \!\right] }{ n^{-2} \sum _i \sum _j \delta _i I(y_i \!<\! y_j) I(\hat{\varvec{\beta }}^T\varvec{X}_i \!\ne \! \hat{\varvec{\beta }}^T\varvec{X}_j) \left\{ \hat{G}(y_i|\hat{\varvec{\beta }}^T\varvec{X}_i)\hat{G}(y_i|\hat{\varvec{\beta }}^T\varvec{X}_j)\right\} ^{-1}}\\&\quad = \psi ^{-1} n^{-3/2} \sum _i \sum _j \frac{\delta _i I\big (y_i < y_j\big ) I\Big (\varvec{\beta }_0^T\varvec{X}_i \ne \varvec{\beta }_0^T\varvec{X}_j\Big ) \left[ I\Big (\varvec{\beta }_0^T\varvec{X}_i > \varvec{\beta }_0^T\varvec{X}_j\Big ) - {\mathcal {C}}_E \right] }{G\Big (y_i|\hat{\varvec{\beta }}^T\varvec{X}_i\Big ) G\Big (y_i|\hat{\varvec{\beta }}^T\varvec{X}_j\Big )} \ + o_p(1) \end{aligned}$$

where $\psi = \lim _{n \rightarrow \infty } n^{-2} \sum _i \sum _j \delta _i I(y_i < y_j) I(\hat{\varvec{\beta }}^T\varvec{X}_i \!\ne \! \hat{\varvec{\beta }}^T\varvec{X}_j) \hat{G}^{-1}(y_i|\hat{\varvec{\beta }}^T\varvec{X}_i)\hat{G}^{-1}(y_i|\hat{\varvec{\beta }}^T\varvec{X}_j)$.

Letting

$$\begin{aligned} e_{ij} = \delta _i I\big (y_i < y_j\big ) I\Big (\varvec{\beta }_0^T\varvec{X}_i \ne \varvec{\beta }_0^T\varvec{X}_j\Big ) \left[ I\Big (\varvec{\beta }_0^T\varvec{X}_i > \varvec{\beta }_0^T\varvec{X}_j\Big ) - {\mathcal {C}}_E \right] \end{aligned}$$

and Taylor expanding $\hat{\varvec{\beta }}$ around $\varvec{\beta }_0$,

$$\begin{aligned}&n^{1/2} \left[ C_{n,E}\big (\hat{\varvec{\beta }}\big ) - {\mathcal {C}}_E \right] \\&\quad = \psi ^{-1} n^{-3/2} \sum _i \sum _j \frac{e_{ij}}{G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_i\Big )G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_j\Big )} \\&\qquad +\,\psi ^{-1} n^{-3/2} \sum _i \sum _j \frac{e_{ij}}{G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_i\Big )G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_j\Big )}\\&\qquad \times \left[ \frac{G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_i\Big ) - \hat{G}\Big (y_i|\varvec{\beta }_0^T\varvec{X}_i\Big )}{G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_i\Big )} + \frac{G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_j\Big ) - \hat{G}\Big (y_i|\varvec{\beta }_0^T\varvec{X}_j\Big )}{G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_j\Big )}\right] \\&\qquad +\psi ^{-1} n^{-3/2} \sum _i \sum _j \left\{ \left[ \frac{\partial }{\partial \varvec{\beta }} \frac{e_{ij}}{G\Big (y_i|\varvec{\beta }^T\varvec{X}_i\Big )G\Big (y_i|\varvec{\beta }^T\varvec{X}_j\Big )}\right] ^T \right\} \Big (\hat{\varvec{\beta }} - \varvec{\beta }_0\Big ) + o_p(1) \end{aligned}$$

The second term may be rewritten using the martingale representation theorem as

$$\begin{aligned} 2 n^{-1/2} \sum _{k=1}^n \int _{t=0}^{\infty } \frac{q_{\varvec{X}_k}(t)}{\theta _{\varvec{X}_k}(t)} dM_{\varvec{X}_k}(t) \end{aligned}$$

where

$$\begin{aligned} q_{\varvec{X}_k}(t)= & {} \lim _{n \rightarrow \infty } \sum _i \sum _j \left\{ \frac{e_{ij}}{G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_i\Big )G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_j\Big )} \right\} \\&\left\{ I\Big (y_j \ge t\Big ) \left[ I\Big (\varvec{X}_i = \varvec{X}_k\Big ) + I\Big (\varvec{X}_j = \varvec{X}_k\Big ) \right] \right\} \\ \theta _{\varvec{X}_k}(t)= & {} \lim _{n \rightarrow \infty } \sum _j I\Big (y_j \ge t, \varvec{X}_j = \varvec{X}_k\Big )\\ M_{\varvec{X}_k}(t)= & {} I\Big (y_k \le t, \delta _k = 0\Big ) - \int _{u=0}^t I\Big (y_k \ge u\Big )d\Lambda _{\varvec{X}_k}(u) \end{aligned}$$

and $\Lambda _{\varvec{X}_k}$ is the cumulative hazard of the censoring random variable belonging to group $\varvec{X}_k$ (Cheng et al. 1995; Fine et al. 1998).

It follows that $n^{1/2} \left[ C_{n,E}(\hat{\varvec{\beta }}) - {\mathcal {C}}_E \right]$

$$\begin{aligned}&= \psi ^{-1} n^{-3/2} \sum _i \!\sum _j \frac{e_{ij}}{G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_i\Big )G\Big (y_i|\varvec{\beta }_0^T\varvec{X}_j\Big )}\\&\quad +\, 2 \psi ^{-1} n^{-1/2} \sum _{k=1}^n \int _{t=0}^{\infty } \frac{q_{\varvec{X}_k}(t)}{\theta _{\varvec{X}_k}(t)} dM_{\varvec{X}_k}(t)\\&\quad + \,\psi ^{-1} \left\{ n^{-2} \sum _i \sum _j\left[ \!\frac{\partial }{\partial \varvec{\beta }} \frac{e_{ij}}{G\Big (y_i|\varvec{\beta }^T\varvec{X}_i\Big )G\Big (y_i|\varvec{\beta }^T\varvec{X}_j\!\Big )}\!\right] ^T \!\right\} n^{1/2}\Big (\hat{\varvec{\beta }}\!-\!\varvec{\beta }_0\Big )\!+ o_p(1) \end{aligned}$$

Each component is asymptotically normal with mean zero. Analytic estimation of the asymptotic variance, however, requires density estimation of the censoring random variable. Alternatively, a stratified bootstrap resampling approach is utilized to estimate the asymptotic variance.