Testing for differences in distribution tails to test for differences in 'maximum' lifespan

Consider an experiment with two groups, treatment and control. The extension to more than two groups is straightforward (see discussion section). Let X be an indicator variable taking the value 1 for observations in the treatment group and 0 for observations in the control group. Let Y denote survival time. Let τ denote some threshold chosen by the investigator to denote an extreme portion of the distribution. In survival studies, τ can be chosen in advance to correspond to an age considered 'old' (e.g., 30 months in mice) or set to some high sample percentile (e.g., the 90th). Critically important, τ must be set to the same value for the two groups. That is, if τ is to be defined by an upper sample quantile, it should be the upper sample quantile of both of the two groups combined, not of each group separately.

Although not described in exactly these terms in the paper by Wang et al. [5], the Wang-Allison tests essentially create a new variable, W, where for the i^th subject, W _i ≡ 0 if Y _i ≤ τ, and W _i ≡ 1 if Y _i > τ, and subsequently tests whether W is associated with X using an appropriate test statistic.

Thus, the Wang-Allison tests test the following null hypothesis:

H _0,A : P (Y > τ|X = 1) = P(Y > τ|X = 0).

A problem with the Wang-Allison tests is that, hypothetically, P (Y > τ|X = 1) may equal P (Y > τ|X = 0) and yet the average magnitude by which lifespans exceed τ when X = 1 may be radically different than when X = 0. This is exemplified in the hypothetical frequency distributions depicted in Figure 1. Note that these hypothetical distributions are not intended to be realistic, but only to clarify the point.

Let X ¹ and X ⁰ denote the numbers of observations with Y _i > τ in the treatment group and control group, respectively. The Wang-Allison tests use the test procedures for two independent binomial proportions [7] and these procedures require that X ¹ and X ⁰ are independent. In the Wang-Allison tests, if the threshold is set in advance according to prior knowledge, X ¹ and X ⁰ can satisfy the requirement of independence. But if τ is set to be the 90-th percentile, X ¹ and X ⁰ may not be independent, this creates a theoretical problem. However, on an empirical level, our simulations show that in the sample sizes we considered, this is not an apparent problem because the Wang-Allison tests have very high power and can control type I error quit well in the simulation studies and are practical for the lifespan studies). When X ¹ and X ⁰ are not independent, simulation studies (including estimation of power and type I error) are an effective way to evaluate the methods (such as Wang-Allison tests) using the test procedures for two independent binomial proportions.

An alternative to testing H _0,A is to test the following conceptually related but mathematically distinct null hypothesis:

H _0,B : μ (Y|Y > τ ∩ X = 1) = μ (Y|Y > τ ∩ X = 0),

where μ (•) denotes the population mean (or expectation) of (•). Though appealing, a problem with testing H _0,B is that when P (Y > τ|X = 1) >> P (Y > τ|X = 0) or P (Y > τ|X = 1) <<P (Y > τ|X = 0), for any finite sample with equal initial assignment to the two groups, E [n ₀] <<E [n ₁] or E [n ₀] >> E [n ₁], where E [n ₀] denotes the expected number of observations in the control group for which Y > τ, and E [n ₁] denotes the expected number of observations in the treatment group for which Y > τ. This imbalance between E [n ₀] and E [n ₁] will greatly reduce the power to reject H _0,B . In fact, in the extreme, when either P (Y > τ|X = 1) or P (Y > τ|X = 0), there will be zero power to reject H _0,B (actually, it is appropriate to say that H _0,B is undefined in such cases). Such a situation is exemplified in the hypothetical frequency distributions depicted in Figure 2. Again, these hypothetical distributions are not intended to be realistic, but only to clarify the point.

Thus, one can conceive situations in which the power to reject H _0,A will be zero and yet the upper tails of the distribution are clearly different. Similarly, one can conceive situations in which the power to reject H _0,B will be zero and yet again the upper tails of the distribution are clearly different. Hence, we propose a single-step union-intersection test [8] of the following compound null hypothesis:

H _0,C : [P(Y > τ|X = 1) = P(Y > τ|X = 0)] ∩ [μ(Y|Y > τ ∩ X = 1) = μ (Y|Y > τ ∩ X = 0)].

We construct the test of H _0,C with the following simple procedure. Define a new variable Z such that Z _i ≡ I(Y _i > τ)Y _i , where I(•) denotes the indicator function taking on values of one if (•) is true and zero otherwise. One can then simply conduct an appropriate test (several candidates will be considered below) of whether the population mean of Z is significantly different between the treatment and control groups. This approach (hereafter new tests), has several desirable properties. First and foremost, when an appropriate test statistic is used, the approach will be valid. That is, unlike the conditional t-tests (CTTs) commonly used and shown to be invalid by Wang et al. [5], when H _0,C is true, it will only be rejected 100*α% of the time at the nominal α level even if f(Y|Y ≤ τ ∩ X = 1) ≠ f(Y|Y ≤ τ ∩ X = 0), where f(•) denotes the probability density function of (•).

Note that expectation (or population mean) of Z, μ(Z) = P(Y > τ) μ(Y | Y > τ). Therefore the new test for H _0,C is really testing whether

P(Y > τ | X = 1) μ (Y | Y > τ ∩ X = 1) = P(Y > τ | X = 0) μ (Y | Y > τ ∩ X = 0),

while the method for H _0,B is testing whether μ (Y | Y > τ ∩ X = 1) = μ(Y | Y > τ ∩ X = 0) and the method for H _0,A is testing whether P(Y > τ | X = 1) = P(Y > τ | X = 0). The mean difference of μ(Z) between two groups consists of two components: the difference between probabilities P(Y > τ | X = 1) and P(Y > τ | X = 0) and the difference between expectations μ (Y | Y > τ ∩ X = 1) and μ (Y | Y > τ ∩ X = 0). The test for H _0,A focuses on the first component and the test for H _0,A focuses on the second one, while the test for H _0,C is related to both components.

We also note that Dominici and Zeger [9] studied similar mean difference components for two groups (cases and controls) by estimating the mean difference Δ(v) for the two groups conditional on a vector of covariates vfor zero-inflated data through smooth quantile ratio estimation with regression,

Δ(v) = P(Y > 0 | X = 1, v) μ (Y | Y > 0, X = 1, v) - P(Y > 0| X = 0, v) μ (Y | Y > 0, X = 0, v),

where, Y is nonnegative random variable denoting the health expenditures. While Dominici and Zeger [9] estimate the mean difference of nonnegative random variables (Y) for two groups, our methods test the mean difference of random variables (Y) which are greater than threshold τ.

We evaluate the tests via computer simulation. For each scenario simulated, we evaluate the tests at the 2-tailed .05 α level and at the 2-tailed .01 α level using 5,000 simulated datasets per scenario (except for permutation tests where we use 1,000 datasets per scenario and 1,000 random permutations by Monte Carlo sampling for each dataset). In simulation 1, we first evaluate performance in simulation under the null hypothesis H _0,C (i.e., both H _0,A and H _0,B are true) and yet f (Y|Y ≤ τ ∩ X = 1) is radically different from f (Y|Y ≤ τ ∩ X = 0). After showing that the tests remain valid even in these extreme circumstances, we compare their power in several scenarios (simulations 2–4) described below. For each scenario, we assumed that there were two groups with an equal number of subjects per group. We ran scenarios with 50, 80, or 100 subjects in each of the two groups, realistic sample sizes for animal model longevity research.

We simulated data using a concatenation of Weibull distributions to flexibly emulate the data observed in a real study [10] of obese animals (control; X = 0) versus animals that were obese and then lost weight via CR (treatment; X = 1). Specifically, For example, in simulations 1–4, we simulated Y from the following distribution:

where j = 0 to 1, lifespan (Y) is measured in weeks, a_j,L and b_i,L are the parameters of a Weibull distribution for the lower 90% of the distribution, and a_j,U and b_i,U are the parameters of a Weibull distribution for the upper 10% of the distribution. r _j is a proportion parameter, for example r _j = 0.9. The specific values of the parameters used are provided in Figure 3.

Each of the tests listed below was implemented in two manners, first with τ set in advance to a fixed lifespan value (130 weeks), and second with τ set at the sample 90^th percentile of the two groups combined. In real-life situations, one usually does know the threshold of interest a priori. We do recognize that we will not have such knowledge in all cases. It is for this reason that when analyzing the simulated data, we also consider a threshold of the 90^th percentile of the data allowing for an ad hoc data-based determination of a threshold.

To illustrate the methods, we applied them to two real datasets. In both of these datasets, prior research had shown differences in overall survival rate and we tested for differences in 'maximum lifespan' herein. The first was a subset of data reported by Vasselli et al [10]. The subset of the data consists of two groups of Sprague-Dawley rats, those kept on a high-fat diet ad libitum throughout life and becoming obese (EO-HF) and those kept on a high-fat diet ad libitum until early-middle adulthood, becoming obese, and subsequently reduced to normal weight via caloric restriction, but on the same high-fat diet (WL-HF). Each group had 49 rats (see Figure 4 for the histograms for the data). The second dataset was from a study comparing the lifespan of Agouti-related protein-deficient (AgRP(-/-)) mice to wildtype mice (+/+) as reported by Redmann & Argyropoulos [14]. This dataset consists of 16 mice with genotype '+/+' and 21 mice with genotype '-/-' (see Figure 5 for the histograms for this dataset). From Figure 4, we can see the upper tails of the histograms of the two groups are different. Similar results can be found in Figure 5.

Results (p values of tests) are shown in Table 6. As can be seen, when setting τ equal to 110 (100) for the first (second) datasets, both the methods for tests of H _0,A and the new methods for tests of H _0,C can detect the differences in 'maximum lifespan' between groups at nominal alpha levels of 0.01 (0.05) for the first (second) datasets. But the methods for tests of H _0,B cannot detect the difference for all different values of τ . The following description may provide some explanation to these results. For the first dataset, when set τ = 110, the proportions of the observations greater than τ in the EO-HF group and WL-HF group (i.e., estimations of P(Y > τ | X = 0) and P(Y > τ | X = 1)) are 0.061 and 0.306, respectively. These two proportions are significantly different and not surprisingly, the methods for tests of H _0,A can detect the difference in 'maximum lifespan' between the two groups. Second, the sample means of the observations greater than τ in the two groups (i.e., estimations of μ (Y | Y > τ ∩ X = 1) and μ (Y | Y > τ ∩ X = 0)) are 117.8 and 122.9, respectively, and there is no much difference between these sample means. However the sample means of the Z-values in the two group (i.e., the estimations of P(Z | X = 0) and P(Z | X = 1)) are 7.210 and 37.633, respectively, and are greatly different, where, Z _i ≡ I(Y _i > τ)Y _i . These may explain that the methods for tests of H _0,B cannot reject the null but the new methods for tests of H _0,C can detect the difference in 'maximum lifespan' between the two groups. Similarly, for the second dataset, when set τ = 100, the proportions of the observations greater than τ in the group with genotype '+/+' and group with genotype '-/-' are 0.188 and 0.571, respectively. The sample means of the observations greater than τ in the two groups are 109.3 and 110.9, respectively. The sample means of the Z-values in the two groups are 20.5 and 63.4 respectively.

From Table 6 we can also see that in almost all situations the p-values of the new methods for tests of H _0,C are somewhat smaller than those of the methods for tests of H _0,A . This is consistent with the simulations showing greater power of the new methods.