Analysis of the Natural History of Dementia Using Longitudinal Grade of Membership Models

Abstract

We present a longitudinal form of the Grade of Membership (GoM) model for time-varying covariates, provide a self-contained description of its estimation, and illustrate its application with a substantively meaningful analysis of the progression of dementia among National Long Term Care Survey (NLTCS) respondents. The chapter has two goals—one methodological and the other substantive. Methodologically, we present the Kuhn-Tucker conditions for convergence of the maximum likelihood estimator and show how the associated estimates can be obtained using a new constrained form of the Newton-Raphson iteration algorithm that preserves the summation constraints at each update; we also present and discuss known results regarding the consistency and asymptotic normality of the longitudinal GoM model and offer a conjecture regarding how these results might be extended to the less restrictive cross-sectional form of the GoM model. Substantively, the natural history of dementia is modeled as a complex irreversible multidimensional process governed by a latent three-dimensional bounded state-space process. Individual dementia cases are initially widely dispersed in the latent state space. Over time, they move to state-space locations associated with severe cognitive and physical impairment and dramatically increased need for care. The application to the NLTCS data has been independently validated using Alzheimer’s disease data from the two cohorts of the Predictors Study.

Appendix

1.1 Synthesis of Known Results Regarding the Consistency of the General (Cross-Sectional) Empirical GoM Model

Haberman’s (1995) challenge was to identify conditions under which the conditional GoM likelihood estimator is or is not consistent; he cautioned that this would be “very difficult” to do (Haberman 1995, p. 1132). Wachter (1999) responded to this challenge by recasting the conditional GoM model in a framework based on concepts of dimensionality reduction; he commented that the theorems needed to respond directly to Haberman’s consistency challenge were not “readily available” so that a direct response would take one “quickly into uncharted territory” (Wachter 1999, p. 441).

We agree with Haberman about the challenge being very difficult but we disagree with Wachter that the territory is almost completely uncharted. Indeed, our review of the statistical literature indicates that the existing theorems are highly informative. They allow unambiguous determination of consistency/inconsistency for most forms of GoM and they provide substantial insight into the issues to be resolved in such determinations. We emphasize that, for cases where the maximum likelihood estimator is inconsistent, the existing theorems (e.g., Huber 1967, Theorem 1) indicate that the resulting maximum likelihood estimates are likely to be optimal (or approximately so) in the sense that they are the closest possible to the true parameter values using the Kullback-Leibler information criterion (i.e., relative entropy, or divergence) as the distance measure (Kullback and Leibler 1951). As explained below, we conjecture that a generalization of Huber’s (1967) Theorem 1 could apply to a form of GoM in which the empirical GoM-score mixing distribution is used to create an empirical marginal likelihood with the same set of λ- and g-parameters as in conditional GoM. If proven true, this would move the empirical cross-sectional GoM model into the mainstream statistical literature, thereby extending the range of applications far beyond the dimensionality-reduction applications considered by Wachter (1999).

We summarize the relevant literature, existing theorems, and implications for consistency in the form of 15 observations. Our goal is to document the findings in one accessible location and to stimulate further work on proving our conjecture:

• 1. Wald (1948, Theorem 2.1) provided necessary and sufficient conditions for the existence of a uniformly consistent estimator of a parameter (like λ; but scalar in Wald’s case) in the presence of a set of vectors of incidental parameters (like the g-parameters). Tolley and Manton’s (1992) proof of consistency for the marginal GoM likelihood for fixed J implies that conditional GoM meets Wald’s existence conditions. Thus, marginal GoM was shown to be consistent; the consistency of conditional GoM was not addressed.

• 2. Tolley and Manton’s (1992) marginal GoM likelihood is difficult to use in practice because it requires one to specify the form of the mixing distribution and this is generally unknown. Indeed, one major reason for performing a GoM analysis is to discover the form of this mixing distribution.

• 3. Kovtun et al. (2007) commented that an empirical estimator of the mixing distribution of the g-parameters can be formed directly from the estimates of the GoM scores with each individual providing a unit contribution to the histogram of the mixing distribution; in this case, they claimed that the empirical distribution converges to the true mixing distribution as J, along with N, goes to infinity. However, they did not provide a proof of convergence for this case. We refer to the marginal GoM likelihood using the empirical estimator in place of the true mixing distribution as the empirical marginal GoM likelihood.

• 4. Mak (1982, Theorem 2.1) implies that conditional and empirical marginal GoM likelihoods with fixed J and increasing N will yield estimators that converge to points in the λ-parameter space that generally differ from the true λ-parameter values; hence the associated estimators are not consistent.

• 5. Mak (1982, Theorem 2.1) also implies that conditional and empirical marginal GoM likelihoods with increasing J but fixed N will yield estimators that converge to points in the g-parameter space that generally differ from the true g-parameter values. Thus, the associated estimators are also not consistent for this case.

• 6. It follows that N and J must both go to infinity for consistency to be established for the conditional and empirical marginal GoM estimators; in this case, if the empirical mixing distribution converges to the true mixing distribution, then the empirical marginal GoM likelihood estimator will yield consistent estimates of the λ- and g-parameters, without prior specification of the form of the mixing distribution.

• 7. Substantial insight into the empirical marginal GoM model can be gained by letting J go to infinity first, and then considering the behavior of the model as N goes to infinity. Letting J go to infinity means that each observed data vector, x _i , follows a multinomial distribution with an uncountably infinite number of cells representing all combinations of response outcomes for a countably infinite number of variables (see Feller 1971, p. 123, Theorem 1). Hence, under the general GoM model, each cell, c, will have a probability ${\pi}_{ic}=\underset{J\to \infty }{ \lim }{\displaystyle \prod_{j=1}^J{\displaystyle \sum_{k=1}^K{g}_{ik}{\lambda}_{kj{l}_{jc}}}}$ for each individual i and a marginal probability ${\pi}_c^0=E\left({\pi}_{ic}\right)$ in the population, where the expectation is taken with respect to the GoM-score distribution. Thus, as J goes to infinity the set {π ⁰ _c } defines a multinomial distribution with a countably infinite number of λ- and g-parameters and an uncountably infinite number of cells (i.e., “points”). The presence of an uncountably infinite number of cells introduces several technical problems in defining countably additive probability measures for this distribution, but standard solutions are well-known (e.g., see Billingsley 1986). Three properties of this distribution are relevant: (1) the observed frequency distribution provides a consistent unrestricted (i.e., nonparametric) maximum likelihood estimator of {π ⁰ _c } (by a generalization of the Glivenko-Cantelli Theorem; Wellner (1981, Theorem 1)—see Gaenssler and Wellner (1981) for discussion); (2) the entropy of {π ⁰ _c } becomes infinite (because variable-specific entropies are additive over j, and J becomes infinite) (Cover and Thomas 1991, Theorem 2.6.6); and (3) the distribution {π ⁰ _c } becomes continuous (because, at the limit, no cell c carries positive probability mass; see Feller 1971, p. 137–138).

• 8. The conditional GoM likelihood is an “empirical estimator” in the sense that the GoM scores are directly represented via the g-parameters without consideration of a mixing distribution, and more importantly, without prior specification of the form of the mixing distribution. It can be shown that the empirical marginal GoM likelihood for fixed N and increasing J converges to a form proportional to the conditional GoM likelihood. Hence, the estimates under the conditional GoM likelihood will converge to a limit point as J goes to infinity that is the same as that of the estimates under the empirical marginal GoM likelihood: if the empirical marginal GoM likelihood estimator is consistent for infinite J, then the conditional GoM likelihood estimator will be likewise consistent. This convergence property implies that the conditional GoM likelihood estimator will provide a good approximation to the empirical marginal GoM likelihood estimator for large J.

• 9. Rao (1958, Assumption A₁) provided a sufficient condition for the uniform consistency of the restricted (i.e., parametric) maximum likelihood estimator for the infinite multinomial distribution as N goes to infinity: the entropy of the distribution {π ⁰ _c } must be finite. Rao (1958) emphasized that while this condition is not necessary for consistency, it is sufficient. Unfortunately, as noted in Observation 7, this condition does not hold for GoM. Nonetheless, given that the unrestricted maximum likelihood estimator is known to be consistent for the infinite multinomial distribution as N goes to infinity, it at least seems plausible that the restricted maximum likelihood estimator may also be consistent.

• 10. To complete our synthesis, we refer again to Mak (1982, Theorem 2.1), from which it follows that the restricted maximum likelihood estimator for GoM will converge to some point in the λ-,g-parameter space. What point is that? Mak’s (1982) Theorem 2.1 does not provide an answer, but it is highly likely that Huber’s (1967) Theorem 1 does, and if it does, then: it will converge to the unique point in the λ-,g-parameter space that minimizes the relative entropy (i.e., Kullback-Leibler divergence) between the restricted and unrestricted models.

• 11. See McCulloch (1988) and Freedman (2006) for non-technical discussion of this result. Note that the use of relative entropies resolves the “problem” in Observations 7 and 9 that the entropy of {π ⁰ _c } becomes infinite for both the restricted and unrestricted models. If the restricted GoM model is true (i.e., is the correct model), then the proof of a generalized form of Huber’s (1967) Theorem 1 will need to identify the conditions under which the parameter estimates for the restricted model converge to the true values as N and J go to infinity.

• 12. These must be the same values as obtained for the unrestricted model.

• 13. As written, Huber’s (1967) assumptions for his Theorem 1 require that a sequence of maximum likelihood estimators can be formed for each N as N goes to infinity: any sequence will do. No consideration, however, was given to forming a second asymptotic sequence for J as J goes to infinity. Such consideration would clearly require some generalization of Huber’s assumptions, which is what is needed to prove our conjecture. This would only work if, in fact, such sequences exist. Thus, we need to consider how at least one sequence of maximum likelihood estimators could be formed for combinations of N and J such that both N and J could go to infinity.

• 14. Before doing so, we first need to note that Wald (1948, Theorem 3.1) provided an additional condition for consistency which implies that the total amount of Fisher information in the empirical marginal GoM model must go to infinity as N and J go to infinity. Second, Kovtun et al. (2014, Theorem 5.4) provided additional conditions that restrict the set of admissible variables to those that yield identifiable mixture distributions with a property that they term “∞-stability.” Intuitively, this restricts the set of admissible variables to some well-defined measurement domain, which should not be a serious restriction for most substantive applications. We assume that these conditions can be met, in theory, by selecting cases and variables such that the associated Hessian matrices (i.e., the “observed” Fisher information matrices) converge to block diagonal form with unbounded positive-definite diagonal blocks as N and J go to infinity. Hence, we assume that a sequence of maximum likelihood estimators that satisfy the requirements of Observation 13 can be formed using the algorithm in Sect. 17.2.8, which is justified by the convergence property in Observation 8, by letting N and J increase in fixed ratios with variables selected so that the diagonal terms of the Hessian matrices are unbounded. Convergence to block diagonal form as N and J increase follows from the structure of Eqs. (17.12) and (17.13): the only nonzero terms outside the diagonal blocks correspond to the cross-derivatives of the λ- and g-parameters and these contain only one additive term, independent of N and J. Hence, the relative sizes of these cross-derivatives will tend to zero as N and J go to infinity. Convergence to positive-definite diagonal blocks will satisfy Kuhn-Tucker Condition 5; the inverse Hessian matrices will be used in eqn. (17.36). Akaike (1973, p. 269–270) showed the close connections between the Hessian matrix, the Fisher information matrix, and the relative entropy measures used to develop the AIC and Kullback-Leibler statistics.

• 15. The method described in Observation 14 allows one to construct a sequence of joint λ- and g-estimators for N and J that permit the conditions of Huber’s (1967) Theorem 1 to be extended from one to two sequences. If needed, one can ensure by setting N = J that the terms of the paired sequences use just a single index, say N, which would most closely match the existing form of Huber’s assumptions. It remains to provide a precise specification of conditions for the asymptotic convergence of these joint sequences and to rigorously determine the changes needed in each step of Huber’s proof. Given that the unrestricted maximum likelihood estimator is known to be consistent (Wellner 1981), we expect that it will be possible to specify such conditions; this expectation forms the basis of our conjecture. An essential part of Huber’s proof is the assumption that the restricted maximum likelihood estimates are unique; Mak (1982, Theorem 2.1) provides conditions that justify this assumption for the GoM model and these could be incorporated into the generalized Huber theorem. Then, if the GoM model is true, the uniqueness of the limit point would ensure that the restricted and unrestricted maximum likelihood estimators would both tend to the same limit as N and J go to infinity, in which case consistency would be proven. The Kullback-Leibler divergence would converge to zero under these same conditions. For the case where the GoM model is true but one of N or J is not infinite, it would follow from the original Huber argument that the Kullback-Leibler divergence would be minimized, confirming our conjecture in its entirety.

The above synthesis clarifies Manton et al.’s (1994, p. 24) statement that the parameters obtained using the conditional GoM likelihood estimator “asymptotically maximize” the marginal GoM likelihood. Further work is needed to generalize Huber’s (1967) Theorem 1 to a form directly applicable to the empirical marginal GoM estimator and to determine for practical applications how large (or small) J needs to be for the approximation in Observation 8 to apply to conditional GoM for given sizes of N and configurations of the empirical mixing distribution. Chapter 18 (Sect. 18.4.3) discusses alternative approaches based on linear latent structure (LLS) analysis to establishing conditions for consistent and asymptotically normal estimators of the λ- and g-parameters for the conditional GoM likelihood—suggesting that the consistency issue can best be completely resolved by several modes of attack. Kovtun et al. (2007) showed that J-values in the range 250–1000 were sufficient to obtain good estimates of the empirical mixing distributions for the generalized GoM scores used in the LLS model for N = 10,000. This suggests that J-values substantially below this range may have acceptable performance characteristics when considered in the context of the associated Kullback-Leibler divergences.