Abstract
We give a large family of simple examples where a sharp analysis of the Gibbs sampler can be proved by coupling. These examples involve standard statistical models — exponential families with conjugate priors or location families with natural priors. Our main approach uses a single eigenfunction (always explicitly available in the examples in question) and stochastic monotonicity. We give a satisfactory treatment of several examples that have defeated previous attempts at analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aldous, D. and Diaconis, P. (1986). Shuffling cards and stopping times. Amer. Math. Monthly, 93, 333–348.
Aldous, D. and Diaconis, P. (1987). Strong uniform times and finite random walks. Adv. in Appl. Math., 8, 69–97.
Andersen, H. and Diaconis, P. (2007). Hit and run as a unifying device. J. Soc. Fr. Stat. & Rev. Stat. Appl., 148, 5–28.
Athreya, K., Doss, H. and Sethuraman, J. (1996). On the convergence of the Markov chain simulation method. Ann. Statist., 24, 89–100.
Barankin, E.W. and Maitra, A.P. (1963). Generalization of the Fisher-Darmois-Koopman-Pitman theorem on sufficient statistics. Sankhyā, Ser. A, 25, 217–244.
Berti, P., Consonni, G. and Pratelli, L. (2009). Discussion on the paper: Gibbs sampling, exponential families and orthogonal polynomials. Statist. Sci., 23, 179–182.
Brown, L.D., Johnstone, I.M. and McGibbon, K.B. (1981). Variance diminishing transformations: A direct approach to total positivity and its statistical applications. J. Amer. Statist. Assoc., 76, 824–832.
Casella, G. and George, E. (1992). Explaining the Gibbs sampler. Amer. Statist., 46, 167–174.
Diaconis, P. (1988). Group Representations in Probability and Statistics. Institute of Mathematical Statistics — Monograph Series, 11. Institute of Mathematical Statistics, Hayward, California.
Diaconis, P. and Fill, J. (1990). Strong stationary times via a new form of duality. Ann. Probab., 16, 1483–1522.
Diaconis, P. and Fulman, J. (2008). Carries, shuffling and an amazing matrix. Amer. Math. Monthly, 116, 788–803.
Diaconis, P. and Fulman, J. (2009). Carries, shuffling, symmetric function. Adv. In Appl. Math., 43, 176–196.
Diaconis, P., Khare, K. and Saloff-Coste L. (2008). Gibbs sampling, exponential families and orthogonal polynomials. Statist. Sci., 23, 151–178.
Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. Ann. Statist., 7, 269–281.
Diaconis, P. and Ylvisaker, D. (1985). Quantifying prior opinion. In Bayesian Statistics, 2 (Valencia, 1983), (J.M. Bernardo, M.H. Degroot, D.V. Lindley, A.F.M. Smith, eds.). North Holland, Amstredam, 133–156.
Diaconis, P. and Zabell, S. (1991). Closed form summation for classical distributions: Variations on a theme of deMoivre. Statist. Sci., 61, 284–302.
Dudley, R.M. (1989). Real Analysis and Probability. Wadsworth, Belmont, CA.
Esch, D. (2003). The skew-t distribution: Properties and computations. Ph.D. Dissertation, Department of Statistics, Harvard University.
Fill, J. and Machida, M. (2001). Stochastic monotonicity and realizable monotonicity. Ann. Appl. Probab., 29, 938–978.
Givens, C. and Shortt, R. (1984). A class of Wasserstein metrics for probability distributions. Michigan Math. J., 31, 231–240.
Jones, G.L. and Hobert, J.P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Statist. Sci.16, 312–334.
Jones, G.L. and Hobert, J.P. (2004). Sufficient burn-in for Gibbs samplers for a hierarchical random effects model. Ann. Statist., 32, 784–817.
Karlin, S. (1968). Total Positivity. Stanford University Press, Stanford.
Khare, K. and Zhou, H. (2009). Rates of convergence of some multivariate Markov chains with polynomial eigenfunctions. Ann. Appl. Probab., 19, 737–777.
Lehmann, E. and Romano, J. (2005). Testing Statistical Hypotheses. Springer, New York.
Liu, J. (2001). Monte Carlo Strategies in Scientific Computing. Springer-Verlag, New York.
Lund, R.B. and Tweedie, R.L. (1996). Geometric convergence rates for stochastically ordered Markov chains. Math. Oper. Res., 20, 182–194.
Meyn, S.P. and Tweedie, R.L. (1993). Markov Chains and Stochastic Stability. Springer-Verlag, London.
Morris, C. (1982). Natural exponential families with quadratic variance functions. Ann. Statist., 10, 65–80.
Morris, C. (1983). Natural exponential families with quadratic variance functions: Statistical theory. Ann. Statist.11, 515–589.
Rosenthal. J.S. (1995). Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Amer. Statist. Assoc., 90, 558–566.
Rosenthal, J.S. (1996). Analysis of the Gibbs sampler for a model related to James-Stein estimations. Statist. Comput., 6, 269–275.
Rosenthal, J.S. (2002). Quantitative convergence rates of Markov chains: A simple account. Electron. Comm. Probab., 7, 123–128.
Saloff-Coste, L. (2004). Total variation lower bounds for finite Markov chains: Wilson’s lemma. In Random Walks and Geometry, (V.A. Kaimanovich, K. Schmidt and W. Woess, eds.). Walter de Gruyter GmbH and Co. KG, Berlin, 515–532.
Stanley, R. (1989). Log-concave and unimodal sequences in algebra, combinatorics, and geometry. In Graph Theory and Its Applications: East and West (Jinan, 1986). Ann. New York Acad. Sci., 576, New York Acad. Sci., New York, 500–535.
Stoyan, D. (1983). Comparison Methods for Queues and Other Stochastic Models. John Wiley and Sons, New York.
Tierney, L. (1994). Markov chains for exploring posterior distributions (with discussion). Ann. Statist., 22, 1701–1762.
Wilson, D.B. (2004). Mixing times of lozenge tiling and card shuffling Markov chains. Ann. Appl. Probab., 14, 274–325.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Diaconis, P., Khare, K. & Saloff-Coste, L. Gibbs sampling, conjugate priors and coupling. Sankhya 72, 136–169 (2010). https://doi.org/10.1007/s13171-010-0004-7
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13171-010-0004-7