Skip to main content
SpringerLink
Log in

A review and some new results on permutation testing for multivariate problems

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

In recent years permutation testing methods have increased both in number of applications and in solving complex multivariate problems. When available permutation tests are essentially of an exact nonparametric nature in a conditional context, where conditioning is on the pooled observed data set which is often a set of sufficient statistics in the null hypothesis. Whereas, the reference null distribution of most parametric tests is only known asymptotically. Thus, for most sample sizes of practical interest, the possible lack of efficiency of permutation solutions may be compensated by the lack of approximation of parametric counterparts. There are many complex multivariate problems, quite common in empirical sciences, which are difficult to solve outside the conditional framework and in particular outside the method of nonparametric combination (NPC) of dependent permutation tests. In this paper we review such a method and its main properties along with some new results in experimental and observational situations (robust testing, multi-sided alternatives and testing for survival functions).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abd-Elfattah, E.F., Butler, R.W.: The weighted log-rank class of permutation tests: P-values and confidence intervals using saddlepoint methods. Biometrika 94, 543–551 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Basso, D., Pesarin, F., Salmaso, L., Solari, A.: Permutation Tests for Stochastic Ordering and ANOVA: Theory and Applications in R. Lecture Notes in Statistics, vol. 194. Springer, New York (2009)

    MATH  Google Scholar 

  • Basso, D., Pesarin, F.: Exact permutation testing for two-sample survival functions. Technical Report N. 13, Department of Management and Engineering, University of Padua (2011, submitted)

  • Bertoluzzo, F., Pesarin, F., Salmaso, L.: On multi-sided permutation tests. J. Stat. Plan. Inference (2011, forthcoming)

  • Birnbaum, A.: Combining independent tests of significance. J. Am. Stat. Assoc. 49, 559–574 (1954)

    Article  MathSciNet  MATH  Google Scholar 

  • Birnbaum, A.: Characterizations of complete classes of tests of some multiparametric hypotheses, with applications to likelihood ratio tests. Ann. Math. Stat. 26, 21–36 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  • Cox, D.R., Hinkley, D.V.: Theoretical Statistics. Chapman and Hall, London (1974)

    MATH  Google Scholar 

  • David, H.A.: The beginnings of randomization tests. Am. Stat. 62, 70–72 (2008)

    Article  Google Scholar 

  • Edgington, E.S., Onghena, P.: Randomization Tests, 4th edn. Chapman and Hall/CRC, London (2007)

    MATH  Google Scholar 

  • Goggin, M.L.: The “Too few cases/too many variables” problem in implementation research. West. Polit. Q. 39, 328–347 (1986)

    Article  Google Scholar 

  • Good, P.: Permutation, Parametric, and Bootstrap Tests of Hypotheses, 3rd edn. Springer, New York (2005)

    MATH  Google Scholar 

  • Hoeffding, W.: The large-sample power of tests based on permutations of observations. Ann. Math. Stat. 23, 169–192 (1952)

    Article  MathSciNet  Google Scholar 

  • Lehmann, E.L.: Consistency and unbiasedness of certain nonparametric tests. Ann. Math. Stat. 22, 165–179 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  • Lehmann, E.L.: Parametric versus nonparametrics: two alternative methodologies. J. Nonparametr. Stat. 21, 397–405 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, 3rd edn. Springer, New York (2005)

    MATH  Google Scholar 

  • Lehmann, E.L., Scheffé, H.: Completeness similar regions, and unbiased estimation. Sankhyā 10, 305–340 (1950)

    MATH  Google Scholar 

  • Lehmann, E.L., Scheffé, H.: Completeness similar regions, and unbiased estimation—part II. Sankhyā 15, 219–236 (1955)

    MATH  Google Scholar 

  • Mehta, C.R., Patel, N.R.: A network algorithm for the exact treatment of the 2×K contingency table. Commun. Stat., Simul. Comput. 9, 649–664 (1980)

    Article  MathSciNet  Google Scholar 

  • Mehta, C.R., Patel, N.R.: A network algorithm for performing Fisher’s exact test in r×c contingency tables. J. Am. Stat. Assoc. 78, 427–434 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  • Mielke, P.W., Berry, K.J.: Permutation Methods. A Distance Function Approach, 2nd edn. Springer, New York (2007)

    MATH  Google Scholar 

  • Moder, K., Rasch, D., Kubinger, K.D.: Don’t use the two-sample t-test anymore! In: Ermakov, S.M., Melas, V.B., Pepelyshev, A.N. (eds.) Proceedings of the 6th St. Petersburg Workshop on Simulation, pp. 258–264 (2009)

    Google Scholar 

  • Pesarin, F.: Multivariate Permutation Tests: With Application in Biostatistics. Wiley, Chichester (2001)

    MATH  Google Scholar 

  • Pesarin, F.: Extending permutation conditional inference to unconditional one. Stat. Methods Appl. 11, 161–173 (2002)

    Article  MATH  Google Scholar 

  • Pesarin, F., Salmaso, L.: Finite-sample consistency of combination-based permutation tests with application to repeated measures designs. J. Nonparametr. Stat. (2009). doi:10.1080/10485250902807407

    MATH  Google Scholar 

  • Pesarin, F., Salmaso, L.: Permutation Tests for Complex Data. Theory, Applications and Software. Wiley Series in Probability and Statistics. Wiley, Chichester (2010)

    Google Scholar 

  • Pesarin, F., Salmaso, L.: On the weak consistency of permutation tests. J. Stat. Plan. Inference (2011, forthcoming)

  • Romano, J.P.: On the behaviour of randomization tests without group variance assumption. J. Am. Stat. Assoc. 85, 686–692 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  • Scheffé, H.: Statistical inference in the non-parametric case. Ann. Math. Stat. 14, 305–332 (1943)

    Article  MATH  Google Scholar 

  • Strasser, H., Weber, Ch.: The asymptotic theory of permutation statistics. Math. Methods Stat. 8, 220–250 (1999)

    MathSciNet  MATH  Google Scholar 

  • Westfall, P.H., Young, S.S.: Resampling-Based Multiple Testing: Examples and Methods for p-Values Adjustment. Wiley, New York (1993)

    Google Scholar 

  • Zhang, Y., Rosenberger, W.F.: On asymptotic normality of the randomization-based logrank test. J. Nonparametr. Stat. 17, 833–839 (2005)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistical Sciences, University of Padua, Padova, Italy

    Fortunato Pesarin

  2. Department of Management and Engineering, University of Padova, Vicenza, Italy

    Luigi Salmaso

Authors
  1. Fortunato Pesarin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luigi Salmaso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luigi Salmaso.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pesarin, F., Salmaso, L. A review and some new results on permutation testing for multivariate problems. Stat Comput 22, 639–646 (2012). https://doi.org/10.1007/s11222-011-9261-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-011-9261-0

Keywords

Search

Navigation