Skip to main content
SpringerLink
Log in

The Normal-Theory and Asymptotic Distribution-Free (ADF) Covariance Matrix of Standardized Regression Coefficients: Theoretical Extensions and Finite Sample Behavior

  • Published:
Psychometrika Aims and scope Submit manuscript

An Erratum to this article was published on 14 January 2016

Abstract

Yuan and Chan (Psychometrika, 76, 670–690, 2011) recently showed how to compute the covariance matrix of standardized regression coefficients from covariances. In this paper, we describe a method for computing this covariance matrix from correlations. Next, we describe an asymptotic distribution-free (ADF; Browne in British Journal of Mathematical and Statistical Psychology, 37, 62–83, 1984) method for computing the covariance matrix of standardized regression coefficients. We show that the ADF method works well with nonnormal data in moderate-to-large samples using both simulated and real-data examples. R code (R Development Core Team, 2012) is available from the authors or through the Psychometrika online repository for supplementary materials.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Specifically, the “textbook” standard error for \(\hat{\beta}_{j}\) is incorrect unless the population regression coefficient for variable j equals zero.

  2. For a review of these works, see Jones and Waller (2013).

  3. Although not pursued in this paper, it is also possible to derive our results using the Jacobian from Section 3 (see Equation (27)) and an ADF estimator of \(\operatorname{acov}(\hat{\boldsymbol{r}}_{vp})\) using population standardized fourth-order central moments; see Browne and Shapiro (1986), Hsu (1949), Isserlis (1916), and Steiger and Hakstian (1982) for details.

  4. Note that RAMONA estimates standard errors using n−1 rather than n. We rescaled our estimates accordingly before making the comparisons.

References

  • Abadir, K.M., & Magnus, J.R. (2005). Matrix algebra. New York: Cambridge University Press.

    Book  Google Scholar 

  • Algina, J., & Moulder, B.C. (2001). Sample sizes for confidence intervals on the increase in the squared multiple correlation coefficient. Educational and Psychological Measurement, 61, 633–649.

    Article  Google Scholar 

  • Astin, H.S. (1967). Career development during the high school years. Journal of Counseling Psychology, 14, 94–98.

    Article  Google Scholar 

  • Austin, J.T., & Hanisch, K.A. (1990). Occupational attainment as a function of abilities and interests: a longitudinal analysis using project talent data. Journal of Applied Psychology, 75(1), 77–86.

    Article  PubMed  Google Scholar 

  • Becker, B., & Wu, M. (2007). The synthesis of regression slopes in meta-analysis. Statistical Science, 22, 414–429.

    Article  Google Scholar 

  • Bentler, P., & Lee, S.Y. (1983). Covariance structures under polynomial constraints: applications to correlation and alpha-type structural models. Journal of Educational and Behavioral Statistics, 8(3), 207.

    Article  Google Scholar 

  • Bollen, K.A., & Stine, R. (1990). Direct and indirect effects: classical and bootstrap estimates of variability. Sociological Methodology, 20, 115–140.

    Article  Google Scholar 

  • Boomsma, A., & Hoogland, J.J. (2001). The robustness of LISREL modeling revisited. In R. Cudeck, S. Du Toit, & D. Sorbom (Eds.), Structural equation modeling: present and future (pp. 139–168). Chicago: Scientific Software International.

    Google Scholar 

  • Bring, J. (1994). How to standardize regression coefficients. American Statistician, 48, 209–213.

    Google Scholar 

  • Browne, M.W. (1974). Generalized least squares estimators in the analysis of covariance structures. South African Statistical Journal, 8, 1–24.

    Google Scholar 

  • Browne, M. (1982). Covariance structures. In D.M. Hawkins (Ed.), Topics in applied multivariate analysis (pp. 72–141). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Browne, M.W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical & Statistical Psychology, 37, 62–83.

    Article  Google Scholar 

  • Browne, M.W., Mels, G., & Cowan, M. (1994). Path analysis: Ramona: Systat for DOS advanced applications. SOFTWARE, Version 6:167–224.

  • Browne, M.W., & Shapiro, A. (1986). The asymptotic covariance matrix of sample correlation coefficients under general conditions. Linear Algebra and Its Applications, 82, 169–176.

    Article  Google Scholar 

  • Card, J.J. (1987). Epidemiology of post-traumatic stress disorder in a national cohort of Vietnam veterans. Journal of Clinical Psychology, 43(1), 6–17.

    Article  PubMed  Google Scholar 

  • Casella, G., & Berger, R.L. (2001). Statistical inference. Belmont: Wadsworth.

    Google Scholar 

  • Cohen, J., Cohen, P., West, S.G., & Aiken, L.S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  • Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105, 317–327.

    Article  Google Scholar 

  • Curran, P.J. (1994). The robustness of confirmatory factor analysis to model misspecification and violations of normality. Unpublished Doctoral Dissertation, Arizona State University.

  • Efron, B. (1982). The jackknife, the bootstrap and other resampling plans. In CBMS-NSF regional conference series in applied mathematics (Vol. 38). Philadelphia: SIAM.

    Google Scholar 

  • Efron, B. (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association, 82(397), 171–185.

    Article  Google Scholar 

  • Efron, B., & Tibshirani, R.J. (1994). Monographs on statistics & applied probability. An introduction to the bootstrap. Boca Raton: Chapman & Hall/CRC.

    Google Scholar 

  • Ferguson, T.S. (1996). A course in large sample theory. New York: Chapman & Hall.

    Book  Google Scholar 

  • Fleishman, A.I. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521–532.

    Article  Google Scholar 

  • Greene, W.H. (2003). Econometric analysis (5th ed.). Upper Saddle River: Prentice-Hall.

    Google Scholar 

  • Greenland, S., Maclure, M., Schlesselman, J.J., Poole, C., & Morgenstern, H. (1991). Standardized regression coefficients: a further critique and review of some alternatives. Epidemiology, 2, 387–392.

    Article  PubMed  Google Scholar 

  • Harris, R.J. (2001). A primer on multivariate statistics (3rd ed.). Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Hays, W.L. (1994). Statistics (5th ed.). Worth Fort: Harcourt Brace College Publisher.

    Google Scholar 

  • Hsu, P.L. (1949). The limiting distribution of functions of sample means and application to testing hypotheses. In J. Neyman (Ed.), Proceedings of the first Berkeley symposium on mathematical statistics and probability (pp. 359–402). Berkeley: Univ. of California Press.

    Google Scholar 

  • Isserlis, L. (1916). On certain probable errors and correlation coefficients of multiple frequency distributions with skew regression. Biometrika, 11, 185–190.

    Article  Google Scholar 

  • Jamshidian, M., & Bentler, P.M. (2000). Improved standard errors of standardized parameters in covariance structure models: implications for construction explication. In R.D. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment (pp. 73–94). Dordrecht: Kluwer Academic.

    Chapter  Google Scholar 

  • Jones, J.A., & Waller, N.G. (in press). Computing confidence intervals for standardized regression coefficients. Psychological Methods. doi:10.1037/a0033269.

  • Jones, J.A., & Waller, N.G. (2013). The normal-theory and asymptotic distribution-free (ADF) covariance matrix of standardized regression coefficients: theoretical extensions and finite sample behavior. Minneapolis: University of Minnesota. Retrieved from http://www.psych.umn.edu/faculty/waller/downloads/techreports/TR052913.pdf.

    Google Scholar 

  • Kano, Y., Berkane, M., & Bentler, P.M. (1993). Statistical inference based on pseudo-maximum likelihood estimators in elliptical populations. Journal of the American Statistical Association, 88(421), 135–143.

    Google Scholar 

  • Kelley, K. (2007). Confidence intervals for standardized effect sizes: theory, application, and implementation. Journal of Statistical Software, 20(8), 1–24.

    Article  Google Scholar 

  • Kelley, K., & Maxwell, S.E. (2003). Sample size for multiple regression: obtaining regression coefficients that are accurate, not simply significant. Psychological Methods, 8, 305–321.

    Article  PubMed  Google Scholar 

  • Kim, R.S. (2011). Standardized regression coefficients as indices of effect sizes in meta analysis. Unpublished Doctoral Dissertation, Florida State University, Tallahassee, Florida.

  • Kwan, J.L., & Chan, W. (2011). Comparing standardized coefficients in structural equation modeling: a model reparameterization approach. Behavior Research Methods, 43(3), 730–745.

    Article  PubMed  Google Scholar 

  • King, G. (1985). How not to lie with statistics: avoiding common mistakes in quantitative political science. American Journal of Political Science, 30, 666–687.

    Article  Google Scholar 

  • Mardia, K.V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57, 519–530.

    Article  Google Scholar 

  • Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166.

    Article  Google Scholar 

  • Muthén, B., & Muthén, L. (2012). Mplus users guide (7th ed.). Los Angeles: Muthen & Muthen.

    Google Scholar 

  • Nel, D.G. (1985). A matrix derivation of the asymptotic covariance matrix of sample correlation coefficients. Linear Algebra and Its Applications, 67, 137–145.

    Article  Google Scholar 

  • Oehlert, G.W. (1992). A note on the delta method. American Statistician, 46, 27–29.

    Google Scholar 

  • Ogasawara, H. (2007). Asymptotic expansion and asymptotic robustness of the normal-theory estimators in the random regression model. Journal of Statistical Computation and Simulation, 77(10), 821–838.

    Article  Google Scholar 

  • Ogasawara, H. (2008). Asymptotic expansion in reduced rank regression under normality and nonnormality. Communications in Statistics. Theory and Methods, 37, 1051–1070.

    Article  Google Scholar 

  • Olkin, I., & Finn, J.D. (1995). Correlations redux. Psychological Bulletin, 118, 155–164.

    Article  Google Scholar 

  • Olkin, I., & Siotani, M. (1976). Asymptotic distribution of functions of a correlation matrix. In S. Ideka (Ed.), Essays in probability and statistics (pp. 235–251). Tokyo: Shinko, Tsusho Co., Ltd.

    Google Scholar 

  • Pearson, K., & Filon, L. (1898). On the probable errors of frequency constants and on the influence of random selection on variation and correlation. Philosophical Transactions of the Royal Society of London, 191, 229–311.

    Article  Google Scholar 

  • Peterson, R., & Brown, S. (2005). On the use of beta coefficients in meta-analysis. Journal of Applied Psychology, 90, 175–181.

    Article  PubMed  Google Scholar 

  • R Development Core Team (2012). R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. ISBN 3-900051-07-0. http://www.R-project.org/.

    Google Scholar 

  • Rencher, A.C. (2008). Linear models in statistics (2nd ed.). Hoboken: Wiley.

    Google Scholar 

  • Steiger, J.H. (1995). Structural equation modeling (SEPATH). Tulsa: Statsoft Inc.

    Google Scholar 

  • Steiger, J.H., & Fouladi, R.T. (1997). Noncentrality interval estimation and the evaluation of statistical models. In L.L. Harlow, S.A. Mulaik, & J.H. Steiger (Eds.), What if there were no significance tests? (pp. 221–257). London: Taylor and Francis.

    Google Scholar 

  • Steiger, J.H., & Hakstian, A. (1980). The asymptotic distribution of elements of a correlation matrix (Technical Report No. 80 3 (May)). Institute of Applied Mathematics and Statistics, University of British Columbia.

  • Steiger, J., & Hakstian, A. (1982). The asymptotic distribution of elements of a correlation matrix: theory and application. British Journal of Mathematical & Statistical Psychology, 35, 208–215.

    Article  Google Scholar 

  • Tukey, J.W. (1954). Causation, regression, and path analysis. In O. Kempthorne, T.A. Bancroft, J.W. Gowen, & J.L. Lush (Eds.), Statistics and mathematics in biology (pp. 35–66). Ames: Iowa State College Press.

    Google Scholar 

  • Vale, C.D., & Maurelli, V.A. (1983). Simulating multivariate nonnormal distributions. Psychometrika, 48, 465–471.

    Article  Google Scholar 

  • Waller, N.G. (2011). The geometry of enhancement in multiple regression. Psychometrika, 76, 634–649.

    Article  Google Scholar 

  • Waller, N.G., & Jones, J.A. (2010). Correlation weights in multiple regression. Psychometrika, 75(1), 58–69.

    Article  Google Scholar 

  • Waller, N.G., & Jones, J.A. (2011). Investigating the performance of alternate regression weights by studying all possible criteria in regression models with a fixed set of predictors. Psychometrika, 76, 410–439.

    Article  Google Scholar 

  • Waller, N.G., Underhill, M., & Kaiser, H.A. (1999). A method for generating simulated plasmodes and artificial test clusters with user-defined shape, size, and orientation. Multivariate Behavioral Research, 34, 123–142.

    Article  Google Scholar 

  • West, S.G., Aiken, L.S., Wu, W., & Taylor, A.B. (2007). Multiple regression: applications of the basics and beyond in personality research. In R. Robins, R.C. Fraley, & R.F. Krueger (Eds.), Handbook of research methods in personality psychology (pp. 573–601). New York: Guilford.

    Google Scholar 

  • Wise, L.L., McLaughlin, D.H., & Steel, L. (1979). The project TALENT data bank handbook. Palo Alto: American Institutes for Research.

    Google Scholar 

  • Yuan, K.H., & Chan, W. (2011). Biases and standard errors of standardized regression coefficients. Psychometrika, 76, 670–690.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, University of Minnesota, 75 East River Road, Minneapolis, MN, 55455, USA

    Jeff A. Jones & Niels G. Waller

Authors
  1. Jeff A. Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Niels G. Waller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Niels G. Waller.

Electronic Supplementary Material

Below are the links to the electronic supplementary material.

(PDF 44 kB)

(PDF 133 kB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jones, J.A., Waller, N.G. The Normal-Theory and Asymptotic Distribution-Free (ADF) Covariance Matrix of Standardized Regression Coefficients: Theoretical Extensions and Finite Sample Behavior. Psychometrika 80, 365–378 (2015). https://doi.org/10.1007/s11336-013-9380-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-013-9380-y

Key words

Search

Navigation