Abstract
The conventional method of measuring ability, which is based on items with assumed true parameter values obtained from a pretest, is compared to a Bayesian method that deals with the uncertainties of such items. Computational expressions are presented for approximating the posterior mean and variance of ability under the three-parameter logistic (3PL) model. A 1987 American College Testing Program (ACT) math test is used to demonstrate that the standard practice of using maximum likelihood or empirical Bayes techniques may seriously underestimate the uncertainty in estimated ability when the pretest sample is only moderately large.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.),Statistical theories of mental test scores (pp. 395–479). Reading, MA: Addison-Wesley.
Birnbaum, A. (1969). Statistical theory for logistic mental test models with prior distribution of ability.Journal of Mathematical Psychology, 6, 258–276.
Bock, R. D., & Aitken, M. (1981). Marginal maximum likelihood estimation of item parameters: An application of an EM algorithm.Psychometrika, 46, 443–459.
Deeley, J. J., & Lindley, D. V. (1981). Bayes empirical Bayes.Journal of the American Statistical Association, 76, 833–841.
DeGroot, M. H. (1986).Probability and statistics (2nd ed.). Reading, MA: Addison-Wesley.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion).Journal of the Royal Statistical Society, Series B,39, 1–38.
Lindley, D. V. (1980). Approximate Bayesian methods.Trabajos Estadistica, 31, 223–237.
Lord, F. M. (1980).Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
Lord, F. M. (1986). Maximum likelihood and Bayesian parameter estimation in item response theory.Journal of Educational Measurement, 23, 157–162.
Mislevy, R. J., & Bock, R. D. (1984).BILOG: Item analysis and test scoring with binary logistic models. Mooresville, IN: Scientific Software.
Tsutakawa, R. K. (1984). Estimation of two-parameter logistic item response curves.Journal of Educational Statistics, 9, 263–276.
Tsutakawa, R. K. (1988).Dirichlet prior in Bayesian estimation of item response curves (Mathematical Sciences Technical Report No. 143). Columbia, MO: University of Missouri, Department of Statistics.
Tsutakawa, R. K., & Soltys, M. J. (1988). Approximations for Bayesian ability estimation.Journal of Educational Statistics, 13, 117–130.
Wingersky, M. S., Barton, M. A., & Lord, F. M. (1982).LOGIST user's guide. Princeton, NJ: Educational Testing Service.
Author information
Authors and Affiliations
Additional information
This work was partially supported under contract No. N00014-85-K-0113, NR150-535, from the Cognitive Science Program, Office of Naval Research. The authors wish to thank Mark D. Reckase for providing the ACT data used in the illustration and two referees, Asociate Editor and Editor for helpful suggestions.
Rights and permissions
About this article
Cite this article
Tsutakawa, R.K., Johnson, J.C. The effect of uncertainty of item parameter estimation on ability estimates. Psychometrika 55, 371–390 (1990). https://doi.org/10.1007/BF02295293
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02295293