A Note on the Sampling Distribution of the Likelihood Ratio Test in the Context of the Linear Logistic Test Model

Rainer W. Alexandrowicz

doi:10.17713/ajs.v38i4.275

Authors

Rainer W. Alexandrowicz Ludwig-Maximilians-Universität München

DOI:

https://doi.org/10.17713/ajs.v38i4.275

Abstract

One important tool for assessing whether a data set can be described equally well with a Rasch Model (RM) or a Linear Logistic Test Model (LLTM) is the Likelihood Ratio Test (LRT). In practical applications this test seems to overly reject the null hypothesis, even when the null hypothesis is true. Aside from obvious reasons like inadequate restrictiveness of linear restrictions formulated in the LLTM or the RM not being true, doubts have arisen whether the test holds the nominal type-I error risk, that is whether its theoretically derived sampling distribution applies. Therefore, the present contribution explores the sampling distribution of the likelihood ratio test comparing a Rasch model with a Linear Logistic Test Model. Particular attention is put on the issue of similar columns in the weight matrixW of the LLTM: Although full column rank of this matrix is a technical requirement, columns can differ in only a few entries, what in turn might have an impact on the sampling distribution of the test statistic. Therefore, a system of how to generate weight matrices with similar columns has been established and tested in a simulation study. The results were twofold: In general, the matrices
considered in the study showed LRT results where the empirical alpha showed only spurious deviations from the nominal alpha. Hence the theoretically chosen alpha seems maintained up to random variation. Yet, one specific matrix clearly indicated a highly increased type-I error risk: The empirical alpha was at least twice the nominal alpha when using this weight matrix. This shows that we have to indeed consider the internal structure of the weight matrix when applying the LRT for testing the LLTM. Best practice would be to perform a simulation or bootstrap/re-sampling study for the weight matrix under consideration in order to rule out a misleadingly significant result due to reasons other than true model misfit.

References

Andersen, E. B. (1970). Asymptotic properties of conditional maximum likelihood estimators. Journal of the Royal Statistical Society, Series B, 32, 283-301.

Andersen, E. B. (1972). The numerical solution of a set of conditional estimation equations. Journal of the Royal Statistikal Society, Series B, 34, 42-54.

Andersen, E. B. (1973). A goodness of fit test for the rasch model. Psychometrika, 38, 123-140.

Baker, F. B. (1993). Sensitivity of the linear logistic test model to misspecification of the weight matrix. Applied Psychological Measurement, 17, 201-210.

Davison, A. C., and Hinkley, D. V. (1997). Bootstrap Methods and Their Application. Cambridge: Cambridge University Press.

Draxler, C. (2007). Sequentielle Tests für das Rasch Modell. Berlin: Logos.

Efron, B., and Tibshirani, R. J. (1993). An introduction to the bootstrap. Boca Raton: Chapman & Hall.

Fischer, G. H. (1972). A measurement model for the effect of mass-media. Acta Psychologica, 36, 207-220.

Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359-374.

Fischer, G. H. (1983). Logistic latent trait models with linear constraints. Psychometrika, 48, 3-26.

Fischer, G. H. (1995). The linear logistic test model. In G. H. Fischer and I. Molenaar (Eds.), Rasch Models: Foundations, Recent Developments, and Applications (p. 131-156). New York: Springer.

Fischer, G. H. (1997). Unidimensional linear logistic Rasch models. In W. J. Van der Linden and R. K. Hambleton (Eds.), Handbook of Modern Item Response Theory (p. 225-243). New York: Springer.

Hohensinn, C., Kubinger, K. D., Reif, M., Holocher-Ertl, S., Khorramdel, L., and Frebort, M. (2008). Examining item-position effects in large-scale assessment using the linear logistic test model. Psychology Science Quarterly, 50, 391-402.

Klein, S. (2003). Neue Methoden zur Entdeckung von Fehlspezifikation bei Latent-Trait-Modellen der Veränderungsmessung. Retrieved 23.01.2010, from http://edoc.hu-berlin.de/dissertationen/klein-stefan-2003-05

-09/PDF/Klein.pdf

Kubinger, K. D., and Draxler, C. (2007). Probleme bei der Testkonstruktion nach dem Rasch-Modell. Diagnostica, 53, 131-143.

Lancaster, T. (2000). The incidental parameter problem since 1948. Journal of Econometrics, 95, 391-413.

Mair, P., and Hatzinger, R. (2007a). CML based estimation of extended Rasch models with the eRm package in R. Psychology Science, 49, 26-43.

Mair, P., and Hatzinger, R. (2007b). Extended Rasch modeling: The eRm package for the application of IRT models in R. Journal of Statistical Software, 20(9), 1-20.

Neyman, J., and Scott, E. L. (1948). Consistent estimation from partially consistent observations. Econometrica, 16, 1-32.

R Development Core Team. (2008). R: A Language and Environment for Statistical Computing [Computer software manual]. Vienna, Austria. Available from http://www.R-project.org

Rasch, D., Kubinger, K. D., Schmidtke, J., and Häusler, J. (2004). The misuse of asterisks in hypothesis testing. Psychology Science, 46, 227-242.

Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen: Danmarks Pædagogiske Institut.

Scheiblechner, H. (1971). CML-parameter-estimation in a generalized multifactorial version of Rasch’s probabilistic measurement model with two categories of answers

(Research Bulletin No. 4). Vienna: Department of Psychology, University of Vienna.

Scheiblechner, H. (1972). Das Lernen und L¨osen komplexer Denkaufgaben. Zeitschrift für Experimentelle und Angewandte Psychologie, 3, 456-506.

Shao, J., and Tu, D. (1995). The Jackknife and Bootstrap. New York: Springer.

van den Wollenberg, A. L. (1982). A simple and effective method to test the dimensionality axiom of the Rasch model. Applied Psychological Measurement, 6, 83-91.

Zimmerman, D. W. (2004). A note on preliminary tests of equality of variances. British Journal of Mathematical and Statistical Psychology, 57, 173-181.