Is it worthwhile to take account of "guessing" in the performance of the Raven test? Calling for the principle of parsimony for test validation

Abstract

The present study compares the fit of two- and three-parameter logistic (2PL, 3LP) models of Item Response Theory in the performance of preschool children on the Raven's Coloured Progressive Matrices. The test of Raven is widely used for evaluating non-verbal intelligence of factor g. Studies comparing models with real data are scarce on the literature and this is the first to compare models of two and three parameters for the test of Raven, evaluating the informational gain of considering guessing probability. Participants were 582 Brazilian preschool children (age mean = 57 months; s.d. = 7 months; 46% female) who responded individually the instrument. The model fit indices suggested that the 2PL fit better to the data. The difficulty and theta parameters were similar between the models, with almost perfect correlations. Differences were observed in terms of discrimination and test e information. The principle of parsimony must be called for comparing models.

Citation

Lucio, P., Vandekerckhove, J., Polanczyk, G., & Cogo-Moreira, H. (2021). Is it worthwhile to take account of "guessing" in the performance of the Raven test? Calling for the principle of parsimony for test validation. Journal of Psychoeducational Assessment, 39, 100–111.

Bibtex

@article{lucio_etal:2021:performance,
    title   = {{I}s it worthwhile to take account of "guessing" in the performance of the {R}aven test? {C}alling for the principle of parsimony for test validation},
    author  = {Lucio, Patricia and Vandekerckhove, Joachim and Polanczyk, Guilherme and Cogo-Moreira, Hugo},
    year    = {2021},
    journal = {Journal of Psychoeducational Assessment},
    volume  = {39},
    pages   = {100--111}
}