Cognitive diagnostic model comparisons

Abstract

Cognitive diagnostic assessment (CDA) is a new theoretical framework that is designed to integrate cognitive psychology into measurement theories. The main purpose of CDA is to provide examinees with diagnostic information while traditional psychometric approaches focus on how latent variables are accurately measured. Many cognitive diagnostic models (CDM) have been developed for CDA. Three cognitive diagnostic models- namely the rule space method (RSM), the high-order deterministic inputs, noisy ‘and’ gate (HO-DINA) model, and the multidimensional latent trait model for diagnosis (MLTM-D) model were compared using simulated data and empirical data. For the simulation study, three methods of data generation are proposed. Each method was designed based on one of the three models. A total of 12 conditions was involved in the simulation study: 2 item designs X 2 level of test X 3 methods of data generation. The diagnostic results were compared by level of test difficulty, level of ability estimates, and level of dimensionality. The effect of number of attributes on accurate classification was also investigated. For the empirical study, a mathematics test data was used and the diagnostic results were compared.