Empirical Comparison of Three Methods for Detecting Differential Item Functioning in Dichotomous Test Items

Abstract

The study classified individual items that function differentially according to the magnitude of DIF in dichotomous test and compared the power of Generalized Mantel Haenszel (GMH), Simultaneous Item Bias Test (SIBTEST), and Logistic Discriminant Function Analysis (LDFA) methods in detecting DIF in dichotomous test items. It also determined the relationship between the proportions of test items that function differentially in dichotomous test items when the different methods were used. These were with a view to improving the quality of dichotomous test items construction. The study adopted a survey design. The population consisted of all undergraduate students who registered for EFC 303 (Tests and Measurement) at Obafemi Awolowo University during 2011/2012 Harmattan Semester. The sample consisted of an intact class of 457 Part 3 undergraduate students who registered for the course. A 50-item multiple -choice achievement test designated “Undergraduate Students’ Achievement Test (USAT)” was developed by the researcher for the study. A total of 445 scripts were found properly completed. Data collected were analysed using Generalized Mantel Haenszel, Simultaneous Item Bias Test, and Logistic Discriminant Function Analysis. The results showed that there was a significant difference in the performance of the GMH, SIBTEST, and LDFA methods in detecting DIF in dichotomous test items. Further, a non significant relationship existed between the proportions of test items that functioned differentially in the dichotomous test items when the different methods were used. The study concluded that GMH, SIBTEST and LDFA were effective in detecting DIF across dichotomous test items but complement one other in their ability to detect DIF in dichotomous test items.