Avaliação da proficiência em inglês acadêmico através de um teste adaptativo informatizado
Abstract
This work describes the steps for converting a linear paper-and-pencil English proficiency test
for academic purposes, composed with multiple choice items that are administered following the
admissible probability measurement procedure (Shuford Jr et al., 1966), adopted by the graduate
program of Institute of Mathematical Sciences and Computing of University of São Paulo (ICMCUSP),
Brazil, to a computerized adaptive test (CAT) based on an item response theory model
(IRT). Despite the Institute recognizes reliable international English-language exams for academic
purposes and non-native speakers, as TOEFL (Test of English as a Foreign Language), IELTS
(International English Language Testing System) and CPE (Cambridge English: Proficiency), for
instance, it is inconsistent that public universities in Brazil require them as certification because of
the cost of approximately US$ 200.00 to US$ 300.00 per exam. The software TAI-PI (computerized
adaptive test for English proficiency) was implemented in Java language, used SQLite as database
engine, and it shall be offered free of charge for English proficiency assessment of the graduate
students from October 2013. The statistical methodology employed for TAI-PI construction was
defined considering the history and the aims of the evaluation and adopted the Samejima's graded
response model (Samejima, 1969), the Kullback-Leibler information criterion for item selection, the
expected a posteriori Bayesian estimation for latent trait (Baker, 2001) and shadow test approach
(Van der Linden & Pashley, 2010) for test constraints (content and size of the test, for example). A
description of the test design, the employed statistical methods, study results of a real application
of TAI-PI to graduate students are presented in this work and the validation studies of the new
methodology for pass/fail classi cation, highlighting the good quality of the new evaluation system
and examination of improvement with the use of the methods of IRT and CAT.