189 / 2018-08-14 18:26:10

External validation of the alignment between a large-scale in-house listening test and CSE

China's Standards of English,aligning,cut scores,external validation

全体主题 > 语言测试与语言能力标准的对接研究

As a national framework of reference for English language education, China’s Standards of English (CSE) has received much attention in the field of language testing and assessment (Jin et al., 2017; He & Chen, 2017; Liu & Han, 2018; Min et al., 2018). Few studies, however, have aimed at interpreting the test scores in relation to the CSE levels. We aligned the listening subtest of an in-house English exit test, with over 7,000 test takers every year, to CSE following the framework prescribed by the Manual (Europe of Council, 2009). There were four components of the alignment, namely specification, familiarization, standard setting, and validation, in which 13 panelists took part and two standard setting methods were adopted. MFRM analysis was conducted by FACETS (Linacre, 2014) to prove the procedural validity of the cut scores which were yielded corresponding to the CSE levels. Since improper interpretation of the CSE descriptors can lead to wrong placement of the test takers, it is essential that follow-up research should be conducted to verify the standard-setting results. Accordingly, this empirical study is to externally examine the cut scores established so as to gather evidence supporting the external validity of the standard-setting process in the linkage project. Five senior college English teachers and 30 sophomores participated in the study. The teachers were asked to judge their students’ English proficiency levels according to the CSE descriptors. Meanwhile the test was administered to those students. Thus, their respective CSE levels were gained from the results of the previous alignment study as well as their teachers’ judgement. The results compared were not significantly different by using Wilcoxon signed rank test. Therefore, the findings validated the standard setting results. The implications for the test interpretation were further discussed.