This report is the second in a multi-part technical report series describing the development, calibration and validation of standards-based tests for English as a first foreign language at the Institute for Educational Progress (Institut zur Qualitätsentwicklung im Bildungswesen, IQB) in Berlin, Germany. It details the empirical basis of the calibration of the test item pool and criterion-referenced standard-setting procedures. The aim is to make transparent the decisions, methods and procedures which led to the setting of cut-scores in alignment with the National Educational Standards (NES) (i.e., the Länderübergreifende Bildungsstandards) and the Common European Framework of Reference for Languages (CEF) for English as a first foreign language. Standards were set for the lower secondary school level of Hauptschulabschluss and Mittlerer Bildungsabschluss. This second report describes the process of relating the standards-based proficiency tests to the CEF levels, delineating the purpose of the tests, the aims of the standard-setting procedures, the rationale of the chosen procedures, and the use of the Manual for Linking Language Examinations to the CEF. This is followed by a detailed description of the two standard-setting methods employed, the Bookmark method and the computer-assisted Criterion Mapping method—developed and implemented by the Berkeley Evaluation and Assessment Research (BEAR) Center, University of California. This volume provides information on how the standard-setting sessions were conducted in collaboration with BEAR Center, and describes procedures, data gathering, and issues and problems which arose during the course of the study. The report concludes with a synthesis of the standard-setting study’s results, and discusses the implications of how these results are reported and presented to stakeholders and policy makers.

This book summarizes the international evidence on methodological issues in standard setting in education. By critically discussing the standard-setting practices implemented in the Nordic countries and by presenting new methodological approaches, it offers fresh perspectives on the current research. Standard setting targets crucial societal objectives by defining educational benchmarks at different achievement levels, and provides feedback to policy makers, schools and teachers about the strengths and weaknesses of a school system. Given that the consequences of standard setting can be dramatic, the quality of standard setting is a prime concern. If it fails, repercussions can be expected in terms of arbitrary evaluations of educational policy, wrong turns in school or teacher development or misplacement of individual students. Standard setting therefore needs to be accurate, reliable, valid, useful, and defensible. However, specific evidence on the benefits and limits of different approaches to standard setting is rare and scattered, and there is a particular lack with respect to standard setting in the Nordic countries, where the number of national tests is increasing and there are concerns about the time and effort spent on testing at schools without feedback being provided. Addressing this gap, the book offers a discussion on standard setting by respected experts as well as profound and innovative insights into fundamental aspects of standard setting including conclusions for future methodological and policy-related research.

A critical examination of the ways in which English is conceptualised for learning, teaching, and assessment in a range of domains, from both social and cognitive perspectives. Researchers and postgraduates working on English in L1 and L2 educational contexts will find it valuable for research and collaboration.

Measuring competencies acquired over the course of higher education is an area of research that largely has been neglected. This deficit can be attributed to the complexity of academic competencies since the variety of institutions, programs, occupational fields and job requirements render it difficult to define and even harder to measure outcomes. Thus, assessing the preconditions for and effects of competency development is a challenge methodologically. In this book, a comprehensive review of the international state of research on modeling and measuring competencies in higher education across several academic disciplines is presented, along with an evaluation of the field’s strengths and weaknesses. Renowned experts share insight into the most important recent research projects and point out controversies and challenges. Competencies are regarded as context-specific dispositions acquired and required to cope successfully with domain-specific situations and tasks. However, the range of these dispositions is controversial. Should only cognitive facets be included or is it important to consider attitudes as well? Insufficient response rates and panel attrition are further challenges, as are the limitations of paper-and-pencil approaches to the complex measurement of higher education outcomes. Thus, this book may serve as a platform for the exchange of research experiences and perspectives and hopefully will instigate improvements in research into higher education.

The Diagnosis of Writing in a Second or Foreign Language is a comprehensive survey of diagnostic assessment of second/foreign language (SFL) writing. In this innovative book, a compelling case is made for SFL writing as an individual, contextual, and multidimensional ability, combining several theoretically informed approaches upon which to base diagnosis. Using the diagnostic cycle as the overarching framework, the book starts with the planning phase, cover design, development, and delivery of diagnostic assessment, ending with feedback and feed-forward aspects to feed diagnostic information into the teaching and learning process. It covers means to diagnose both the writing processes and products, including the design and development of diagnostic tasks and rating scales, as well as automated approaches to assessment. Also included is a range of existing instruments and approaches to diagnosing SFL writing. Addressing large-scale as well as classroom contexts, this volume is useful for researchers, teachers, and educational policy-makers in language learning.

This book is a practical guide to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR) (Council of Europe 2001) and the CEFR Companion Volume (CEFR/ CV; COE 2018), which have increasingly been used to inform the language policies and teaching practices of countries within and outside of Europe. It helps practitioners to (i) grasp essential and core concepts of the Common European Framework of Reference, (ii) identify parts of the CEFR and the CEFR/CV as well as other CEFR-related resources and documents that are relevant for readers’ different purposes, and (iii) utilise and adapt these resources for their own needs. Written by practitioners for practitioners, this hands-on guide covers the philosophy of the CEFR, curricula, assessment, learner autonomy, the task-based approach, and teacher development. Logically explaining all aspects of the framework and its application, this manual helps readers deal with many of the difficulties encountered when using CEFR and the CEFR CV. The book will appeal to a wide audience, including teacher educators; curriculum and materials developers; examination boards unfamiliar with the CEFR; university language departments and language centres responsible for developing their own curricula, teaching/learning approaches and assessment instruments; and policy-makers wanting to learn more about the implications of adopting the CEFR. It is a guidebook, a reference book and a workbook all in your hand.

This book addresses challenges in the theoretically and empirically adequate assessment of competencies in educational settings. It presents the scientific projects of the priority program “Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes,” which focused on competence assessment across disciplines in Germany. The six-year program coordinated 30 research projects involving experts from the fields of psychology, educational science, and subject-specific didactics. The main reference point for all projects is the concept of “competencies,” which are defined as “context-specific cognitive dispositions that are acquired and needed to successfully cope with certain situations or tasks in specific domains” (Koeppen et al., 2008, p. 62). The projects investigate different aspects of competence assessment: The primary focus lies on the development of cognitive models of competencies, complemented by the construction of psychometric models based on these theoretical models. In turn, the psychometric models constitute the basis for the construction of instruments for effectively measuring competencies. The assessment of competencies plays a key role in optimizing educational processes and improving the effectiveness of educational systems. This book contributes to this challenging endeavor by meeting the need for more integrative, interdisciplinary research on the structure, levels, and development of competencies.

The Routledge Encyclopedia of Language Teaching and Learning is an authoritative reference dealing with all aspects of this increasingly important field of study. Offering a comprehensive range of articles on contemporary language teaching and its history, it has been produced specifically for language teaching professionals and as a reference work for academic studies at postgraduate level. In this new edition, every single entry has been reviewed and updated with reference to new developments and publications. Coverage has been expanded to reflect new technological, global and academic developments, with particular attention to areas such as online and distance learning, teacher and learner cognition, testing, assessment and evaluation, global English and teacher education. Themes and disciplines covered include: Methods and materials, including new technologies and materials development Contexts and concepts, such as mediation, risk-taking in language learning and intercomprehension Influential figures from the early days of language teaching to the contemporary Related disciplines, such as psychology, anthropology and corpus linguistics It covers the teaching of specific languages, including Japanese, Chinese, Arabic and African languages, as well as English, French, German and Spanish. There are thirty five overview articles dealing with issues such as communicative language teaching, early language learning, teacher education and syllabus and curriculum design. A further 160 entries focus on topics such as bilingualism, language laboratories and study abroad. Numerous shorter items examine language and cultural institutions, professional associations and acronyms. Multiple cross-references enable the user to browse from one entry to another, and there are suggestions for further reading. Written by an international team of specialists, the Routledge Encyclopedia of Language Teaching and Learning is an invaluable resource and reference manual for anyone with a professional or academic interest in the subject.

Learning English as a foreign language in any formal education context requires opportunities for learners and teachers to give and receive feedback on the teaching learning process as it is happening. These opportunities could be created via various in-class activities specifically designed for this purpose. Teachers who create and use these diagnostic opportunities effectively detect what learners need in a timely fashion, and provide remedial teaching in the right time and mode, so that chances can be created for learners to improve their learning. There is no one universally accepted way of how to do this, however, with various approaches for collecting, analyzing and reviewing data for this purpose. This book encapsulates the unbreakable relationship between teaching, learning and assessment through a range of articles which scrutinize assessment from a wide spectrum, ranging from the role of assessment in language learning to ELT teacher assessment literacy, from the use of technology in classroom-based assessment to practicing teachers’ reflections on their teacher classroom action research, and from the role of the Common European Framework of Reference for Languages (CEFR) to empirical data analysis.

This Handbook, with 45 chapters written by the world’s leading scholars in second language acquisition (SLA) and language testing, dives into the important interface between SLA and language testing: shared ground where researchers seek to measure second language performance to better understand how people learn their second languages. The Handbook also reviews how to best measure and evaluate the second language (L2) learners’ personal characteristics, backgrounds, and learning contexts to better understand their L2 learning trajectories. Taking a transdisciplinary approach to research, the book builds upon recent theorizing and measurement principles from the fields of applied linguistics, cognitive science, psychology, psycholinguistics, psychometrics, educational measurement, and social psychology. The Handbook is divided into six key sections: (1) Assessment concepts for SLA researchers, (2) Building instruments for SLA research, (3) Measuring individual differences, (4) Measuring language development, (5) Testing specific populations, and (6) Measurement principles for SLA researchers.