This book explores the use of machine learning and NLP in digital assessment, highlighting their applications in automated scoring, feedback, cheating detection, and cognitive diagnosis. It covers topics like automated item generation, adaptive testing, and science assessment.
With the exponential increase of digital assessment, different types of data in addition to item responses become available in the measurement process. One of the salient features in digital assessment is that process data can be easily collected. This non-conventional structured or unstructured data source may bring new perspectives to better understand the assessment products or accuracy and the process how an item product was attained. The analysis of the conventional and non-conventional assessment data calls for more methodology other than the latent trait modeling.
Natural language processing (NLP) methods and machine learning algorithms have been successfully applied in automated scoring. It has been explored in providing diagnostic feedback to test-takers in writing assessment. Recently, machine learning algorithms have been explored for cheating detection and cognitive diagnosis. When the measurement field promote the use of assessment data to provide feedback to improve teaching and learning, it is the right time to explore new methodology and explore the value added from other data sources. This book presents the use cases of machine learning and NLP in improving the assessment theory and practices in high-stakes summative assessment, learning, and instruction. More specifically, experts from the field addressed the topics related to automated item generations, automated scoring, automated feedback in writing, explainability of automated scoring, equating, cheating and alarming response detection, adaptive testing, and applications in science assessment. This book demonstrates the utility of machine learning and NLP in assessment design and psychometric analysis.
Chapter
1. The Item Factory: Intelligent Automation in Support of Test
Development at Scale; Alina A. von Davier, Andrew Runge, Yena Park, Yigal
Attali, Jacqueline Church, and Geoff LaFlair.
Chapter
2. Applications of Transformer Neural Networks in Processing Examinee
Responses; Susan Lottridge.
Chapter
3. Integrating Natural Language Processing for Writing Assessment:
Writing Trait Model; Paul Deane and Duanli Yan.
Chapter
4. Empirical Ensemble Equating Under the NEAT Design Inspired by
Machine Learning Ideology; Zhehan Jiang, Yuting Han, Jihong Zhang, Lingling
Xu, Dexin Shi, Haiying Liang, and Jinying Ouyang.
Chapter
5. Test Security in Remote Testing Age: Perspectives From Process
Data Analytics and AI; Jiangang Hao and Michael Fauss.
Chapter
6. Using Language Models to Detect Alarming Student Responses;
Chirstopher Ormerod, Milan Patel, and Harry Wang.
Chapter
7. EPIC Analysis: Evaluating Phrases in Context to Better Understand
AI Scoring of Essays; Steven Tang.
Chapter
8. Fully Data-Driven Computerized Adaptive Testing; Aritra Ghosh,
Wanyong Feng, Stephen Sireci, and Andrew Lan.
Chapter
9. From Adaptive Testing to Personalized Adaptive Testing:
Applications of Recommender Systems; Okan Bulut.
Chapter
10. AI and Machine Learning for Next Generation Science Assessments;
Xiaoming Zhai.
Chapter
11. An AI-Based Platform for Real Time Assessment, Scaffolding, and
Alerting on Students' Science Practices; Janice D. Gobert and the Inq-ITS
Research & Development Team.
