This state-of-the-art resource brings together the most innovative scholars and thinkers in the field of testing to capture the changing conceptual, methodological, and applied landscape of cognitively-grounded educational assessments.

Table of Contents

• List of Tables
• List of Illustrations
• Dedication
• Notes on Contributors
• Foreword
• Acknowledgements
• 1 Introduction to Handbook
• Motivation for Handbook
• Handbook Structure
• Closing Words
• Part I: Frameworks
• 2 The Role of Theories of Learning and Cognition in Assessment Design and Development
• A Brief History of Evaluation Criteria for Theories of Learning and Cognition
• Principled Assessment Design as an Evaluative Lens for Theories of Learning and Cognition
• Principled Assessment Design Evaluation Criteria for Theories of Learning and Cognition
• Interface of Evaluation Criteria with Validity
• Conclusion
• References
• 3 Principled Approaches to Assessment Design, Development, and Implementation
• Motivations for Principled Approaches
• Overviews of Principled Approaches
• Elements of Principled Approaches
• Use of the Elements in Principled Approaches
• Discussion
• Acknowledgments
• References
• 4 Developing and Validating Cognitive Models in Assessment
• Definition of Cognitive Models
• Model Development and Validation Methods
• Offline Evidence: Methods and Approaches
• Conclusion
• References
• 5 An Integrative Framework for Construct Validity
• Unified Framework for Construct Validity
• Example: Test Form Development with Generated Items
• Conclusion
• References
• 6 The Role of Cognitive Models in Automatic Item Generation
• Introduction
• The Relentless Demand for Test Items
• Automatic Item Generation: A Three‐Step Method
• Identifying Content for Item Generation Using Cognitive Modeling
• Evaluating the Cognitive Models Used in Automatic Item Generation
• Summary and Directions for Future Research
• Acknowledgments
• References
• 7 Social Models of Learning and Assessment
• Introduction
• Defining “Social” Models of Learning
• Examples of Assessment Informed by Social Models of Learning
• Example 1: Carbon: Transformations in Matter and Energy (Carbon TIME)
• Example 2: Contingent Pedagogies
• Goals for Learning and their Justification
• Example 3: Data Modeling
• Comparing the Examples: Key Sources of Variation
• Looking to the Future: Additional Considerations for Assessment from a Sociocultural Perspective
• References
• 8 Socio‐emotional and Self‐management Variables in Learning and Assessment
• Introduction
• Importance of Social, Emotional, and Self‐Management Variables in School and in Life: Correlational Studies
• A Social, Emotional, and Self‐Management Framework
• Role of Socio‐Emotional, Attitudinal, and Self‐Management Variables on Summative and Formative Test Scores
• Assessing Socio‐Emotional, Attitudinal, and Self‐Management Variables
• Conclusions and Future Directions
• References
• 9 Understanding and Improving Accessibility for Special Populations
• Accessibility of Educational Assessments
• Cognitive Models of Learning and Performance
• Putting It All Together: Assessing Accessibility for Individual Students
• Conclusion
• References
• 10 Automated Scoring with Validity in Mind
• An Overview of Automated Scoring Applications
• Automated Scoring with Validity in Mind
• Design Decisions in Automated Scoring
• Discussion
• Acknowledgments
• References
• Part II: Methodologies
• 11 Explanatory Item Response Models
• Explanatory Item Response Models: An Approach to Cognitive Assessment
• EIRM Framework and Modeling Principles
• Data Set and Research Questions
• Software and Model Specification
• Example 1: Test Construction and Item Generation
• Example 2: Diagnostic Assessment
• Example 3: Correction for Confounding
• General Discussion
• Acknowledgments
• References
• 12 Longitudinal Models for Repeated Measures Data
• Reading Comprehension Data
• Latent Growth Curve Models
• Growth Mixture Models
• Conclusions
• References
• 13 Diagnostic Classification Models
• Technical Foundations for DCMs
• Evaluating Cognitive Theories through DCM Criticism and Refinement
• Design Considerations for Diagnostic Assessments
• Concluding Remarks
• References
• 14 Bayesian Networks
• Evidentiary Motivations and Logical Bases
• Statistical Definition and Estimation
• Building and Using a BN
• Understanding Models and Results across Communities
• Conclusion
• References
• 15 The Rule Space and Attribute Hierarchy Methods
• Test Design with the RSM and the AHM
• Statistical Pattern Classifications with the RSM and the AHM
• Demonstration of RSM and AHM Using a Real Data Example
• Summary and Future Directions
• References
• 16 Educational Data Mining and Learning Analytics
• Emerging Complex Assessment and Learning Environments
• Case Studies in Assessment of Cognition via EDM
• Conclusions
• References
• Part III: Applications
• 17 Large‐Scale Standards‐Based Assessments of Educational Achievement
• Challenges of Using Cognitive Modelsfor Large‐Scale Assessments
• Examples of Best Practices from Current Testing Programs
• Conclusions and Recommendations
• References
• 18 Educational Survey Assessments
• Introduction
• Building Cognitive Models
• The Role of Cognitively‐based Assessment in ESAs
• Definition and Guiding Examples
• Three Methodological Perspectives
• Summary, Trends and Upcoming Development
• References
• 19 Professional Certification and Licensure Examinations
• Overview of Certification and Licensure
• Assessment Frameworks for Certification and Licensure
• Meeting Practical Certification and Licensure Assessment Challenges
• Conclusions
• References
• 20 The In‐Task Assessment Framework for Behavioral Data
• Introduction
• Evidence‐Centered Design
• In‐task Assessment Framework
• Example I‐TAF Instantiations
• Leveraging the I‐TAF Framework to Instantiate ECD Models
• Discussion
• Acknowledgments
• References
• 21 Digital Assessment Environments for Scientific Inquiry Practices
• Foundations for the Design of the Inq‐ITS System
• Case Studies with Think‐Aloud Components
• The Inq‐ITS System
• Summary
• References
• 22 Assessing and Supporting Hard‐to‐Measure Constructs in Video Games
• Hard‐to‐measure Constructs and Why They are Hard to Measure
• Well‐designed Games as Vehicles for Assessing and Supporting These Constructs
• Evidence‐centered Design and Stealth Assessmentin Well‐designed Games
• Examples of Stealth Assessments of Hard‐to‐Measure Constructs in Two Games
• Validation of In‐Game Measures
• Discussion
• Acknowledgments
• References
• 23 Conversation‐Based Assessment
• Performance‐Based and Conversation‐Based Assessments
• An Augmented Taxonomy for Assessment Tasks
• An Illustration of Conversation‐Based Assessments
• Discussion
• References
• 24 Conclusion to Handbook
• Starting Points for Reflection
• Final Words
• References
• Glossary