Assessment in Driving Simulators: Where We Are and Where We Go

Kappé, Bart; de Penning, Leo; Marsman, Maarten; Roelofs, Erik · 2009 · Crossref

DOI: 10.17077/drivingassessment.1320

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper outlines the initial design and theoretical framework for a three-year project initiated in 2008 by TNO, in collaboration with the Dutch licensing authority (CBR), RCEC, ANWB driving schools, and ADS Technique Inc. The research addresses the challenge of validating driving simulators for formal driver assessment and testing, a domain where simulator use has historically been limited compared to research and rehabilitation. The primary motivation is to develop a generic, automated testing module that can reliably estimate a candidate’s driving ability by combining objective simulator performance data with subjective human rater judgments. The authors argue that while simulators offer deterministic, repeatable traffic scenarios ideal for assessment, previous studies have shown low correlations between simulator performance and practical test outcomes, often due to a focus on low-level control tasks rather than higher-order competencies like hazard perception and traffic participation. The proposed methodology involves developing a computer-adaptive test using Item Response Theory (IRT) to select specific traffic scenarios based on the examinee’s estimated ability. The project utilizes driving simulators at ANWB driving schools, which feature VW Golf mock-ups with full controls and a 180-degree field of view. Approximately 50 standardized test items, representing specific traffic assignments (e.g., merging, turning), will be administered to hundreds of students. Performance data will be collected alongside ratings from driving instructors and examiners. A key component of the design is an adaptive cognitive model using hybrid neural-symbolic learning. This model aims to map the relationship between contextual item information, objective simulator metrics (such as Time to Line Crossing), and subjective rater judgments to generate automated assessments. To ensure interoperability, the assessment module is built on the SimSCORM platform, which integrates SCORM e-learning standards with HLA simulation frameworks, allowing test items and results to be managed via the MOODLE learning management system. The paper does not present empirical results, as it describes the mindset and plans at the start of the project. However, it reviews existing literature to justify the approach, noting that previous simulator assessments had high correlations with instructor judgments when bias was controlled, but lower predictive validity for practical tests when focusing solely on steering errors. The authors posit that by targeting basic procedural skills and traffic participation rules—areas where simulators are effective—and using a robust statistical model to handle rater bias and measurement error, the system can provide a valid assessment of driving ability. The significance of this work lies in its potential to standardize simulator-based assessment, creating a transparent, interoperable tool that could facilitate both driver testing and research. By bridging the gap between objective simulator data and subjective expert judgment through adaptive cognitive modeling, the project aims to establish driving simulators as a reliable alternative or supplement to on-road testing.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-09; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• simulator training transfer
• simulator validity fidelity
• simulator sickness
• situational awareness
• fitness to drive assessment
• dms validation

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Methodological Resource: validation psychometrics, tool software
• Theoretical Contribution: computational model