Convergent validity of video-based observer rating of drowsiness, against subjective, behavioral, and physiological measures

Uchiyama, Yuji; Sawai, Shunichiroh; Omi, Takuhiro; Yamauchi, Koichiro; Tamura, Kimimasa; Sakata, Takuya; Nakajima, Kiyofumi; Sakai, Hiroyuki · 2023 · Crossref

DOI: 10.1371/journal.pone.0285557

archive: archived pipeline: cataloged

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study addresses the convergent validity of video-based Observer Rating of Drowsiness (ORD), a method widely used as a ground truth for driver drowsiness in crash risk assessments and detection system development. Despite its prevalence, concerns persist regarding whether ORD levels reliably correlate with other established drowsiness metrics. The research aimed to validate video-based ORD by examining its correlations with subjective, behavioral, and physiological measures. The study involved 17 participants who completed eight sessions of a simulated driving task. During these sessions, participants performed a psychomotor vigilance task and an 18-minute simulated drive on a monotonous expressway at night, verbally responding to the Karolinska Sleepiness Scale (KSS) at specific intervals. Data collected included infrared facial video, lateral vehicle position, eye closure, electrooculography (EOG), and electroencephalography (EEG). Three experienced raters evaluated ORD levels by observing facial video segments, using a 5-point scale (D1–D5) based on Japanese Ministry of Land, Infrastructure, Transport, and Tourism guidelines. Inter-rater reliability was confirmed with a concordance rate of at least 0.7. The analysis focused on calculating correlation coefficients between ORD scores and other drowsiness indicators: KSS, standard deviation of lateral position (SDLP), percentage of time occupied by eye closure (PERCLOS), percentage of time occupied by slow eye movement (SEM), and EEG alpha and theta power. The results demonstrated significant positive correlations between ORD levels and all examined drowsiness measures. Specifically, higher ORD scores were associated with higher KSS ratings, increased SDLP, greater PERCLOS, higher percentages of SEM, and elevated EEG alpha and theta power. These findings confirm that video-based ORD aligns with subjective self-reports, behavioral driving errors, and physiological markers of sleepiness. The significance of this work lies in providing empirical support for the convergent validity of video-based ORD. By establishing strong correlations with diverse drowsiness metrics, the study suggests that ORD is a suitable ground truth for evaluating driver drowsiness. This validation is crucial for the development of reliable drowsiness detection systems and for interpreting crash risk data in naturalistic driving studies, where ORD is frequently employed as a reference standard.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.8-27b-gittensor on 2026-08-23; verification: pending re-verification.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• drowsiness detection algorithms
• drowsiness
• microsleep
• workload measurement
• dms validation
• drowsy as impairment

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: physiological data
• Methodological Resource: validation psychometrics, measurement protocol