A CNN2D-LSTM Framework for Rule-Based Pedestrian-Vehicle Risk Scenario Detection

BENKHADDA, Oumaima; MANDAR, Meriem · 2026 · Crossref

DOI: 10.19139/soic-2310-5070-3458

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the critical safety challenge of pedestrian-vehicle interactions, motivated by high pedestrian fatality rates, such as the 24.7% share of road traffic deaths in Morocco in 2024. The authors propose a deep learning framework to automatically detect "risky" versus "non-risky" scenarios from video sequences, aiming to enhance intelligent driver-assistance systems and accident prevention. The study focuses on overcoming the limitations of existing methods that either ignore temporal dynamics or rely on rigid rule-based assumptions, by leveraging a hybrid architecture that jointly analyzes spatial visual features and temporal dependencies. The proposed method utilizes a CNN2D-LSTM architecture. A two-dimensional convolutional neural network (CNN2D) extracts compact spatial visual features from individual video frames, which are then fed into a Long Short-Term Memory (LSTM) network to model temporal dependencies across consecutive frames. The final hidden state is passed to a fully connected layer for binary risk classification. The model was trained and evaluated on the JAAD dataset, which contains 346 annotated videos. Crucially, the model does not use explicit numerical inputs like distance or speed; instead, these variables were used only to generate heuristic ground-truth labels. Risk labels were derived using rules based on pedestrian crossing status, gaze direction, vehicle action, and estimated proximity (calculated via YOLOv11 detection and monocular geometry). Hyperparameters were optimized using Optuna, and class imbalance was handled via weighted binary cross-entropy loss. Experimental results demonstrate that the hybrid CNN2D-LSTM model significantly outperforms standalone CNN and LSTM baselines. The CNN-only model failed to capture temporal dynamics, resulting in low recall (0.26) and high false negatives. The LSTM-only model, while better, still missed 153 true risk events. In contrast, the proposed hybrid model achieved an overall accuracy of 97.1%, with a precision of 92.2% and a recall of 99.0% for the "Risk" class. This high recall indicates that the model successfully detects nearly all dangerous situations, minimizing critical false negatives. The model also achieved an F1-score of 0.955 and demonstrated robust performance across varying decision thresholds, as evidenced by near-perfect ROC-AUC values. The significance of this work lies in its demonstration that fusing spatial and temporal visual information is essential for accurate risk detection in complex traffic environments. The framework effectively captures subtle contextual cues, such as trajectory changes and occlusions, without requiring explicit sensor data inputs. The results suggest that this lightweight, spatio-temporal approach offers a reliable and computationally efficient solution for early-stage risk detection, providing a strong foundation for future developments in interpretable and deployable traffic safety systems.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• telematics crash prediction