Image Processing Approaches to Traffic Situation Understanding, Risk Assessment, and Safety

Yang, Dongfang; Ozguner, Umit; Redmill, Keith; Yurtsever, Ekim; Capito, Linda; Zhang, Haolin · 2023 · ROSA P / Mobility21 (UTC)

archive: archived pipeline: cataloged verified

Get this paper ↗ (full text — opens at the source; we link to it, we don't host it)

Summary

This final research report details seven distinct studies investigating image processing techniques for automated driving, traffic surveillance, and safety monitoring. The work addresses the limitations of conventional rule-based systems and pure learning-based approaches by integrating computer vision with control theory, deep reinforcement learning, and sensor fusion. The primary motivation is to enhance the robustness, safety, and realism of intelligent vehicle systems and traffic analysis tools. The methodologies span several domains. For automated driving, the authors developed a hybrid framework integrating Deep Reinforcement Learning (DRL) with model-based path planners, where a DRL agent learns to follow waypoints generated by an A* algorithm while prioritizing collision avoidance. Another study utilized sparse optical flow from monocular cameras to generate visual potential fields for obstacle detection and vehicle control via a Gradient Tracking Sliding Mode Controller. For traffic surveillance, the team leveraged existing cameras on transit buses, employing YOLOv4 for detection, SORT for tracking, and homography transformations to extract vehicle counts and trajectories in bird’s-eye-view coordinates. Pedestrian safety was addressed through two approaches: a system using Faster R-CNN to detect social distancing violations and estimate critical crowd density during the COVID-19 pandemic, and a spatio-temporal attention model fusing visual and non-visual features to predict pedestrian crossing intentions. Additionally, the report presents “Faraway-Frustum,” a method fusing 2D vision with sparse LiDAR points to improve 3D object detection at long ranges, and a generative approach blending Conditional Generative Adversarial Networks (cGANs) with partial rendering to create photorealistic driving simulations. The findings demonstrate significant improvements over baseline methods. The hybrid DRL agent achieved higher normalized rewards and faster navigation in CARLA simulations compared to end-to-end counterparts. The optical flow-based controller successfully navigated synthetic and real-world datasets, maintaining trajectory accuracy even in rainy conditions. The bus-based surveillance system achieved near-ideal vehicle counting accuracy by using dynamic regions of interest to exclude parked vehicles, outperforming alternative detector-tracker combinations. The pedestrian intention prediction model achieved state-of-the-art performance on the JAAD and PIE datasets, with ablation studies confirming the efficacy of hierarchical feature fusion. The Faraway-Frustum method successfully detected distant pedestrians and cars where standard LiDAR-based methods failed due to point sparsity. Finally, the blended simulation imagery achieved higher Inception scores and better semantic retention than pure rendering or pure GAN approaches, closely matching real-world data distributions. These results imply that hybrid architectures combining model-based constraints with learning-based adaptability offer superior safety and performance in automated driving. Furthermore, utilizing existing infrastructure like transit buses provides a cost-effective solution for high-resolution traffic monitoring. The advancements in pedestrian intention prediction and long-range sensor fusion directly contribute to safer human-vehicle interactions, while the improved simulation techniques facilitate more robust training environments for autonomous systems.

Key finding

The project demonstrates that integrating image processing techniques, such as deep reinforcement learning, optical flow, and sensor fusion, significantly enhances automated driving capabilities in path planning, traffic surveillance, object detection, and simulation realism.

Methodology

mixed_methods

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via bulk_ingest_rosap on 2026-05-23 (6 acquisition events logged).

Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• situational awareness

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Methodological Resource: tool software
• Theoretical Contribution: computational model