Ego-motion and Surrounding Vehicle State Estimation Using a Monocular Camera

Hayakawa, Jun; Dariush, Behzad · 2019 · OpenAlex

DOI: 10.1109/ivs.2019.8814037

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of estimating ego-motion and the state of surrounding vehicles using only a single monocular camera, aiming to reduce the cost and complexity associated with multi-sensor fusion systems like LiDAR and radar. While monocular cameras are low-cost and simple, they lack inherent depth information, making accurate 3D position, velocity, and orientation estimation difficult. The authors identify that existing monocular methods often rely on fixed ground plane assumptions, which fail during vehicle acceleration, braking, or on uneven road surfaces. To solve this, the paper proposes a novel machine learning framework that integrates three deep neural networks to estimate 3D bounding boxes, depth, and optical flow, coupled with a real-time ground plane correction algorithm. The proposed method utilizes a sequence of images as input. First, a 3D bounding box network, based on DenseBox and SSD architectures, detects vehicles and outputs 2D coordinates and height. Second, a depth estimation network, trained on the KITTI dataset using unsupervised learning, predicts pixel-level depth. Third, an optical flow network (PWC-Net) estimates motion vectors. A critical component of the framework is the real-time ground plane correction. The system selects nine fixed points on the road surface in the lower portion of the image, uses the depth estimator to determine their 3D positions, and applies the RANSAC algorithm to fit the ground plane coefficients dynamically. This allows the system to account for changes in road inclination and vehicle pitch/roll. Ego-velocity is calculated from the optical flow of stationary ground regions, while surrounding vehicle velocity is derived by combining relative optical flow within the detected 3D bounding boxes with the estimated ego-velocity. Experimental evaluations were conducted using a dataset captured in Palo Alto, CA, synchronized with LiDAR, GPS, and CAN-BUS data for ground truth verification. The results demonstrate that real-time ground plane correction significantly improves estimation accuracy. Quantitative analysis of 50 samples showed that correcting the ground plane reduced the longitudinal distance error Mean Absolute Error (MAE) from 1.92 meters to 1.24 meters and improved orientation similarity scores from 97.15% to 98.97%. Velocity estimation results for 31 samples indicated a Root Mean Square Error (RMSE) of 1.18 m/s for ego-velocity compared to CAN-BUS data, and an RMSE of 2.49 m/s for surrounding vehicle velocity compared to LiDAR data. Visual comparisons confirmed that the corrected method provides more stable and accurate 3D position and orientation estimates than methods without ground plane correction. The significance of this work lies in demonstrating that high-accuracy 3D state estimation for autonomous driving can be achieved using a cost-effective monocular camera setup. By addressing the limitations of fixed ground plane assumptions through real-time correction and integrating multiple deep learning models, the proposed framework offers a viable alternative to expensive LiDAR-based systems. The results suggest that this approach provides sufficient accuracy for decision-making and path planning in automated vehicles, facilitating the broader adoption of advanced driving assistance technologies by minimizing hardware costs and complexity.

Key finding

Real-time ground plane correction integrated into a monocular camera-based deep learning framework significantly improves the accuracy of 3D position, orientation, and velocity estimation for both ego and surrounding vehicles.

Methodology

on_road

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-27.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• situational awareness