Deep Learning Based Motion Planning For Autonomous Vehicle Using Spatiotemporal LSTM Network

Bai, Zhengwei; Cai, Baigen; ShangGuan, Wei; Chai, Linguo · 2018 · Crossref

DOI: 10.1109/cac.2018.8623233

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of robust motion planning for autonomous vehicles in complex, dynamic traffic environments. Traditional model-based approaches often struggle with the uncertainty of edge cases and the complexity of real-world scenarios. While previous end-to-end deep learning methods, such as NVIDIA’s DAVE2, offered scalability, they suffered from large model volumes and lacked theoretical support due to their "black box" nature. To overcome these limitations, the authors propose a novel deep learning model called the Spatiotemporal LSTM Network, designed to generate real-time steering commands by effectively extracting spatiotemporal features from sequential visual data. The proposed architecture combines Convolutional Long-Short Term Memory (Conv-LSTM) layers with a 3D Convolutional Neural Network (3D-CNN) and Fully Connected Neural Networks (FCNN). The system processes multi-frame image segments, created by combining three consecutive frames to capture temporal dynamics. Four layers of Conv-LSTM, augmented with batch normalization to prevent distribution offset, extract hidden temporal features. These are followed by a 3D-CNN layer, which captures spatial features and inter-frame motion information more effectively than traditional 2D-CNNs. Finally, FCNN layers with LeakyReLU activation and dropout regularization predict the vehicle’s steering angle. The model was trained using the Comma dataset, which contains over 80GB of raw image data and vehicle state metrics collected at 20Hz. Training utilized the Keras framework with the Adam optimizer and Mean Square Error loss function. Experimental results demonstrate that the Spatiotemporal LSTM Network generates robust and accurate motion planning outputs. Visual comparisons showed that the predicted steering angles closely matched the actual driving motions. When compared against a baseline method proposed by Hotz et al. (based on 2D-CNN and FCNN), the proposed model exhibited superior performance. Specifically, the Spatiotemporal LSTM Network achieved better average evaluation performance and lower variation, indicating greater stability and reliability in generating autonomous motion planning results. The authors attribute this improvement to the model's ability to leverage both temporal memory from Conv-LSTM and spatial feature extraction from 3D-CNN, allowing it to handle the continuous nature of driving better than discrete frame-based approaches. The significance of this work lies in its contribution to more generalizable and stable autonomous driving control systems. By integrating spatiotemporal feature extraction, the proposed method enhances the adaptability of deep learning models to complex environments without relying on rigid model-based controls. The findings suggest that combining Conv-LSTM with 3D-CNN is a promising direction for improving the accuracy and reliability of end-to-end autonomous vehicle navigation, potentially leading to safer and more efficient self-driving technologies.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• situational awareness