E2ETrADS: end-to-end transformer based autonomous driving system for adverse weather conditions

Spanogianopoulos, Sotirios; Ahiska, Kenan · 2026 · DOAJ

DOI: 10.1186/s12544-026-00793-6

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper introduces E2ETrADS, an end-to-end transformer-based autonomous driving system designed to maintain robust performance under adverse weather conditions such as rain, fog, snow, and low illumination. The research addresses the significant challenge of sensor degradation in autonomous vehicles (AVs), where conventional modular pipelines and CNN-based fusion methods often fail due to noisy or sparse data from LiDAR and cameras. By leveraging the self-attention mechanisms of transformers, the proposed framework aims to capture long-range dependencies and inter-modal relationships, allowing the system to dynamically weigh sensor inputs based on contextual relevance rather than relying on fixed spatial receptive fields. The system was developed and evaluated using the CARLA simulator, which provides high-fidelity sensor models and diverse weather presets. A comprehensive dataset was generated by training an expert driver controller consisting of a weather-adaptive Model Predictive Control (MPC) planner and Proportional-Integral-Derivative (PID) controllers. This expert controller prioritizes safety by adjusting speed and maneuverability based on weather conditions. The E2ETrADS model, which processes RGB camera and LiDAR inputs through separate encoders before fusing them in a six-layer transformer encoder, was trained via behavioral cloning (imitation learning) to replicate the expert’s control commands. The training utilized 72,000 camera frames and 36,000 LiDAR scans collected over approximately 200 hours of simulated driving in two distinct towns, with one used for training and the other for testing to ensure generalization. Experimental results demonstrate that E2ETrADS outperforms the TransFuser baseline in adverse weather scenarios. While TransFuser achieved a higher route completion rate (86.5% vs. 72.3%), it incurred significantly more infractions, resulting in a lower driving score (61.2 vs. 66.3). E2ETrADS exhibited fewer collisions, lane invasions, and missed turns by dynamically reducing vehicle speed to maintain control stability and adhering to safer trajectories. The system successfully internalized behavioral strategies that generalize across varying weather conditions, showing human-like reasoning in safety-critical long-tail scenarios. Although the safety-prioritizing approach led to lower route completion scores due to time limits, the reduction in infractions highlights the model’s ability to preserve mission integrity under sensor uncertainty. The significance of this work lies in demonstrating that transformer-based end-to-end architectures can effectively handle multimodal sensor degradation without explicit fusion algorithms. By integrating weather-aware control policies into the learning process, E2ETrADS provides a more resilient alternative to traditional modular systems. The findings suggest that such frameworks are promising for real-world deployment, though the authors note that future work must address the simulation-to-reality gap and sensor mismatch issues to bridge the divide between synthetic training and physical validation.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-25; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• automation surprise