Knowledge Transfer via Dynamic Policy Fusion Between Autonomous Driving Agents

Maad, Fatima-Ezzahra; Guériau, Maxime; Ainouz, Samia · 2025 · OpenAlex-citations

DOI: 10.1109/ictai66417.2025.00193

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of enabling autonomous vehicles (AVs) to adapt to new, unsignalized driving environments—specifically T-intersections and ramp merges—with minimal exploration and without retraining from scratch. The authors propose a Fine-tuned Transfer Learning framework that facilitates dynamic knowledge transfer between agents. The core problem is that standard reinforcement learning requires extensive, often unsafe, random exploration in new scenarios, while existing transfer learning methods suffer from issues like catastrophic forgetting or domain mismatch. The proposed solution allows a "source" agent (an expert in one scenario) to share its learned policy with a "target" agent (the ego vehicle encountering a new scenario) via a dynamic policy fusion mechanism. The methodology employs a Markov Decision Process (MDP) modeled in the SUMO microscopic traffic simulator. Two agents are defined: a Red agent expert in T-intersections and an Orange agent expert in ramp merging. The authors compare their proposed method, "Transfer Learning with Fine-Tuning" (TL + fine-tuning), against several baselines: expert agents trained in the target environment, Zero-Shot Learning (using pre-trained models without adaptation), and Basic Transfer Learning (minimal training without expert guidance). The key innovation is a Q-Learning policy fusion mechanism where the target agent’s Q-values are dynamically blended with the source agent’s Q-values using a time-dependent confidence coefficient ($\epsilon_t$). Initially, the target agent relies heavily on the source expert’s policy ($\epsilon_t \approx 1$), and as it gains experience in the new environment, it gradually shifts reliance to its own learned policy. This fusion occurs without additional training overhead, allowing immediate, context-aware decision-making. Experimental results demonstrate that the proposed dynamic policy fusion significantly outperforms baseline approaches in both safety and efficiency. In cross-domain transfer scenarios (e.g., the Red agent navigating a ramp merge), the method achieved a 26% reduction in collision rates compared to no-transfer baselines. The fused policy enabled faster convergence to optimal speeds and maintained safer inter-vehicle distances. Specifically, the tabular Q-Learning agent with policy fusion showed higher episode success rates than Deep Q-Network (DQN) baselines, indicating more robust decision-making in these complex, unsignalized environments. The approach effectively balanced the trade-off between minimizing travel time and ensuring operational safety, avoiding the unsafe random exploration typical of standard reinforcement learning in novel settings. The significance of this work lies in its demonstration that context-aware, dynamic policy sharing can accelerate AV adaptation to unseen environments while enhancing safety. By leveraging peer-to-peer knowledge transfer via Q-value fusion, autonomous systems can generalize learned behaviors across structurally related tasks (such as intersections and merges) without the computational cost of retraining. This approach offers a viable pathway for collaborative autonomous driving systems, where vehicles can share expertise to improve real-time decision-making, reduce data requirements, and ensure safer interactions in dynamic, mixed-traffic environments.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• situational awareness