Burn-In Demonstrations for Multi-Modal Imitation Learning

Kuefler, Alex; Kochenderfer, Mykel J. · 2018 · Crossref

DOI: 10.65109/fgmb4505

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of modeling human driving behavior for autonomous systems, specifically focusing on multi-modal imitation learning where expert demonstrations exhibit significant variability in style (e.g., aggressiveness, attentiveness). Previous approaches, such as Information Maximizing Generative Adversarial Imitation Learning (InfoGAIL), often fail to maintain consistency with an expert’s specific style over long time horizons because they sample latent codes randomly at the start of each trial, ignoring the context of the initial trajectory. The authors introduce Burn-InfoGAIL, a method that conditions the learned policy on “burn-in demonstrations”—partial expert trajectories that provide context for the latent style variable. This allows the model to reproduce expert behavior over extended periods by ensuring the latent code reflects the actual style demonstrated in the initial segment. The proposed method reformulates the imitation learning setting using a dynamic Bayesian network where a latent factor $z$ determines driving style. Unlike standard InfoGAIL, which samples $z$ from a prior, Burn-InfoGAIL draws $z$ from a learned inference distribution conditioned on the burn-in demonstration $\tau$. The objective function combines a Wasserstein GAIL term for imitation, a cross-entropy term to ensure the inference model accurately predicts the latent code, and an entropy maximization term to prevent the model from collapsing to a single latent mode. The implementation uses a simulator based on an oval racetrack populated with vehicles exhibiting four distinct driving styles (aggressive, passive, speeder, tailgating). The policy, critic, and inference model are implemented as multilayer perceptrons, trained using Trust Region Policy Optimization (TRPO), RMSProp, and Adam, respectively. Observations include LIDAR data and road features. Experiments demonstrate that Burn-InfoGAIL outperforms standard InfoGAIL, GAIL, and Variational Autoencoder (VAE) baselines. In terms of style recovery, Burn-InfoGAIL achieved the highest Adjusted Mutual Information (AMI) score (0.38) on validation data, significantly surpassing InfoGAIL (0.16) and VAE+K-Means (0.24). In trajectory reproduction, Burn-InfoGAIL maintained the lowest root mean squared error (RMSE) for speed and position over 30-second horizons, whereas GAIL drifted toward an average policy and VAE suffered from cascading errors leading to off-road events. Additionally, Burn-InfoGAIL achieved a balanced safety profile, with lower off-road rates than GAIL and VAE, and lower collision rates than VAE. Qualitative results confirmed that different latent codes corresponded to distinct driving behaviors, such as aggressive lane-changing or passive following, validating the model’s ability to capture multi-modal styles. The significance of this work lies in its ability to enable stable, long-horizon imitation learning that respects the specific style of an expert demonstration. By conditioning on burn-in trajectories, the method overcomes the instability and mode-averaging issues inherent in previous multi-modal imitation learning techniques. This approach is particularly relevant for autonomous driving systems that must seamlessly take over control from human drivers or other agents, requiring the policy to adapt to the immediate context and style of the preceding trajectory. The results suggest that incorporating contextual conditioning into the latent variable sampling process is crucial for robust behavioral cloning in sequential decision-making tasks.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• distraction detection algorithms