Uncertainty-Aware Reinforcement Learning for Collision Avoidance

Kahn, Gregory; Villaflor, Adam; Pong, Vitchyr; Abbeel, Pieter; Levine, Sergey · 2017 · OpenAlex-citations

DOI: 10.48550/arxiv.1702.01182

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the safety challenges inherent in deploying reinforcement learning (RL) for autonomous robots navigating unknown environments. The core problem is that to learn collision avoidance, a robot must experience collisions during training; however, high-speed collisions can cause catastrophic damage. The authors propose an uncertainty-aware, model-based RL algorithm that enables a robot to navigate safely by estimating the probability of collision alongside a statistical measure of uncertainty. This allows the robot to proceed cautiously in unfamiliar situations and increase speed only when confident in its predictions. The method employs a collision prediction model based on deep neural networks that process raw sensory inputs, such as camera images. To estimate uncertainty, the authors combine bootstrapping (training multiple models on resampled data) with dropout (randomly masking units during training and inference). This ensemble approach provides variance estimates for the collision probability. The algorithm integrates these estimates into a risk-averse collision predictor, which feeds into a velocity-dependent collision cost function. Specifically, the cost penalizes speed proportional to the predicted collision probability and its uncertainty. This cost is used within a receding-horizon model-predictive control (MPC) framework to select actions. Consequently, the robot naturally chooses low-speed actions when uncertainty is high, ensuring that any collisions experienced during training are gentle and non-destructive. Experimental evaluations were conducted on simulated and real-world quadrotors, as well as a real-world RC car. The results demonstrate that the uncertainty-aware approach significantly reduces the speed of collisions experienced during training compared to baseline methods that do not explicitly reason about uncertainty. By tuning the uncertainty weight parameter, the system achieves a favorable trade-off between training safety and final task performance. While higher uncertainty weighting leads to slower learning due to conservative behavior, it effectively prevents high-speed crashes. The method allows the robot to learn effective navigation policies while maintaining safety, validating the utility of combining bootstrapping and dropout for uncertainty estimation in safety-critical robotic applications.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• time to collision looming