A deep learning based stereo matching model for autonomous vehicle

Deepa, Deepa; Kupparu, Jyothi · 2023 · Crossref

DOI: 10.11591/ijai.v12.i1.pp87-95

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of accurate depth estimation for autonomous vehicles, specifically focusing on stereo matching in ill-posed regions such as occluded, texture-less, and discontinuous areas. While Light Detection and Ranging (LiDAR) sensors provide depth data, they are expensive and lack semantic information like traffic light colors. Traditional stereo matching methods, including local, global, and semi-global algorithms, often fail to produce accurate results in these difficult regions due to reliance on hand-crafted features. The authors propose a hybrid Convolutional Deep Stereo Network (CDSN) that combines Convolutional Neural Networks (CNNs) with Generative Adversarial Networks (GANs) to generate reliable disparity maps. The methodology involves a two-stage process. First, feature descriptors are extracted from rectified stereo images using the 9th layer of a pre-trained VGG-16 model. These features are used to compute unary and smoothness costs, which are minimized using a max-product variation of Loopy Belief Propagation (LBP) over 10 iterations to generate an initial disparity map. Second, to refine this map and handle ill-posed regions, the authors employ a Pix2Pix GAN model. This GAN uses a U-Net auto-encoder generator trained with adversarial loss and L1 loss to translate the initial disparity map into a refined one that closely matches the ground truth. The model was implemented using PyTorch and trained on the Middlebury stereo dataset for 300 epochs with an Adam optimizer and a learning rate of 0.0002. Experimental results demonstrate that the proposed CDSN model outperforms existing techniques, including Deep Pruner, ACR-GIF-OW, and LPSM. Quantitative evaluation on Middlebury 2014 images ("Jade plant," "Piano," "Pipes," and "Recycle") shows that CDSN achieves lower Percentage of Bad Matching Pixels (PBMP) and Root Mean Square Error (RMSE) compared to the baselines. For instance, the average RMSE for CDSN was 8.04, significantly lower than Deep Pruner (12.58) and ACR-GIF-OW (31.02). An ablation study further confirmed the superiority of the Pix2Pix GAN over CycleGAN and DualGAN, yielding lower Absolute Relative Distance (ARD) and RMSE values. Qualitative results indicate that the model effectively fills occluded areas and produces smoother disparity maps. The significance of this work lies in providing a cost-effective, computer vision-based alternative to expensive hardware sensors for autonomous driving systems. By integrating CNN-based feature extraction with GAN-based refinement, the model successfully addresses the persistent problem of inaccurate matching in ill-posed regions. The findings suggest that this hybrid approach enhances the safety and reliability of autonomous vehicles by providing more precise 3D environmental perception, particularly in complex visual scenarios where traditional methods fail.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• situational awareness