SynthoGestures: A Multi-Camera Framework for Generating Synthetic Dynamic Hand Gestures for Enhanced Vehicle Interaction
DOI: 10.1109/IV55156.2024.10588662
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the challenge of creating diverse, high-quality datasets for dynamic hand gesture recognition in automotive human-machine interfaces. Collecting real-world gesture data in dual-task driving scenarios is expensive and time-consuming, while existing synthetic data tools often produce static, unrealistic gestures that fail to generalize. To overcome these limitations, the authors introduce **SynthoGestures**, a framework built on Unreal Engine that generates realistic, dynamic hand gestures with customizable variations in speed, performance, and hand shape. The system simulates multiple camera viewpoints (e.g., above the driver, behind the wheel) and sensor types (RGB, infrared, depth) with hardware-specific noise modeling, aiming to augment or replace real datasets to improve model generalization and reduce overfitting. The methodology involves a systematic generation process where users define initial settings via JSON files or direct input. The framework iterates through camera configurations, gesture definitions, and performance parameters to produce comprehensive datasets. It supports both description-based and animation-based gesture execution, utilizing splines and inverse kinematics (Control Rig) to ensure natural arm and hand movements. Specific sensor simulations include grayscale noise textures for depth cameras and Fresnel effects for infrared cameras. For evaluation, the authors generated 600 synthetic gesture videos (100 variants each) for six distinct gestures (e.g., swipes, peace signs, rotations) using varied parameters for speed, position, and finger spacing. These synthetic data were combined with the NVIDIA Dynamic Hand Gesture Dataset, which provided real-world data from 20 participants. The results demonstrate that SynthoGestures significantly enhances gesture recognition accuracy when used to augment real data. Using a state-of-the-art recognition model, the authors compared baseline real-data training against models pre-trained or jointly trained with synthetic data. Models trained simultaneously with synthetic and real data achieved an accuracy of **89.58%**, compared to **79.86%** for sequential training and **27.78%** for synthetic-only training. The baseline real-data-only model achieved lower performance than the augmented models. Further analysis revealed that the range of variations (speed, position, finger spacing) critically impacts performance; median-range variations yielded optimal results, while low-range variations reduced accuracy and high-range variations introduced excessive noise. The framework successfully generated 108,864 synthetic gestures, proving its capacity to create large-scale, diverse datasets efficiently. The significance of this work lies in providing a cost-effective, flexible tool for generating synthetic dynamic gestures that closely mimic real-world conditions. By enabling the simulation of multiple sensor types and camera angles without additional hardware costs, SynthoGestures facilitates the development of robust gesture recognition systems for automotive applications. The findings confirm that synthetic data can effectively augment real datasets, improving model generalization and mitigating biases inherent in small real-world datasets. This approach accelerates the development of in-vehicle interaction systems by reducing the reliance on extensive, costly data collection efforts.
Key finding
Training gesture recognition models with a combination of synthetic and real data yielded significantly higher accuracy than training exclusively with real data.
Methodology
simulation_modeling
Sample size: 20
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via openalex_abstract on 2026-05-08 (3 acquisition events logged).
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | — | — | — | 1 | 2026-05-07 |
| archive | success | canonical_url | — | — | 9 | 2026-06-06 |
| extract | success | cached | — | — | 3 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-04 |
| chunk | success | chunk | — | — | 1 | 2026-06-04 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-04 |
| enrich | success | openalex | — | — | 3 | 2026-05-08 |
| promote | success | — | — | — | 1 | 2026-05-07 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Methodological Resource: tool software