ASP-Driven Visual Commonsense: A General Framework for Reasoning About Embodied Interaction in the Wild
DOI: 10.24963/kr.2025/61
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper introduces a general, neurosymbolic framework for visual commonsense reasoning designed to support embodied interaction in naturalistic, "in-the-wild" settings. The research addresses the challenge of enabling autonomous systems to perform active sensemaking—interleaving multimodal perception, interpretation, and decision-making under tight temporal constraints. Motivated by the need for explainable, robust AI that aligns with emerging European regulatory standards for technical robustness and transparency, the authors aim to bridge the gap between low-level computer vision and high-level knowledge representation and reasoning (KRR). The framework is intended for diverse applications, including autonomous driving, digital media, and cognitive robotics. The proposed architecture integrates deep learning-based visual computing with Answer Set Programming (ASP) to model spatio-temporal dynamics. The neural substrate extracts quantitative perceptual features, such as object bounding boxes and motion tracks, from inputs like RGB-D video and eye-tracking data. These features serve as low-level counterparts to high-level semantic characterizations. The symbolic core utilizes ASP to declaratively model foundational aspects of space, time, events, actions, and motion. Specifically, the framework employs a domain-independent theory based on the Functional Event Calculus to handle dynamic properties (fluents) and high-level abducibles (events). It uses the Rectangle Algebra to define spatial relationships and supports non-monotonic visual abduction, allowing the system to form hypotheses about missing data, maintain object identity through occlusions, and explain state transitions caused by actions. The framework computes "space-time mental models" by semantically interpreting sensory perception and maintaining consistent beliefs about the environment. It performs visual abduction to associate observed scene elements with predicted motion tracks and high-level events, ensuring consistency with background knowledge. This process enables the system to interpolate missing information, make default assumptions about object persistence, and generate explanations for observed dynamics. The authors provide an open-source release of the framework, including Python bindings, ROS integration, and Docker support, along with systematic evaluation mechanisms and tutorials. The significance of this work lies in its demonstration of a modular, extensible platform that supports general-purpose computational visual commonsense. By grounding visual inference in declarative logic, the framework offers explainability and elaboration tolerance, addressing key requirements for next-generation cognitive vision systems. The paper showcases the framework’s applicability through case studies in autonomous driving, psychology, and media studies, illustrating its ability to handle complex, multimodal interactions such as joint attention, gesture interpretation, and collaborative tasks. This approach provides a robust foundation for reasoning about embodied interaction in ecologically valid settings, promoting independent extensions and real-world applied KRR.
Key finding
The proposed framework successfully integrates declarative knowledge representation with neural perception to enable robust, explainable visual commonsense reasoning about spatio-temporal dynamics in embodied interactions.
Methodology
theoretical
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-29 (2 acquisition events logged).
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | author_sweep | — | — | 3 | 2026-05-29 |
| archive | success | canonical_url | — | — | 1 | 2026-06-06 |
| extract | success | cached | — | — | 3 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-04 |
| chunk | success | chunk | — | — | 1 | 2026-06-04 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-04 |
| enrich | success | — | — | — | 1 | 2026-05-29 |
| promote | success | — | — | — | 1 | 2026-06-04 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.