Detection and classification of sensor anomalies for simulating urban traffic scenarios

Bachechi, Chiara; Rollo, Federica; Po, Laura · 2021 · Crossref

DOI: 10.1007/s10586-021-03445-7

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the critical issue of data quality in smart city traffic management, specifically focusing on the detection and classification of anomalies in sensor data streams. Urban traffic simulations rely on real-time inputs from sensor networks, such as induction loop detectors, to create digital twins of mobility patterns. However, sensor malfunctions and erroneous data can propagate into these simulations, leading to inaccurate traffic analysis and monitoring. The authors aim to improve the performance of traffic simulation models by implementing a real-time data cleaning process that distinguishes between sensor faults and unusual but valid traffic conditions, removing only the former from the simulation inputs. The methodology employs a multi-step pipeline applied to real-world traffic data collected from 318 reliable sensors in Modena, Italy, over a 30-day period. The process begins with a flow-speed correlation filter that identifies and flags observations exceeding physical capacity limits based on average speed and vehicle length. These "filtered" observations are repaired using weighted averages from neighboring time intervals to prevent data gaps. Subsequently, the authors apply Seasonal-Trend decomposition using Loess (STL) and its robust variant, RobustSTL, to decompose time series into trend, seasonal, and residual components. Anomalies are detected within the residuals. A key innovation is the classification phase, which categorizes detected anomalies into "sensor faults" or "unusual traffic conditions" by analyzing correlations among sensors. Only observations classified as sensor faults are excluded from the input of the traffic simulation model. Experiments demonstrate that this combined detection and classification approach significantly enhances the accuracy of traffic simulations compared to methods that either include all data or remove all anomalies indiscriminately. The study found that distinguishing between faults and genuine traffic anomalies prevents the loss of valuable data regarding unusual traffic patterns while eliminating noise caused by malfunctioning sensors. The statistical analysis of the Modena dataset revealed that traffic time series are stationary after accounting for daily and weekly trends, validating the use of STL decomposition. The results indicate that the proposed method effectively reduces the error rates in traffic flow emulation, providing local authorities with more reliable data for urban mobility management. The significance of this work lies in its contribution to intelligent transportation systems and IoT data processing. By providing a framework for real-time, unsupervised anomaly detection and classification, the paper offers a practical solution for maintaining the integrity of large-scale sensor networks without requiring labeled historical data. This approach ensures that traffic simulation tools remain robust against sensor failures while preserving the ability to detect and analyze significant traffic events, thereby improving the overall reliability of smart city infrastructure and decision-making processes.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• naturalistic crash near crash