Managing Big Data for Addressing Research Questions in a Collaborative Project on Automated Driving Impact Assessment
DOI: 10.3390/s20236773
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the challenge of managing sensitive big data in collaborative research projects, specifically focusing on the impact assessment of automated driving technologies. The authors identify a gap in existing literature regarding how multiple partners can handle sensitive data to collectively answer research questions (RQs) while protecting intellectual property and ensuring methodological soundness. To investigate this, the authors draw on their experience in L3Pilot, a 34-partner European project assessing the impact of SAE Level 3 and Level 4 automated driving functions across 10 countries. The study aims to develop a robust workflow that allows for the secure sharing and analysis of data from diverse sources, including vehicle manufacturers, research institutions, and suppliers. The methodology employed is based on the Field opErational teSt supporT Action (FESTA) framework. The process begins with a top-down definition of RQs covering technical performance, user acceptance, traffic impact, and societal effects, followed by a bottom-up revision to ensure feasibility. Data requirements were translated into specific performance indicators (PIs) and logging needs. The system distinguishes between objective data from vehicular sensors (e.g., cameras, lidars, CAN bus signals) and subjective data from participant questionnaires. A key component of the implementation is the Consolidated Database (CDB), which aggregates data from all pilot sites. To protect confidentiality, raw time-series data are not shared; instead, summarized PIs are uploaded. The system employs SHA-256 hashing for pseudonymization, ensuring that neither the pilot site nor individual participants can be identified, and that specific vehicle manufacturer behaviors cannot be reverse-engineered. The findings highlight the successful deployment of a coherent workflow that integrates data collection, processing, and analysis. The CDB allows partners to upload data recursively and download it in formats suitable for statistical analysis (JSON or CSV). The system supports four types of vehicular PIs: Trip PIs, Scenario-Specific Trip PIs, Scenario Instance PIs, and Datapoints, which are computed based on detected driving scenarios and experimental conditions. The workflow also includes mechanisms for data providers to post-edit PIs before final upload. The authors note that while the system effectively protects commercially sensitive information and privacy, it merges data to reflect generic impacts of automated driving rather than OEM-specific performance. The significance of this work lies in providing a practical reference methodology for large-scale collaborative projects involving sensitive big data. The authors conclude that a theoretically informed reference methodology is essential for coherently managing project steps, while effective tools are necessary to support the daily work of diverse research teams. The L3Pilot workflow demonstrates how to balance the need for rigorous quantitative impact assessment with the imperative to protect partner intellectual property and participant privacy. This approach offers a replicable model for future industrial research projects in connected and automated driving and other domains requiring secure, multi-party data collaboration.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | DOAJ | — | — | 1 | 2026-06-18 |
| archive | success | openalex | — | — | 4 | 2026-06-25 |
| extract | success | cached | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-18 |
| chunk | success | chunk | — | — | 1 | 2026-06-18 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-18 |
| promote | success | — | — | — | 1 | 2026-06-18 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-18 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Methodological Resource: dataset resource, tool software