ObjectVisA-120: Object-based Visual Attention Prediction in Interactive Street-crossing Environments
Igor Vozniak,
Philipp Müller,
Nils Lipp,
Janis Sprenger,
Konstantin Poddubnyy,
Davit Hovhannisyan,
Christian Müller,
Andreas Bulling,
Philipp Slusallek
Proc. IEEE Intelligent Vehicles Symposium (IV),
2026.
Abstract
Links
BibTeX
Project
The object-based nature of human visual attention is well-known in cognitive science, but has only played a minor role in computational visual attention models so far. This is mainly due to a lack of suitable datasets and evaluation metrics for object-based attention. To address these limitations, we present ObjectVisA-120 - a novel 120-participant dataset of spatial street-crossing navigation in virtual reality specifically geared to object-based attention evaluations. The uniqueness of the presented dataset lies in the ethical and safety affiliated challenges that make collecting comparable data in real-world environments highly difficult. ObjectVisA-120 not only features accurate gaze data and a complete state-space representation of objects in the virtual environment, but it also offers variable scenario complexities and rich annotations, including panoptic segmentation, depth information, and vehicle keypoints. We further propose object-based similarity (oSIM) as a novel metric to evaluate the performance of object-based visual attention models, a previously unexplored performance characteristic. Our evaluations show that explicitly optimising for object-based attention not only improves oSIM performance but also leads to an improved model performance on common metrics. In addition, we present SUMGraph, a Mamba U-Net-based model, which explicitly encodes critical scene objects (vehicles) in a graph representation, leading to further performance improvements over several state-of-the-art visual attention prediction methods. The dataset, code and models will be publicly released.
@inproceedings{vozniak26_iv,
title = {{{ObjectVisA-120}}: {{Object-based Visual Attention Prediction in Interactive Street-crossing Environments}}},
booktitle = {Proc. IEEE Intelligent Vehicles Symposium (IV)},
author = {Vozniak, Igor and Müller, Philipp and Lipp, Nils and Sprenger, Janis and Poddubnyy, Konstantin and Hovhannisyan, Davit and Müller, Christian and Bulling, Andreas and Slusallek, Philipp},
year = {2026},
doi = {}
}