Object-level scene descriptions and attention in visual search

Photo by Freddy Marschall on Unsplash

Principal Investigators:

Klaus Obermeyer
Olaf Hellwich
Martin Rolfs
Marianne Maertens

Team Members:

Nicolas Roth (Doctoral researcher)
Olga Shurygina (Doctoral researcher)

Working on related projects:

Lynn Schmittwilken (Doctoral researcher)

Exploring visual scene analysis in humans and assessing its potential for use in computer vision systems

Research Unit 1, SCIoI Project 01

This project focuses on the potential of object-level search processes for understanding visual scene analysis in humans and assesses their potential for real-world scene interpretation and search tasks by computer vision systems. In order to perform this research, we will study an ecologically valid setting where free-viewing mode, real-world dynamic scenes, and visual exploration tasks that have to be performed simultaneously. This will provide novel insights into the question of how human subjects balance attentional processing under – potentially conflicting – task demands. To this end, we will combine eye-tracking experiments in humans with the computational modelling of fixation sequences, with algorithm development, and with the design of computer vision systems using foveated attention and camera movements akin to eye movements.

Schmittwilken, L., & Maertens, M. (2022). Fixational eye movements enable robust edge detection. Journal of Vision. https://doi.org/10.1167/jov.22.8.5
Roth, N., Bideau, P., Hellwich, O., Rolfs, M., & Obermayer, K. (2021). A modular framework for object-based saccadic decisions in dynamic scenes. CVPR EPIC Workshop / arXiv:2106.06073. https://doi.org/10.48550/arXiv.2106.06073
Roth, N., Bideau, P., Hellwich, O., Rolfs, M., & Obermayer, K. (2021). Modeling the influence of objects on saccadic decisions in dynamic real-world scenes. PERCEPTION / 43rd European Conference on Visual Perception (ECVP) 2021. https://journals.sagepub.com/doi/full/10.1177/03010066211059887