Object-level scene descriptions and attention in visual search
Exploring visual scene analysis in humans and assessing its potential for use in computer vision systems
Photo: Unsplash.com/Freddy-Marschall
This project focuses on the potential of object-level search processes for understanding visual scene analysis in humans and assesses their potential for real-world scene interpretation and search tasks by computer vision systems. In order to perform this research, we will study an ecologically valid setting where free-viewing mode, real-world dynamic scenes, and visual exploration tasks that have to be performed simultaneously. This will provide novel insights into the question of how human subjects balance attentional processing under – potentially conflicting – task demands. To this end, we will combine eye-tracking experiments in humans with the computational modelling of fixation sequences, with algorithm development, and with the design of computer vision systems using foveated attention and camera movements akin to eye movements.
Project Results
The team discovered and further described several attentional mechanisms using psychophysical experiments and computational models. Their findings emphasized that vision is not a passive process but requires active decisions about how to move the eyes and where to look next. The team could show that object-based attention, which has been primarily investigated in reduced and specifically tailored visual environments, strongly influences human sampling strategies in the real world. Object-based attention requires a representation of the scene in terms of objects on which attentional mechanisms and selection can act. They propose a mechanism for forming these object representations through an Active InterCONect (AICON) between the object segmentation and the decision-making process of where and when to move the eyes. Further findings of this project include how fixational eye movements enable edge detection, how eye movement characteristics reflect object-based attention, how objects influence the decisions to make eye movements, and how our knowledge about a scene being dynamic or static influences our exploration behavior.
