Research

Knowledge-augmented face perception

Bridging the gap between human and synthetic face processing

Research Unit: 1

Project Number: 8

Example Behavior:
Social Intelligence

Disciplines:
Computer Vision
Psychology

 

Principal Investigators:
Olaf Hellwich
Rasha Abdel Rahman

Doctoral Researchers:
Florian Blume

Postdoctoral Researchers:
Julia Baum
Pia Bideau
Martin Maier

 

Expected Project Duration
2019 - 2024


← Projects Overview

Knowledge-augmented face perception

Bridging the gap between human and synthetic face processing

Generated via Jasper Illustrator with prompt "smiling face, vector art"

Face perception and categorization is fundamental to social interactions. In humans, input from facial features is integrated with top-down influences from other cognitive domains, such as expectations, memories and contextual knowledge. For instance, whether a face is perceived as depicting an angry expression may depend on prior knowledge about the context (Aviezer et al., 2007) or person (Abdel Rahman, 2011; Suess, Rabovsky & Abdel Rahman 2014). Furthermore, humans have a strong tendency to infer traits such as trustworthiness directly from faces. In contrast to human perception, automatic systems of face processing are typically based purely on bottom-up information without considering factors as prior knowledge. Even in modern deep learning approaches where system performance depends on massive amounts of training data a combination of visual input data and given knowledge is regularly not considered. This principle difference to human face perception limits the scope of understanding and successful interactions between artificial agents and humans. The aim of the project is therefore to bridge the gap between human and synthetic face processing by integrating top-down components typical for human perception into synthetic systems. This will be done by linking empirical observations with computational modelling and state-of-the-art image analysis methods. An intermediate result of the investigations may be an improved understanding of meaning and impact of prior knowledge w.r.t. visual feature representations in the process of computing an interpretation of a facial expression. In perspective, the insights on human-like face perception may be integrated in humanoid robots to adapt to social perception and face-to-face communication.


Maier, M., Blume, F., Bideau, P., Hellwich, O., & Abdel Rahman, R. (2022). Knowledge-Augmented Face Perception: Prospects for the Bayesian Brain-Framework to Align AI and Human Vision. Consciousness and Cognition, 101. https://doi.org/10.1016/j.concog.2022.103301
Maier, M., Frömer, R., Rost, J., Sommer, W., & Abdel Rahman, R. (2022). Linguistic and semantic influences on early vision: evidence from object perception and mental imagery. Cognitive Neuroscience of Language Embodiment and Relativity.
Maier, M., Leonhardt, Alexander, & Abdel Rahman, R. (2022). Bad robots? Humans rapidly attribute mental states during the perception of robot faces. KogWis 2022.
Maier, M., & Abdel Rahman, R. (2024). Transient and Long‐Term Linguistic Influences on Visual Perception: Shifting Brain Dynamics With Memory Consolidation. Language Learning, 74(S1), 157–184. https://doi.org/10.1111/lang.12631
Leonhardt, A., Maier, M., & Abdel Rahman, R. (2021). The impact of affective knowledge on the perception and evaluation of robot faces. 5th Virtual Social Interactions (VSI) Conference. https://www.so-bots.com/s/VSI_5_VIRTUAL_2021_UPDATE_29June.pdf
Halawa, M., Blume, F., Bideau, P., Maier, M., Abdel Rahman, R., & Hellwich, O. (2024). Multi-Task Multi-Modal Self-Supervised Learning for Facial Expression Recognition. CVPR Workshop. https://doi.org/10.48550/arXiv.2404.10904
Halawa, M., Blume, F., Bideau, P., Maier, M., Abdel Rahman, R., & Hellwich, O. (2024). Multi-Task Multi-Modal Self-Supervised Learning for Facial Expression Recognition. IEEE Computer Vision and Pattern Recognition Conference Workshops (CVPRW) 2024. https://doi.org/10.48550/arXiv.2404.10904
Enge, A., Süß, F., & Rahman, R. A. (2023). Instant Effects of Semantic Information on Visual Perception. Journal of Neuroscience. https://doi.org/10.1523/jneurosci.2038-22.2023
Eiserbeck, A., Maier, M., Baum, J., & Abdel Rahman, R. (2023). Deepfake smiles matter less—the psychological and neural impact of presumed AI-generated faces. Scientific Reports. https://doi.org/10.1038/s41598-023-42802-x

Research

An overview of our scientific work

See our Research Projects