Abstract:
Social interaction and communication are supported by the integration of multimodal signals. One crucial social cue when interacting with other humans are facial expressions. In Project 8, we study how people not only read information from faces, but how they read meaning into faces based on context and prior knowledge. Incorporating sources of information in addition to what is actually visible (top-down processing) supports efficient, robust, context-adaptive visual perception. Neural networks designed to recognize facial expressions largely ignore such contextual information and are therefore inherently misaligned with human social perception. Closing this gap promises to make synthetic face processing at the same time more intelligent, useful for human-machine interaction, and ethically responsible.

Successful social interaction relies on additional social factors and cognitive processes including partner co-representation (i.e., the representation of the partner’s actions alongside one’s own actions), emotion processing, theory of mind (i.e., the ability to consider mental states – such as beliefs, desires, intentions – to predict people’s behaviour) and trust. In project 9, we study processes underlying social communication in humans and assess potential changes in these processes when the interaction partner is an artificial agent. We use this knowledge to implement similar mechanisms in our robots and assess how this affects their performance along other dimensions, such as trust or scaffolding. Our ultimate goal is to create robots with higher social intelligence that can interact smoothly with humans and other agents.

Photo by Yuyeung Lau on Unsplash