On 20 March 2023, the Einstein Center for Digital Future (ECDF) celebrated the start of its second funding phase with an event, where its numerous partners from science, politics, business and society came together. At the event at the Futurium, the ECDF aimed to set new impulses for the digital future and to strengthen its research through collaboration with its partners. During an exhibition that was included in the program, several demonstrators were showcased to the audience. Friedhelm Hamann, doctoral researcher and Guillermo Gallego, principal investigator at Science of Intelligence (SCIoI) and Professor of the ECDF presented a live demo about their work on event cameras.

In their work, the researchers use a combination of video cameras and event-based cameras to extract meaningful motion information on complex animal behavior. Animals exhibit complex behavior that is often studied through video recordings, which provide an economical and non-invasive way to acquire abundant data during experiments. However, to make sense of this raw data, computer vision tools are needed.The new approach proposed combines traditional frame-based cameras and novel event-based cameras to extract meaningful motion information in individuals, either in isolation or as part of groups. By using both types of cameras, researchers can capture high-frequency temporal content and slowly-varying content, which are complementary.

In the demonstrator for the ECDF at Futurium, Friedhelm Hamann presented the co-capture system that he is developing for his PhD thesis on long-term monitoring and animal behavior quantification [1]. He uses it to study fish, mice and birds in collaboration with other researchers at SCIoI and the University of Oxford. Visitors were excited during the demonstration to learn more about event-based cameras, which are novel visual sensors that mimic the transient pathway of the human visual system, and the possibilities that they bring to study natural intelligence.

[1] Hamann and Gallego, “Stereo Co-capture System for Recording and Tracking Fish with Frame-and Event Cameras”. 26th Int. Conf. Pattern Recognition, Workshop on Visual observation and analysis of Vertebrate And Insect Behavior, 2022.