Title: Multi-view Tracking in an Outdoor Aviary Abstract: The ability to capture detailed interactions among individuals in a social group is foundational to our study of animal behavior and neuroscience. Recent advances in deep learning and computer vision are driving rapid progress in methods that can record the actions and interactions of multiple individuals simultaneously. Many social species, such as birds, however, live deeply embedded in a three-dimensional world. This world introduces additional perceptual challenges such as occlusions, orientation-dependent appearance, large variation in apparent size, and poor sensor coverage for 3D reconstruction, that are not encountered by applications studying animals that move and interact only on 2D planes. In this talk, I will present a system for studying the behavioral dynamics of a group of songbirds as they move throughout a 3D aviary. We study the complexities that arise when tracking a group of closely interacting animals in three dimensions and introduce a novel dataset for evaluating multi-view trackers. We analyze captured ethogram data and demonstrate that social context affects the distribution of sequential interactions between birds in the aviary. Finally, we present preliminary results of tracking with event-based cameras.

Title: Imageomics: Images as the Source of Information about Life Abstract: Introducing the new field of imageomics: from images to biological traits using biology-structured machine learning. Images are the most abundant, readily available source for documenting life on the planet. Coming from natural history collections, laboratory scans, field studies, camera traps, wildlife surveys, autonomous vehicles on the land, water, and in the air, as well as tourists’ cameras, citizen scientists’ platforms, and posts on social media, there are millions of images of living organisms. But their power is yet to be harnessed for science and conservation. Even the traits of organisms cannot be readily extracted from images. The analysis of traits, the integrated products of genes and environment, is critical for biologists to predict effects of environmental change or genetic manipulation and to understand the significance of patterns in the four billion year evolutionary history of life. Knowledge-guided machine learning and computer vision can turn massive collections of images into high resolution information database about living organisms, enabling scientific discovery, conservation, and policy decisions. This is our vision of the new scientific field of imageomics.

Oral 1 by Mohannad Elhamod Discovering Novel Biological Traits From Images Using Phylogeny-Guided Neural Networks

Oral 2 by Valentin Gabeff Scene and animal attributes retrieval from camera trap data with domain-adapted language-vision models

Oral 3 by Otto Brookes PanAf20K: A Large Video Dataset for Wild Ape Detection & Behaviour Analysis

see poster board assignment here: https://www.cv4animals.com/

Title: Developing imaging technologies to search for, discover, and understand life in the deep sea Abstract: The deep sea is the largest habitable ecosystem on the planet and remains one of the least explored. Very little is known about deep sea inhabitants, their behavior, and the limits and drivers for their survival. Ecomechanics, the multidisciplinary research field of the mechanisms that underlie organismal interactions and survival within their environment, has proven largely successful in terrestrial fields and lab-based organismal systems, but has had limited applicability to deep sea animals. The reason for this deficit is largely due to the technological challenges to access this environment for study, and the typically short duration of any observations, precluding documentation of many critical behaviors. Here we present a number of developments from MBARI’s Bioinspiration Lab that can be used to quantify biodiversity, biomechanics, and behavior of animals in the deep sea. All powered by computer vision, these developments include: two 4000 m-rated, ROV-based imaging systems DeepPIV and EyeRIS used to quantify time-resolved particle fields and structures in 2D and 3D, a 1500 m-rated AUV-based plenoptic imaging system Chiton, an underwater labeled image database called FathomNet that can be used to fuel machine learning algorithm development, and machine learning-integrated vehicle control algorithms called ML-Tracking that enable long-duration observations of individual animals. These instruments and approaches can be applied to a wide range of science use cases, both in the water column and on the seafloor.

Title: Remote sensing enables monitoring life under water Abstract: The oceans are a major source of biodiversity and food. The preservation of the quality of the foodchain, as well as of life underwater in general is a primary concern for society. However, observing water media routinely is a complex task that remote sensing technology empowered by AI can unlock. In this talk, I will present some recent researches, where a deep learning system, based on consumer grade cameras, is deployed to 3D reconstruct and map coral ecosystems at scale.

Title: AI and Animal Wellbeing Abstract: Rapid progress in computer based automation approaches creates new opportunities and challenges for animal wellbeing. On the one hand, innovative applications are being implemented to assist pet owners, people who work with animals, veterinarians, and for tasks that require monitoring of animals in different settings. On the other hand, we see challenges similar to those in automatic human behavior analysis, where automation may cause us to lose the human touch, and ethical issues and risks ensue from the potential usage of such applications, both in the short and in the long run. In this talk, I would like to draw parallels between computer based human and non-human animal behavior analysis, providing examples from our interdisciplinary work in equine and canine pain estimation, as well as other work at the newly established AI and Animal Wellbeing Laboratory at Utrecht University, to raise a number of issues and to discuss both technical and humanity challenges of the field.

Oral 4 by Luke Meyers Towards Automatic Honey Bee Flower-Patch Assays with Paint Marking Re-Identification

Oral 5 by Medha Sawhney Detecting and Tracking Hard-to-Detect Bacteria in Dense Porous Backgrounds

see poster board assignment here: https://www.cv4animals.com/