Projects

Current Projects

Pacific salmon ecology

This project is centered on Pacific salmon ecosystems in the Bristol Bay watershed of Alaska and is done in close collaboration with Dr. Andrew Berdahl at University of Washington and Dr. Albert Kao at UMass Boston. We use a combination of imaging drones and time-lapse cameras to record salmon during the final stages of their migration as well as when they are nesting and confronting hunting brown bears. We design and use custom deep-learning tools to automatically process the imagery to create deatiled datasets of individual behavior embedded in complex natural social and environmental contexts. Broad topics of the research include 1) salmon sensing and navigation from the individual to the collective level at stream confluences during migration; 2) hunting and collective evasion dynamics during brown bear hunts of nesting salmon; 3) ecosystem scale description of salmon and bear landscape use over the nesting season from individuals to populations; 4) building ML powered pipelines for efficient image processing.

Wyoming rangeland dynamics

With support from a University of Wyoming and Cardiff University seed grant, we are starting a project based at the 9H Ranch located just outside of Laramie, WY to investigate the potential of combing high resolution satellite imagery, fixed wing drone imagery, and novel computer vision algorithms to better understand the seasonal dynamics of plant growth and the wild and domestic animals that depend on them.

Previous Work

A general solution for using drones to study animal behavior and environments

See coverage of this work on Hello Future and Phys.org

From our paper’s abstract:

1. Methods for collecting animal behaviour data in natural environments, such as direct observation and biologging, are typically limited in spatiotemporal resolution, the number of animals that can be observed and information about animals’ social and physical environments.
2. Video imagery can capture rich information about animals and their environments, but image-based approaches are often impractical due to the challenges of processing large and complex multi-image datasets and transforming resulting data, such as animals’ locations, into geographical coordinates.
3. We demonstrate a new system for studying behaviour in the wild that uses drone-recorded videos and computer vision approaches to automatically track the location and body posture of free-roaming animals in georeferenced coordinates with high spatiotemporal resolution embedded in contemporaneous 3D landscape models of the surrounding area.
4. We provide two worked examples in which we apply this approach to videos of gelada monkeys and multiple species of group-living African ungulates. We demonstrate how to track multiple animals simultaneously, classify individuals by species and age–sex class, estimate individuals’ body postures (poses) and extract environmental features, including topography of the landscape and animal trails.
5. By quantifying animal movement and posture while reconstructing a detailed 3D model of the landscape, our approach opens the door to studying the sensory ecology and decision-making of animals within their natural physical and social environments.

Counting Africa’s largest bat colony

See coverage of this work in ‘The Conversation’

From our paper’s abstract:

Estimating animal populations is essential for conservation. Censusing large congregations is especially important since these are priorities for protection, but efficiently counting hundreds of thousands of moving animals remains a challenge. We developed a deep learning-based system using consumer cameras that not only counts but also records behavioral information for large numbers of flying animals in a range of lighting conditions including near darkness. We built a robust training set without human labeling by leveraging data augmentation and background subtraction. We demonstrate this approach by estimating the size of a straw-colored fruit bat (Eidolon helvum) colony in Kasanka National Park, Zambia with cameras encircling the colony to record evening emergence. Detection of bats was robust to deteriorating lighting conditions and changing backgrounds. Combined over five days, our population estimates ranged between 750,000 and 976,000 bats with a mean of 857,233. In addition to counts, we extracted wingbeat frequency, flight altitude, and local group polarity for 639,414 individuals. This open access method is an inexpensive but powerful approach that, in addition to radial emergences from central locations, can also be applied to unidirectional movements of flying groups, such as migratory streams of birds.

Computer vision to explore honey bee nest building

See coverage of this work on Earth.com and Phys.org

From our paper’s abstract:

Form follows function throughout the development of an organism. This principle should apply beyond the organism to the nests they build, but empirical studies are lacking. Honeybees provide a uniquely suited system to study nest form and function throughout development because we can image the three-dimensional structure repeatedly and non-destructively. Here, we tracked nest-wide comb growth in six colonies over 45 days (control colonies) and found that colonies have a stereotypical process of development that maintains a spheroid nest shape. To experimentally test if nest structure is important for colony function, we shuffled the nests of an additional six colonies, weekly rearranging the comb positions and orientations (shuffled colonies). Surprisingly, we found no differences between control and shuffled colonies in multiple colony performance metrics—worker population, comb area, hive weight and nest temperature. However, using predictive modelling to examine how workers allocate comb to expand their nests, we show that shuffled colonies compensate for these disruptions by accounting for the three-dimensional structure to reconnect their nest. This suggests that nest architecture is more flexible than previously thought, and that superorganisms have mechanisms to compensate for drastic architectural perturbations and maintain colony function.