Flying over Arctic waters, you’d never spot them. Beluga whales vanish against the ice below — their white bodies nearly invisible from above.

Fisheries and Oceans Canada (DFO) has always watched these animals closely. For years, biologists like Bryanna Sherbo and her team at DFO have collected hundreds of thousands of images across the Canadian Arctic as part of their marine mammal monitoring program. Their arsenal included Twin Otter aircraft equipped with Nikon D850 cameras, various drones (DJI Phantom 4, Mavic 3E, and Matrice 300), and even WorldView-3 satellite imagery.

The problem is, they have collected so many aerial photos that nobody can possibly look through them all!

But what if there was a way to spot them in minutes instead of hours?

That’s exactly what DFO and Esri Canada set out to do. They had a straightforward goal: use AI to find whales faster. What seemed impossible turned into something that could prove to be a handy tool for wildlife conservation and protection.

Why Belugas Matter

Belugas are vital to the Arctic ecosystem. Their numbers are dropping as the climate warms, ships crisscross their territory, and industry moves into their homes. With some populations listed as endangered, not keeping track of this iconic Arctic species isn’t an option.

Traditionally, biologists would manually go through these images to count whales. It was a really time-consuming and labor-intensive work.

This is where AI could make a real difference.

“It was clear that AI could help solve this problem, but it needed to be trained to spot the belugas in these complex images that contain ice, waves, sun reflections, and others,” says Mohamed Ahmed, lead data scientist at Esri Canada. “Our goal in this project was to develop a deep learning model that could scan these images and identify beluga whales much, much faster while maintaining accuracy.”

The Challenge: Spotting Whales in a Sea of White

In early 2021, DFO teamed up with Esri Canada, which had already been working on using AI for environmental monitoring. This seemed like the perfect real-world test.

The task wasn’t simple: teach a computer to tell the difference between actual whales and everything else that looks like them. Ice chunks, waves, sunlight on water—all these things fool even experienced researchers. Mistake ice for whales, and you might think there are more animals than really exist, potentially wasting conservation resources.

“We wanted to create a model that could quickly identify whales, and that would allow biologists to focus on interpreting the data and making conservation decisions rather than spending hours on manual counting,” says Ahmed.

The Solution: Teaching AI to See the Whales

Everything started with a mountain of photos. DFO had thousands of aerial shots, and Ahmed’s team needed to teach the AI which white spots were whales and which weren’t. They used deep learning to figure out patterns on their own after seeing tons of examples.

The team spent weeks labeling images—this is a whale, this is ice, this is water. Tedious work, but essential.

“We didn’t just want the AI to spot a whale in a perfect scenario,” Ahmed explains. “The Arctic isn’t always calm and clear. We needed the model to work in all kinds of conditions—ice, bad weather, and murky waters.”

As the team pored through the images, they created a few of what Ahmed calls “vision transformer models,” or state-of-the-art image recognition computer vision tasks. These work by analyzing relationships between different parts of an image.

They trained the AI to look for contrast between the white whales and the darker ocean background. “The high contrast between the beluga whales and the water made them easier to detect. The grey or blue whales tend to blend in with the water, but the belugas have a stark white appearance that stands out,” explains Ahmed.

Once the AI was trained, it detected belugas with 88% accuracy — a huge improvement over traditional manual methods, which involved tired humans squinting at screens for hours, and often making mistakes.

The Hiccups: Ice and Whitecaps

It wasn’t all smooth sailing as the model got confused sometimes. Ice looks a lot like whales from above—both are, after all, white blobs. Whitecaps on waves also tricked the system. Sometimes, sunlight reflecting off water washed out the whole image, making everything look the same.

“Ice is one of the challenges,” Ahmed admits. “It looks very similar to a beluga when you’re looking at it from above, especially when it’s very white and smooth. And then you’ve got the whitecaps that look like whales, too.”

The fix? The team made two separate models: one for mostly clear water, another for icy conditions. They also specifically taught the system what ice and whitecaps look like, so it could learn these aren’t whales.

The Binding Factor: Human Connection

Ahmed credits the success of this project to the collaboration between Esri Canada’s tech team and DFO’s experts. “We couldn’t have done this without the DFO biologists,” says Ahmed. “They know these species and their environments better than anyone. They provided us with the insights we needed to make sure the model worked in the real world.”

The goal wasn’t to replace human effort but to assist it. “The AI is fast and consistent,” Ahmed explains. “It can flag where it thinks a whale is, but it’s the human analysts who have to step in to verify the results. It’s a partnership.”

“AI can help speed things up, but humans will always play a critical role in conservation work,” he says.

DFO can now review images during the same season they are collected, thanks to these models, providing more timely data. With additional training samples, it could become even more precise. The tool integrates with software systems that analysts already use daily, and frees up scientists to focus on actual data analysis and conservation efforts, rather than spending endless hours sifting through imagery to count whales manually.

The Big Picture: AI for Conservation

“As we get better with the AI model and have access to more data, we will be able to scale this effort massively,” says Ahmed. ” In the future, we could be processing millions of images in real time, and that opens up whole new possibilities for conservation.”

The same technology could spot seals, dolphins, or fish. With a few tweaks, it would work for all sorts of marine life.

This matters especially in the Arctic, where everything is changing fast. As ice melts, animals must adapt or move. Tracking these changes helps us protect them better.

Esri’s ArcGIS Living Atlas aims to make these tools available to everyone, not just big organizations with deep pockets.

“The beauty of this project is that it’s not just for the experts. The tools are becoming more accessible,” says Ahmed. With pre-trained models, anyone—from local governments to NGOs—can use them to monitor their own areas.

As the technology gets simpler to use, more people can help protect wildlife. “The more people that can use AI, the more data we can gather, and the more we can protect our ecosystems,” says Ahmed.

It all started with a simple question: could AI find whales faster than people? Now it’s grown into something that could change how we protect ocean life.

For more information, explore this ArcGIS StoryMaps story: Beluga Whale Detection using Deep Learning (AI)

Listen to Esri Canada’s Geographical Thinking Podcast: Episode 98 – Whales or Whitecaps?