Proximity awareness for mining and construction site safety is getting a boost from powerful edge computing, computer vision, neural processing, open-source and proprietary image recognition databases.

Anyone who has experienced or witnessed close calls at busy mining or construction sites has probably wished they had an extra set of eyes. Short of adding human spotters to heavy equipment or constantly looking over one’s shoulder, there have not been many good options to turn chaos into clarity until recent years. We don’t have to go into the reasons why safety must always come first; a safe site touches all bottom lines.

For several decades, there has been a steady evolution in technological approaches to site safety. There have been radar, sonic, beacon-based, and more, for proximity awareness and action systems. Many machine control systems come with coordinate-based zone limit features (e.g., to avoid hitting underground infrastructure). And, more recently, real-time camera systems have made inroads. Adopting even some of the more rudimentary legacy systems has helped improve site safety. Any effort can be worthwhile; you can’t put a price on a life lost. This subject hits home for me. I remember as a young surveyor, scrambling around a busy mine pit with a rod, trying to take shots, all the while dodging gargantuan loaders.

While various hazard detection and action systems are now standard on many new cars, including collision avoidance, emergency steering, automated braking, and more, the environments in which construction equipment works are much more complex. R&D for the nascent consumer vehicle autonomy market has laterally benefited the construction sector, though only to a limited degree.

Very recent developments have taken multiple, rapidly maturing technologies, a big step forward—a much-needed and anticipated step. A new one just launched in North America this week, that could outfit nearly any piece of construction equipment: dozers, loaders, excavators—any type of any size, regardless of brand.

Smart Eyes

Last June and the HxGN LIVE 2025 (Hexagon’s global conference and exhibition), I got a glimpse of a new solution called “Xsight360”: computer vision (CV) based, real-time feature/hazard recognition system for mounting on heavy equipment. It was still undergoing testing in the UK at the time. What I found intriguing was that on the back end, it employs powerful edge computing and AI detection models trained on a vast proprietary database of over 65 million annotated construction site images, a dataset that continuously improves through real-world field data. It uses live feeds from up to six cameras mounted on any vehicle, working online or offline. And, despite the deep tech stack, the operator interface was well thought out; simplified to assist the operator without burdening them with additional tasks.

Ok, I’ll have to admit, the name “Xsight” seemed a bit “markety” to me at first, but it is a serious, and devilishly clever bit of kit.

When I heard that it was going prime time (Feb 10, 2026), I asked Troy Dahlin, Vice President of Heavy Construction and Machine Control at Hexagon, for more details. “Part of how this came about was our [Hexagon’s] acquisition of Xwatch, which develops 3D zone avoidance applications for machine control,” said Dahlin. “Their core competence was how to have hydraulic intervention on a machine, stopping it from encroaching where it should not, or exceeding safe movements. For example, raising something too far, that’s too heavy, or travelling too far.”

Building on this type of success, Hexagon set about designing a new type of proximity awareness and action system, leveraging some of Hexagon’s newer core competencies: geospatial neural processing, edge computing, and computer vision-based applications.

Many Eyes, a Big Brain, and a Deep Memory

It is a given that nearly any new piece of equipment or supporting tech system will leverage neural processing (AI) to some degree, and Xsight360 does this to recognize features in a fraction of a second. There are a number of camera-based systems out there, but as Dahlin noted, Hexagon chose a big hammer for their on-board edge computing: NVIDIA Orin graphics processors. Not only does this make short work of what Xsight does, but it could also lead to future capabilities (more musings on that later).

Up to six cameras can be attached and calibrated for just about any piece of moving construction equipment out there. Dahlin notes that the machine control system does not have to be in the Hexagon ecosystem, that Xsight can operate as a standalone companion system to any equipment, no matter what brand.

Each camera has a 120° field of view. Then why six cameras? “Machines are different sizes and have different view profiles,” said Brad Mullis, Product Manager of Safety Awareness Solutions at Leica Geosystems (part of Hexagon). “You can only mount them in certain places on certain machines, right? For instance, there are huge loaders, with many blind spots, whether you’re looking forward, aft, left, right, up, or even down. So, supporting up to six will cover almost every piece of construction equipment we’ve ever come across.”

The eyes can look everywhere, all at once, and there’s the big brain to process this flood of video in real-time. As long as the object appears in at least one camera, it can do an analysis. But how does it know what to look for? This is handled via the other key component: a massive and continuously growing AI training pipeline. “We use detection architectures like YOLO [You Only Look Once] as a foundation,” said Mullis. “But the real differentiator is our proprietary training dataset, over 65 million unique annotated images from real construction and heavy industry environments. That’s what gives us the edge in recognizing the things that actually matter on a job site.”

“The system is trained on many thousands of variations of people in different postures, clothing, lighting conditions, partial occlusions,” said Mullis. “But what if it encounters something it hasn’t seen before, like a particular type of hazmat suit? The AI will still flag it as a probable person, and that detection event gets captured. That data feeds back into our training pipeline, where it’s validated and incorporated into the next model update. Improved models are then pushed out to every system in the field via over-the-air updates. We always benchmark any new model before deploying it. We never push an update that performs worse than what’s already running. So, the day you get the system is the worst it will ever be.”

“The reason we built it this way is that we want the system to keep getting better without requiring hardware changes,” said Mullis. “As open-source AI architectures like YOLO advance, we benefit from those improvements using proprietary training data, a continuous improvement pipeline, and the way it has been tuned for construction and heavy industry environments. Every improvement in the underlying AI ecosystem brings improvement for our customers, and we layer our own domain expertise on top of that.”

Operations

A construction firm does not have to have the system query the entire database every time. “A mining company in the U.S., for instance, would want to preload the MSHA (Mining Safety and Health Administration) requirements into the system,” said Dahlin. “A construction company might upload the OSHA (Occupational Safety and Health Administration) standards; it uploads these in an instant. And some large construction firms have their own standards, preloaded with objects that might frequently be on the types of sites they work at.” By preloading portions of the database, the system can even work offline.

“The construction images section of the database is huge,” said Dahlin. “But it has to be. There can’t just be one image of a traffic cone; there need to be thousands, different types, in various conditions, and visual environments. How does an object look in different conditions, or in dusty conditions, or through chain link fences? The system is self-learning. As it encounters objects in different conditions, it remembers. And if there is a very specific variation, or a new type of object not yet encountered before, a human can examine the video, verify it, and submit it to the database.”

Mullis likes to pose the “has anyone seen my red wheelbarrow?” question. Beyond real-time safety detection, the system captures footage that can be analysed using proprietary techniques, a form of AI post-processing that lets you query what’s been observed on site in natural language. “So yes, you could ask the system to find that red wheelbarrow,” said Mullis. “But more practically, think about identifying whether people in certain areas are wearing the right PPE (personal protective equipment), understanding machine activity patterns, or reviewing what happened in a specific zone during a shift. The safety detection runs in real time on the edge, but there’s a whole additional layer of insight you can extract from the captured data.”

A fun example that demonstrates just how specific the system can get is the “blue hard hat” scenario. “Let’s say that a construction site wants to reduce confusion by limiting the different colors of hard hats” said Dahlin. “Maybe they do not allow blue hard hats. The system could easily recognize a blue hard hat, and a supervisor could go out and make the switch. That might be a silly example, but it’s not being ‘big brother’. Rather, it is an example of automating crucial, safety-related observations that people would otherwise have to make—in real time.”

The video and identified features can be run through AI to create training videos for specific sites. These can be used to orient new crew members in anticipating the types of hazards they might encounter.

The User Interface

The heavy equipment operator needs to be rapidly informed, not distracted. The UI is deliberately small and simple. As Mullis notes, they only need to see it from the corner of their eye while getting the job done. A small circular display, that can be arranged in the cab in any manner the operator wishes, shows the direction of the detected objects, and icons indicate, for instance, if it is a person, another piece of equipment, pallets, pipes, etc.

The system can be configured to trigger further actions. “It only needs to look for a fraction of a second to make a decision, under 200 milliseconds,” said Mullis. “It displays the direction and type of objects, and can be set up to trigger other actions, like activating external alerts, for example a loudspeaker on the cab that broadcasts a warning to folks to get out of the way, or a siren. For situations requiring actual machine intervention, slowing down or stopping the equipment, that’s where Hexagon’s Xwatch safety system comes in as a complementary solution.

Looking Ahead

How else could the power of this type of tech stack be leveraged? “With the NVIDIA Orin providing significant processing headroom, the platform isn’t limited to safety detection alone,” said Mullis. “The system continuously captures rich visual data from multiple cameras across every machine on site, and that opens up a world of operational insight. Think activity tracking, idle time analysis, understanding what each machine is doing and what’s happening around it. By taking selected frames and processing them through additional AI models, either on the edge or via the cloud, you start building a queryable picture of site operations that goes well beyond safety.”

In broader terms, CV is powering many geospatial applications, beyond safety. For example, there’s Blyncsy (acquired by Bentley Systems). It can Leverage cameras on vehicles, even fleet vehicles travelling roads (during the course of other operations). The captured images can be processed for road corridor asset inventory and road management applications. There are other applications where incidentally captured images can be leveraged photogrammetrically to calculate, for instance, stockpile volumes and more. Cameras and powerful edge computing combinations open doors to potentially updating digital twins of sites on a continuous basis.

“The potential applications are broad, from combining camera data with other sensor inputs to leveraging the multi-camera coverage for site awareness applications we haven’t even imagined yet,” said Mullis. Dahlin and Mullis didn’t promise anything specific, but they were clear that the Xsight360 platform is designed with expandability in mind. “It’s not just a safety system, it’s an AI platform for construction equipment,” said Dahlin. “And safety is its first and most critical application.”

These types of systems, and other developments, will continue to improve site safety and efficiency. If solutions like this had been in place back in the day, working in those open-pit mines might not have been as terrifying. This is the kind of progress that thoughtful and clever tech brings.

Beep on!