Practical, Present-day Autonomy

Ubiquitous vehicular autonomy may still be many years away. However, there is a growing market for defined zone applications. How an autonomous vehicle solution provider has found success with a vehicle-agnostic approach.

“People have real autonomy needs that can be fulfilled right now; that is our focus. A key aspect of our approach is to ensure it is running on vehicles that are already safety certified.” –   Paul Perrone, founder, and CEO of Perrone Robotics.

Amid the flurry of highly-publicized news about vehicle autonomy, with much of that buzz focused on personal transportation, some of the real and more immediate progress goes unheralded. Certainly, the prospect of stepping into your car and asking it to drive you to your destination while you relax, play a game, stream a video, or (*gasp*) take a nap, seems both appealing and frightening. Safety concerns are paramount, and this puts driverless highways on a far horizon. However, autonomous shuttles have been around for many years—you may have already taken a ride on one.

A free-for-all of autonomy on any and all roads and highways is a huge bar to clear. However, defined zones like campuses, industrial complexes, airports, and central business districts are already being plied by driverless vehicles. These are much more controlled and mapped environments, with predictable ridership levels high enough to support business cases. There have been some highly publicized concept implementations for defined zones, often futuristic and highly stylized shuttles, buses, or people pods. However, there are many impracticalities—and safety concerns—about what Perrone calls “bespoke” vehicles. We recently asked Perrone why he chose a different path forward.

AV Pioneers

Perrone Robotics has been in the autonomy space for 20 years, going back to the days of the DARPA Grand Challenges. They were an early autonomous vehicle (AV) research company, then began doing specialized projects.

For instance, they built a low profile robotic “skate”: a platform on which a foam-block facsimile of a standard passenger car was attached. This was deployed in tests of early driver assist features, such as adaptive cruise control, automated braking and collision avoidance features being developed by brand name car manufacturers. The “dummy” vehicle that their robot powered was a moving “target” for the cars being tested, with any collision only dislodging the foam blocks and not damaging the robot. Tests with their flat robot were performed at the Insurance Institute for Highway Safety (IIHS) research facility in Ruckersville Virginia, famed for crash tests.

With an influx of investment capital, Perrone Robotics started to work directly in the automotive industry, with such firms as Jaguar Land Rover. “Originally we wanted to be a mostly software firm, with companies licensing our sensor integration, navigation, operational intelligence and control systems,” said Perrone. They found out that the best way to ensure success of an implementation was to do the hardware integration themselves. Additionally, they wanted to develop systems that could be implemented for immediate needs. They came up with hardware and software bundles that can utilize a wide variety of different brand components, and even a retro-fit kit that can be put on nearly any modern vehicle. “Since 2019, we have started to concentrate mostly on defined zone applications, such as autonomous shuttles and transit vans,” said Perrone. “Our 20-year history has been a wild ride, but I’ve loved every minute of it.”

Perrone explained that a challenge with many “bespoke” vehicles is that they do not meet federal Motor Vehicle Safety Standards (FMVSS), as they might not have steering wheels, crash-worthiness, airbags, etc. Such futuristic vehicles might be quite appealing, and part of an eventual future, but are not yet viable. Running them on public roads in the United States requires waivers from the National Highway Transportation Safety Administration (NHTSA, Perrone pointed out. “Then you only get it for a year—that’s a costly thing. For instance, there is a Box Seven waiver, meaning that it must be imported as a specialty vehicle for you to be able to run it on public roads.” Some U.S. developers have worked a loophole: sending the vehicle to Canada, then bringing it back across the border as an imported specialty vehicle to obtain a waiver. Then there are further stipulations, such as pre-approved routes as well as limited speeds. Perrone took a different approach, fitting systems on vehicles that are already highway-ready and fully safety compliant. “Many of these bespoke vehicles are running at speeds of 12mph or below,” said Perrone. “When we implement our autonomous solutions, we run at posted speed limits.”

Turnkey and Retrofit Solutions

Perrone has worked with existing vehicle manufacturers to integrate their TONY (To Navigate To You) AV systems. Among their turnkey shuttles solutions is the Perrone Robotics TONY MicroTransit LSV (low speed vehicle); their AV system integrated into an electric shuttle built by Waev Inc.; it comes in a 2-, 4-, or 6-passenger configuration. For mid-range shuttles that are FMVSS compliant, meaning they can run on regular streets and highways, there is the TONY Mid-Sized Transit Van, built on a Ford Transit chassis. Then there is the TONY Large Capacity Transit Van built on a GreenPower EV Star vehicle. There are Perrone Robotics implementations using and being fielded with these vehicles in many places already: Florida, Hawaii, Texas, Virginia, Pennsylvania, Wisconsin, Ohio, Maryland and more.

The TONY Retrofit solution brings all the same capabilities, for doing just about any task a driver would do, into a kit. Perrone Robotics can fit this onto just about any vehicle, even with drive-by-wire options, in a few days, with a few weeks of certification. They’ve remained committed to being vehicle- and component-agnostic, working with many manufacturers. This has also provided flexibility for adoption of continual technological improvements.

Key Components

Key to their AV system, as with any, is a GNSS receiver. A dual antenna system provides position and heading. They have done a lot of work with Trimble’s Applanix GNSS/IMU components but have worked with several other GNSS manufacturers as well, such as Novatel and Oxford Technology Solutions (OxTS). For RTK corrections, they can employ any public or commercial corrections service. For example, for some implementations they have used the Verizon Hyper Precise service, which is available in more than 100 markets in the United States, as full RTK or lower precision DGNSS depending on location. However, AV systems do not rely purely on GNSS, whether RTK or other, for position, especially not relative positioning. “When we do lane keeping, this does not rely on just the mapped roadway and GNSS position alone,” said Perrone. “Because we have LiDAR and cameras, we have excellent lane keeping that tells us where the sides of the road are, and it could be grass, gravel, or some other kind of transition. In our tests we show that our lane keeping is better than that on a [big name car brand].”

Systems can include lidar, radar, ultrasonic, and cameras. “We’ve worked with many lidar manufacturers over the years, such as SICK, Velodyne, and others,” said Perrone. “We’ve used 360° scanner lidars with typically 32 planes or 16 planes, then some of the more solid state lidars, 1000 by 400-point grids.” The technology keeps evolving, and they are continually trying new components. He said that radar provides, at a low price point, detection of objects at long ranges, though not with the precision of lidar. He notes, though, that some newer lidars are reaching 200m and 300m useable ranges. The radar lets the system know of something out there and act as a redundant detection source, whereas the lidar provides more precision and cameras provide color, all of which can help determine what the object is: a car, cyclist, pedestrian, dog, or just a shrub on the side of the road or a paper bag. A lot of work has gone into to object recognition, and they’ve been doing work with thermal imaging cameras to assist. They’ve also been employing multiple lidars, radars, and cameras on vehicles, for redundancy and overlap.

A growing trend is using SLAM lidar for position stabilization, augmenting the GNSS and IMU of mobile mapping systems. While Perrone believes that SLAM can be useful, there are also situations where it could be hazardous for autonomous vehicles. Developed initially for indoor applications, SLAM can work very well in such controlled environments. However, outdoors there may be stretches of road without features to use to localize, and changing environments. “We had a customer in Europe that had used SLAM, which worked well enough in some outdoor environments, such as mines, but found out that their performance along some roadways changed from leaf-on periods of summer to leaf-off periods of winter,” said Perrone.

SLAM is not alone to challenges in varied and dynamic outdoor environments. Their multi-sensor systems, tapping the strengths of each type of sensor, enable degradation strategies. When GNSS is not available, such as in urban canyons or parking garages, Perrone said that they can employ what he calls super-simple-SLAM, or even Ultra-Wideband (UWB) for fixed locations. Like that which they have employed for an implementation for the Jacksonville (Florida) Transportation Authority. If all else fails (and the vehicle is safely brought to a stop) a customer can, if the option is purchased, tele-operate remotely as they’ve partnered with teleoperation product providers such as Guident.

Cameras can detect color, such as light traffic signals. However, they are also tapping into smart infrastructure, such as Dedicated Short-Range Communications (DSRC) systems that transmit traffic signal status. This has been deployed for such customers as Jacksonville. Working with Commsignia, a provider of V2X (vehicle-to-everything) solutions, their TONY systems will also be ready for a nascent federal work zone V2X notification system.

“We think of the modular TONY kit as building blocks for AVs,” said Perrone. “That is our approach in a nutshell; we can upgrade road-certified vehicles simply and affordably.” We are already seeing defined zone applications such as shuttles, but also autonomy in construction, mining, warehouses, port operations, agriculture, and more. We might just have to wait a bit (or quite a bit) longer for driverless highways.