Avoiding a Pointless-verse

A “metaverse” holds so much potential beyond just social media and marketing platforms. Substantive uses for infrastructure design, construction, and management get little mention. The geomatics, geospatial, and AEC communities need to change the narrative. Perhaps a bit more  “infrastructureverse”?

It’s not to say that current popularized vison of a metaverse is completely pointless. It will be a boon for social media and marketing platforms, and a flood of firms peddling NFTs, digital property rights(?), and all manner of related crypto schemes. And who wouldn’t want to spice up online meeting with “Hey, let’s have our avatars meet up at the virtual Eiffel Tower… woohoo!”.

Cynicism aside (I’ll try) the foundational elements of what the broader public is being introduced to as a metaverse is nothing new to AEC, geomatics, and geospatial professionals, practitioners, and related industries. Namely, how a spatial construct can be enriched with data, to enable immersive experiences and collaboration. After all what are digital twins? And even at a granular level, what is BIM?

Another Boom of Opportunity

Surveying, mapping, engineering, construction, operations, and asset management workflows and collaboration resources have increasingly moved to the cloud. Some in AEC have been doing this virtual thing for decades (albeit in rudimentary form for early adopters).

Large public works projects are commonly digitally twinned, through all phases of design, construction, and operations. – Ezhou Huahu AirportThe global infrastructure boom is unrelenting. It is driven not solely by population growth, but more from global prosperity; expected to add two billion more people to the middle class by 2050. These people want more infrastructure, smarter infrastructure, greener, and more sustainable. I’ve heard this demand could add up to as much as the equivalent of six Europe’s of new or improved infrastructure. Legacy tools and technologies are never going to meet such demand and erase existing infrastructure gaps. There is great hope in the further digitalization of AEC and “constructioneering”, moving to full 3D in all phases of infrastructure lifecycles. And some of the productivity gains and reduction of waste in construction have been realized. An example of how virtuality is benefiting AEC is not just VR (virtual), but in AR (augmented) and XR (mixed/extended reality). In fact, the first no-gaming application developed for the HoloLens2 mixed reality headset was for AEC, and there have been further developments for XR construction layout. By contrast, VR face boxes would be pointless (and dangerous) on a construction site.

Project collaboration in immersive models is very “metaversey”. As are the 3D city models that are used for planning and public engagement. What the metaverse promoters appear to fail to understand is the practicalities (and impracticalities) of creating, updating, and maintaining mega-scale digital twins.

Where Does the 3D Come From?

There is a scene in the 2001 documentary “Startup Dot Com” that speaks to gaps between aspiration and practical reality. The film originally set out to follow a startup firm during the dotcom boom, but instead it saw the firm go from dotcom to dot-bomb, to dot-gone. One of the founders was discussing how they could gather public records nationwide for their online service portal for local and state governments. He commented that “we can send a guy to the basement of every city hall with a camera and get the records”. It was at that point in watching the film, that I realized the startup was doomed. Data capture, classification, and management is far more complex than that. Aspirants in the realm of metaverses are going to run into similar harsh realities. Unless they utilize the services of folks who are already well versed in mass spatial data capture and management.

And, as few on the hype cycle seem to acknowledge, where did the 3D representations of the world they are seeking to mirror come from? Yep, the folks who do reality capture. The metaverse is not going to map and model itself. It is high time for geo folks to make it known how essential they are, and will continue to be, if this mirrored world aspiration has any chance of success.

On the other hand, it was a big tech firm, Google, that essentially (roughly) mapped most of the world and is a default geospatial portal for so many. But when more serious and exacting work needs to be done, there are better resources, tools and technologies that can be tapped into, or commissioned to capture the data. Aerial survey, UAS, mobile mapping systems, backpacks, terrestrial scanners, robot-carried scanners, and terrestrial photogrammetry; while these are evolving rapidly, geo folks have a realistic understanding of these and what they can yield.

But how hard is it to build say, a digital twin at city scale? It depends on what your expectations are. An aerial survey can easily be processed into a 3D point cloud, and then into a city-scale meshed model, good for planning and constituent engagement, but not precise or complete enough for design and construction. And to do any kind of infrastructure management, a lot of work needs to be done to add the intelligence to the model. This does not stop folks from improperly using the term digital twin: I have actually seen some firms fly a site, process a point mesh, and then claim it is a “digital twin”. And what worries me is that when I talk to folks involved in building metaverses, they often seem to think it is as simple as a flyover, or drive-by, and pressing a button.

Mind-boggling Amounts of Data

Take mobile mapping systems as an example. When seeking to make a digital twin of a small city, the driving is the easy part. Then you need to preprocess that captured raw images/lidar, geospatially register it by processing the GNSS/IMU/DMI data, do the “anonymization”  (blur things like people and license plates), then create a 3D point cloud and/or mesh. Then, you may be running AI-driven automated feature recognition, like to identify roads signs. Next, to truly make it a digital twin, you need to attach other data, like attributes for utilities. Anyone harboring a notion of throwing together an amalgam of stray, off the-shelf resources for shake-n-bake models may end up with a fuzzy-verse.

There are some great examples of mega-scale data capture and digital twinning. Singapore has been digital twinned. And while this is a major achievement, keep in mind that it is a relatively small city-state that has been a leader in the application of geospatial technologies.  The German mapping agency BKG has announced it is building a nationwide digital twin, and there are numerous cities around the globe that have begun or completed digital twinning, albeit at different levels of accuracy, currency, and depth of connected attribute data. One of the categories in the annual Year-in-Infrastructure awards, hosted by Bentley Systems, is Digital Cities. Look at some of the projects, they are astounding—moonshot level geomatics in action.

It is not just the public sector that is driving (no pun intended) digital twinning; the private sector seems to be more proactive in this matter. HERE, a global location data and technology provider offers a library of street level lidar covering most of the roads in 50 countries as-a-service, and there are numerous aerial imaging vendors that fly entire countries on a regular basis that also provide city-scale modeling. RISE3D is capturing the entirety of the Netherlands, and will do so on and ongoing basis, with a digital twin portal as-a-service. They are doing this with multiple mobile mapping systems that are part of a new wave of such systems that can yield centimeter precision data at highway speeds.

And while all of these examples are great, the data management aspect may be even more of a technological achievement than the wizardry of the data capture devices. The data volume can really add up, but is not insurmountable for those with the expertise, and a well thought out data management plans. RISE3D, operating 2 full mobile mapping systems 10 hours per day, seven days a week, is logging about 16TB per week. But that this the Netherlands; multiply that by 6 if you are thing about the UK, or 10 times for California. Yet this flood of data has not been an insurmountable issue for RISE3D; they are managing that just fine and using AI-driven automated feature recognition and a slew of other correlated digital data to identify the items their customers want mapped.

A true digital twin is not a one-and-done prospect. “”A digital twin is just a digital representation of something; for Smart Digital Reality you need constant data updates,” said Hexagon AB President and CEO Ola Rollén. Hexagon is a global technology company that has many divisions, but geo folks would be familiar with their Leica Geosystems and Safety, Infrastructure and Geospatial (SIG) divisions. What he put so well was the critical element of currency in the data. You can’t do any kind of planning, engineering, or management of things with obsolete data.

Smart Blocks

Point clouds, even true color point clouds may be the best medium for certain applications but become unmanageable at mega-scale. Meshed models offer some data management and visualization advantages (albeit sometimes with that melted ice cream effect). But there may be a better way to store, update, and manage captured reality data, and the ideas has been around for a long time.

In the 1995 novel “Microserfs”,  author Douglas Coupland introduces us to a fictional startup project called “Oops!” (a nod to object-oriented programming) that his characters describe as a  digital “cross between Legos and an erector set, with the smarts of AutoCAD built in.” It was not a completely novel idea, and developers had been working on such things since the early days of computing. Ironically, the most prominent examples of this idea materialized in the gaming industry in the 1990’s, like in the Quake III engine, and later in Minecraft. In fact several countries like Denmark have been completely “digital twinned” in Minecraft.

The more formal name for such spatial approaches is Voxels. Described by TechTarget as: “A voxel is a unit of graphic information that defines a point in three-dimensional space. In 3-D space, each of the coordinates is defined in terms of its position, color, and density.” They are like 3D pixels that can carry other attributes. Voxels are utilized in medical imaging, and robotics, and in the geospatial sector, the firm Voxelmaps has developed this idea tailored for reality capture and digital twinning.

In short, Voxelmaps solution is a global grid of 1cm-5cm voxels that occupy all space, this even means indoors—it is true volumetric mapping. Point clouds and meshes only give you the surface. And each voxel carries attribute information, including its relationship to nearby and thematically linked voxels. This sounds like it would be very storage intensive, but by managing voxels as arrays, and voxel occupancy grids (VOGs), the data management can be more efficient than other 3D data representation approaches.  Voxels do not necessarily have to be displayed as little cubes; the can be colorized spheres that look like point clouds. At mega-scale, it is likely there would need to be tradeoffs between visualization and storage/speed. Of course there could be additional topical rendering for certain applications.  This approach is already being explored for some global mapping initiatives.

What’s in a Name?

Yes, “metaverse” is a popular buzz term lately, like how “multiverses” is being a bit overworked as movie “fix-any-situation” plot devices. Then, a very large social media company went ahead and rebranded to (in-effect to try to) coopt the term “metaverse”. It seems a little silly on one level. Can you imagine that back during the dotcom era if a company like AOL rebranded as “Cyber”; people would still be doing memes about that.

But the real potential harm that move could end up doing is jading perceptions about what a metaverse is, or could be, or any term that begins with the prefix “meta”. Especially by a not-very-well-liked company, and when some of the early examples of what their metaverse might look like featured legless, dead-eyed avatars that look like something from a 2008 Wii game, that did not help their cause.

Part of the ambivalence toward the metaverse might come from the fact that many folks are already experiencing what they are planning to offer. Gamers have been able to gather avatars in a 3D virtual environment to go on epic quests for decades; this “new thing” seems comical to the young folks I’ve asked about it. But hey, they need to find new ways to do push online advertising and sell a lot of VR “face boxes”.

Ola Rollén also said: “The metaverse could be a great escape, but what is needed to realize sustainability is ‘Smart Digital Reality’.” Yes, could the energy and 3D modelling being put into creating a metaverse also in some way serve society beyond just amusement and marketing? Like achieving sustainable development goals, and reducing the infrastructure gap? The big geo companies, like Hexagon, Esri, Bentley Systems, Topcon, Trimble, and many more are doing an excellent job of promoting an “infrastructureverse” (I could not find a good term for this, so I made one up).

It may be incumbent on not just the geospatial industry, but all of us geomatics and geospatial professionals and practitioners to promote the creation of value in a virtual world. And if you are not so much into the idealistic motivations, remember that building such a thing would create a lot of business opportunity.