When (Virtual) Worlds Collide

We are increasingly exposed to visually rich 3D environments, in games, entertainment, and in architecture, engineering and construction applications. Navigable geospatial environments like zoomable globes are broadly integrated into our personal and work lives. Imagine the power of these two worlds combined. A Canadian geospatial developer is on track to realizing this vision.

“Zoomable globes” have become ubiquitous. While there are present day tools for building data-rich 3D environments like digital twins and BIM models, these have inherent limitations when it comes to becoming fully immersive, visually and data-rich 3D worlds—especially not on a global scale.

I’m quite relieved that the buzz term “metaverse” from a few years ago is not being bandied about as much anymore. It seemed none of the high-profile initiatives even tried to build a “useful-verse”. So, I was heartened to find out about the work of Sean Treleaven, the founder of Bending Time, who is bringing disparate virtual worlds together… and with an appropriate focus on open-source data.

Let me open this discussion by noting that no one thinks that there is anything terribly wrong with the many amazing zoomable globes already out there. I’m not alone in being a huge fan and frequent user; they are wonderful resources. However, bending Time would like to create one with some additional functionality.

These two worlds already exist, and there are multiple wonderful examples of each, with development proceeding continuously. However, the kind of mash-up Bending Time has in mind does not otherwise seem on a near horizon; it makes me wonder why it has not happened already. That question was the impetus for reaching out.

“3D geospatial visualizations and 3D virtual worlds are on a collision course however, the industry behemoths are moving at a glacial pace,” said Treleaven. “I think the main reason is people are resistant to change. We are creatures of habit. It’s hard to create something new, not just from a technical standpoint but, maybe more so, from a business standpoint as Clayton Christensen laid out for all of us in “The Innovator’s Dilemma.”

The Concept

What would this virtual global mash-up look and feel like? Imagine the familiar feel of a navigable globe in an app or web portal where you zoom to any place of interest. But instead of static 2D data, rudimentary 3D terrain, or limited imports of more detailed 3D models, it feels like you are immersed in a visually rich global game world. Treleaven’s vision goes beyond just visualization.

“There’s a fundamental downside to today’s 3D globes and that’s the general fact that nothing moves inside these visualizations,” said Treleaven. “They tend to be static views. Further, the area surrounding the project’s data tends to not look visually appealing to the average person.

“A good example of this is a 3D mesh created from LiDAR data. Whether it be a mesh of a building or a tree, the final product looks like a withered version of the real-life object. Replace these decaying meshes with 3D models like those seen in video games, and the visualization suddenly comes to life. When selling to stakeholders, more appealing 3D visualizations like these could go a long way in helping to gain confidence and win people over.”

I tend to agree. I am not sure how many times I’ve been shown a “digital twin” that was just some aerial or terrestrial photos processed into a 3D mesh, where the tops of buildings look like melted ice cream. While some can be quite detailed, I am not sure if they would fit the definition of a true digital twin.

And, as Treleaven pointed out, what you see is static. One key goal of Bending Time’s vision is to accommodate truly dynamic data. Not just timely updates of the underlying 3D data, but also purely kinematic data, such as wind, water, light, potentially traffic, pedestrians and more. To some degree, GIS is accommodating this, as are developers of engineering design and construction software that enable simulations. Rather amazing examples, albeit limited spatially, are presently in use. However, Bending Time seeks to accommodate this anywhere in the 3D globe where such external source data can be accessed.

Game On

How does Bending Time propose to improve the world of 3D worlds? For one, Treleaven is employing, among many tools, a game engine. It’s not hard to see that an accurate, gameable 3D globe (a 3D globe in a game engine environment) could open new ways of interacting with geospatial data.

“One of the first things that might come to mind when you picture these new interactions is simulator games like Microsoft’s latest version of its Flight Simulator,” said Treleaven. “A truly wondrous project that allows recreational and commercial pilots alike to fulfill their passion for flying planes all over the world. But this is just the beginning. Beyond games, we start seeing more practical real-world use cases like training for instance.”

Interactive 3D simulation can be used to train people in a wide array of jobs from merchant mariners to new drivers to surveyors to name a few (e.g. “Ready Surveyor One”). With new viewing formats like virtual reality (VR) and new[-ish] network architectures like the cloud, we are now in a position to deliver training anywhere without the need for students to travel to central training sites, pay for lodging, etc. Take this one step further and individual subject matter experts can set up their own schools online and offer virtual learning and training services to students across the planet. This leverages their experience enabling them to make money with very little up-front investment—especially true for the ageing, experienced baby boomer population.

Treleaven outlined other examples of applications that would benefit from this enhanced 3D globe world: “Project data can be imported for say, seeking public feedback on proposed development. You could be riding a virtual streetcar down a proposed greenway in VR with people on the sidewalk. Visualize forest cut blocks (or the fact you can’t see them) from the vantage point of someone on a ferry or cruise ship passing by. Transit proposals where you can see how the traffic is impacted by a ground-level light rail system.”

“From here, we get into simulations. Driver training has significant potential (2M new drivers in the US each year). Online maritime operations training. Proposed master plans and development can be presented to the public in a more convincing manner. In this last case, there are some “digital cities” now, where this is the primary use of the models.”

It was great to hear that Bending Time is working with the Unity game engine. Game engines are completely changing the landscape (no pun intended) for industries like manufacturing, engineering, and construction—nearly all industries for that matter. It is amazing what developer communities of Bentley Systems, Autodesk, and Esri have done with engines like Unreal and Unity.

“Extend these possibilities further into the future and you can start to imagine the beginnings of a Metaverse,” said Treleaven. “However, there is so much change involved in the activities I describe above—we can’t get too far ahead of ourselves. We need to take it one—geospatial— step at a time.”

Focus On the Third “D”

We might have drafted and designed in 2D due to the technological limitations of the past, but we live in a 3D world, and (many) are increasingly more comfortable in 3D virtual worlds. Adding the third “D” adds complexity to world-building, though I have some reservations about how the broader end-user communities would adapt to 3D enhanced environments.

“That one extra dimension adds just enough information to increase the cognitive load of the viewer,” said Treleaven. “This increase in cognitive load is enough of a barrier that most map users, like some city planners for example, who still prefer to work in 2D. However, with increasingly better viewing capabilities, 3D visualizations continue to be on the rise to communicate one’s geospatial project designs to others.”

That is changing fast. In the construction sector, for example, the use of 3D BIM [building information models] is making inroads—mandatory in some places. Larger firms often have virtual design and construction [VDC] teams. Think of BIM models, and digital twins developed for say campuses and industrial sites, as being mini-merged virtual worlds.

“For Main Street, what better example of this than Google Earth?” Said Treleaven. “When Google’s 3D globe hit the streets in 2005, it wowed millions around the world. And to this day Google Earth is an excellent tool to visualize your geospatial data in 3D. And while lots of users have been able to leverage it for the addition of their own data, typically though it is only as simple overlays. For some platforms, there are frustrating license and tech limitations that make it difficult to make it into something much richer, in terms of visualization and data.”

Esri’s ArcGIS Earth is a splendid example of a globe that can, among many other features, provide access to massive, and continually growing libraries of geospatial data, mostly 2D but with some 3D. This is an example of a globe that provides data richness, which enables infinite possibilities for spatial analysis—GIS exemplified. There are others. However, Bending Time is not seeking to become just another spatial aggregator. Instead, the focus is on visual richness and dynamics.

As noted, there are other zoomable globe resources, but what are the limitations of various approaches with regards to visual richness? “Unfortunately, some of the key developers are no longer putting significant resources into improving the visualization aspect of the products,” said Treleaven. “There are high quality 3D globes like Cesium, a spinoff from Analytical Graphics, Inc. [Ansys]. However, revenue for resources like these typically comes from subscriptions to data. That, and the fact that some 3D globes are JavaScript-based and run in a browser, makes it less accessible to the person/organization making the visualization. Of course, a browser-based 3D globe makes it very accessible to end users, but if it’s not easy for the producers, there is less data for consumers.”

Source Data

Industry has brought us great (albeit limited) virtual world tools. But they’ve done so when there is revenue to fuel that. However, the “free” elements they often provide are ultimately funded by ads or marketing end-user information. How does Bending Time propose to populate their merged virtual world with rich, relevant, dynamic data?

“By using open data and monetizing in-world products and services,” said Treleaven. “For decades, the global open data community has been collecting, preparing, managing, and publishing open data with a large part of it having geospatial attributes. This trend has seen steady growth and many developed countries around the world now have their own geospatial open data portals. Further, many provincial, county, and municipal levels of government collect data like LiDAR, imagery, and, of course, surveyed features. They, too, often have their own open data portals where anyone can download the data to use it. Even better, much of the data comes with a commercial-friendly open data license (e.g., Creation Commons Attribute-Share Alike (CC BY-SA). This is important as it widens the net for potential use cases.”

The Open Geospatial Consortium (OGC) comes to mind, making great progress in developing standards and fostering open geo initiatives. Treleaven elaborated on an example that illustrates how a specific source would fit into this proposed merged world:

“The OpenStreetMap project is a prime example. With millions of registered users worldwide (with many of those registered users contributing to the data), OpenStreetMap is a shining example of open data being put to good use. And with its commercial-friendly data license, companies like MapBox can develop and sell value-added products and services on top of OpenStreetMap. Some see this as companies profiting from the hard work done by ordinary citizens. I see it differently. I think the products and services companies provide, and the jobs they create, are a net positive for society.”

There is an explosion of open-source data. I see it being used to some degree for just about any project. One can do a shake-n-bake base map with nearly everything needed, short of the survey-grade data, for say civil engineering projects. In terms of a merged virtual world, now that we have this cornucopia of open data, I asked Treleaven how this would be leveraged.

“Interactive 3D visualization and simulation. In the case of OpenStreetMap, the data is currently being used as raster maps with optional vector services like vehicle routing, e.g., MapMyIndia [India], Baidu Maps [China], HERE [global], etc. Yes, 2D maps are still the easiest way for us humans to understand location, but in adding 3D data like openly published LiDAR and earth observation data, this opens up so much more functionality and enables the addition of even more useful information, e.g., visualizing height, volumetric data, etc.”

Next Steps

Treleaven: “I have been building this open 3D virtual world at Bending Time for over 6 years. With the plethora of map data collected and a 3D globe with 1 cm precision down to surface level running in the Unity 3D game engine, I am actively looking to work with organizations and/or individuals to bring their projects to life.”

Bending Time is seeking collaborators and investors. Treleaven can be reached via the Bending Time website, on LinkedIn, Twitter/X @streleav, and will be posting articles on Medium.