Manufacturing in the metaverse

It is still another couple of years before production is due to start on BMW’s projected next generation of all-electric vehicles that it entitles appropriately the Neue Klasse. The location will be Debrecen in Hungary in a production facility that will be as new as the vehicles because it also has not yet been built – construction has only just started.

Nevertheless, the company is confident that when manufacturing does get underway everything will run smoothly. That is because in what the company claims as a breakthrough for itself and even for the wider automotive industry, all the relevant production operations will already have been simulated and validated in the virtual world.

The basic fact is confirmed by BMW’s board member for production Milan Nedeljković. “Virtualisation and artificial intelligence (AI) are accelerating and refining our planning,” he states. “With the various planning systems consolidated within a digital twin, our teams around the world can now work together in real-time and make decisions faster and on a more solid foundation.”

Strong software foundation
That foundation, he also confirms, is a partnership between BMW and software provider NVIDIA. This close collaboration is focused on the carmaker’s use of the software company’s Omniverse system, which effectively enables the exploration of a virtual world constructed using AI and various third-party application software packages and databases. The practical application is an ability to “run real-time digital twin simulations to optimise layouts, robotics and logistic systems virtually”.

More details are provided by Felix Theurer, digital solutions architect with BMW. “The platform has been tested and jointly industrialised for a period of about two years,” he confirms. “We have now reached a state where we are confident that the platform can be integrated into the planning environment and into the process so that a significant benefit can be provided.”

In fact, as Theurer also confirms, the implementation of the Omniverse-based approach to the planning of the projected Debrecen plant is only one part of a company-wide initiative to make its use pervasive at a global level. “The roll-out has just started,” he states. “Omniverse will be available to planners all over the world and will be used to plan updates of existing as well as future plants.” This means that when fully implemented it will be used to support the operations of over 30 manufacturing sites worldwide.

As Theurer further explains, the value of the software is in the way it can facilitate both the integration and consequently enhanced comprehension of highly varied data. “Omniverse helps us in the planning process as it can visualise information from different data sources in just one platform,” he states. “That is important for planning entirely new locations as well as for the reconstruction of existing plants.”

Several further application software systems from multiple suppliers are involved in feeding data into the Omniverse environment. “They currently include Bentley Systems MicroStation for layout planning, ipolog for logistics planning, Siemens Process Simulate, Dassault Systemes CATIA for vehicle design and Autodesk Revit for building planning,” states Theurer. “There will be more tools to follow”.

“It is possible to do visual inspections of parts in a way that is very similar to how it is done in the real world. In fact, with the use of AI it is even possible to detect and evaluate flaws on the surface before an actual part has ever been run off the press” – Felix Theurer, BMW

Versatile digital tools
Theurer makes a point of stressing the versatility of the software meaning that it can be used to equal effect as a planning tool for both complete new-build projects (as at Debrecen) and the upgrading of already operational installations. “Existing production lines have to be remodelled on occasion in order to be able to integrate new models into the production process and this always means changes of equipment like robots, tooling, conveyers and so on,” he observes. “But now whenever a new car model is about to be integrated into a production line, we are able to validate the whole process virtually before the hardware is installed and robots are programmed.”

The benefits this brings are tangible, quantifiable and substantial. “This brings us very close to a zero loss launch because the amount of rework to the production system can be reduced to nearly zero,” says Theurer.

Theurer also makes the point that in itself Omniverse is not any sort of complex simulation system but instead provides a means for the information generated by the various application packages it collates and organises to be evaluated in the most immediate way possible – by the human eye. “Simulations aren’t generated in Omniverse but visualised based on industry software which we’ve been using for decades,” he says. An important implication of this methodology is that the Omniverse environment is not a reserved space for a group of dedicated specialists but instead, with the aid of a little further customisation, provides a generally available resource for people from all areas of the company. “We’ve developed custom applications for non-3D-experts to create and maintain the virtual factories,” says Theurer.

Identifying planning mistakes
The primary objective of the whole exercise is also quite clear. “The main purpose is to enable improvements even before the hardware is realised,” says Theurer. “It also provides a means of identifying planning mistakes that were previously only detected during the realisation phase.” Nor are the sort of potential faults that can now be detected trivial or inconsequential. “There were already situations where the visualisation of robot arm movements in the virtual world prevented the potential destruction of a catwalk in real life,” Theurer says.

Indeed, Theurer can provide extensive further details on how BMW exploits the possibilities this new approach opens up. He indicates that they range from ensuring individual part quality right up to process validation. “It is possible to do visual inspections of parts in a way that is very similar to how it is done in the real world,” he says. “In fact, with the use of AI it is even possible to detect and evaluate flaws on the surface before an actual part has ever been run off the press.”

But it is perhaps at the large-scale level where potential of the approach is most fully realised. “It is now possible to validate the assembly process in combination with human simulation,” Theurer confirms. “That way we can get the line set-up optimised before the hardware is bolted to the floor and we can also validate if safety margins are maintained where parts must be moved, and humans are involved.”

The consequence, Theurer continues, is the elimination of costly reorganisation after a production system has been built in real life. “So even if the virtual factory doesn’t revolutionise the design process of the car, we can still say that it helps to improve the return on capital employed when setting up the whole production system,” he says. As such, says Theurer, the use of the Omniverse approach for the intended Debrecen plant is not in any way exceptional except for its extent. “It is a showcase as it is our latest project and we are planning it virtually from scratch,” he states.

Data for AI training
For the future, though, BMW’s intentions for the use of the Omniverse are even more ambitious. “Our goal is to use virtual factory models to generate synthetic data for AI training,” says Theurer. “This AI could then, for instance, be used for quality inspection via camera portals or for navigation and object recognition in small transport robots. The advantage of this approach is that we can begin training before the environment exists in hardware.”

Theurer makes it clear that the use of Omniverse will enhance digital operations in the broadest terms and that BMW and NVIDIA teams are already working towards this. “This will allow operational faults to be localised in a matter of seconds and thus prevent longer production downtimes,” he states. “It is also an important step towards, for example, integrating and automating the virtual commissioning of new systems into a continuous planning process.”

This whole approach is, of course, an example of the application in practice of a concept that was unheard of even just a few years ago but which is now becoming almost commonplace – that of the ‘metaverse’.

That, in turn, is confirmed by Mike Geyer, product manager, Omniverse Manufacturing, for NVIDIA. “Omniverse is an open reference development platform for building and operating metaverse applications whether for designing and manufacturing goods or simulating future factories,” he states. The present ‘Enterprise’ version, he says, was launched as recently as 2021 with BMW amongst the earliest adopters.

In practical terms this means that the system “connects 3D design products to enable a more seamless, collaborative workflow between those tools,” says Geyer. He further explains that users can then move between these different software environments through Omniverse because of the platform’s support for open 3D standards like Universal Scene Description (USD). “We are working with over 40 software companies to build even more real-time connections to make it easier and faster for users to build virtual worlds and the content within them,” he adds.

Interestingly Geyer indicates that the computing power required to support these capabilities is not necessarily immense. The appropriate “graphics card in a PC or workstation” will run Omniverse, he says. But he adds that Microsoft Azure is now the first cloud service provider for the platform. For highly complex workloads Geyer says that NVIDIA has just unveiled the third generation of its OVX computing system, which is designed to power large-scale digital twins built on NVIDIA Omniverse.

Moreover, Geyer confirms that though the approach is generally applicable the automotive industry is at the forefront of its implementation. “The auto industry is leading the charge to deliver savings across their entire workflow and product development cycle,” he states. Indeed, apart from BMW he adds that “Mercedes-Benz, Lotus Cars and Toyota are using Omniverse to streamline factory planning and production”.