Artificial intelligence – New developments in surface inspection

Progress in the automotive paintshop is often linked to efficiency and sustainability, with the latest developments in technology typically boosting the scalability and modularity of systems. Cutting water consumption and reducing energy-intensive thermal processes have been key focus areas as carmakers look to lower the carbon footprint of their production facilities. But there have also been improvements in accuracy and quality thanks, in part, to upgrades in surface inspection tools. 

“Automated optical inspection (AOI) has had a significant impact on the production processes and structures in our paint shops,” Rudolf Hummelsberger, vice president, painted body, BMW, tells AMS. “It increases the efficiency of finishing processes and is a key contributor to the zero-defect strategy and our digitalisation plan.”

Control through AI
Surface inspection technology is used to find customer-relevant defects, typically “tiny specks and bumps on the topcoat surface,” Hummelsberger continues. These are often caused by fine dust particles in the air and can be removed with robots using sanding and polishing.

But conventional machine vision technology struggles to read polished and reflective surfaces like car bodies, so paintshops deploy a method called deflectometry whereby cameras are used alongside monitor screens that display a sequence of striped patterns, reflecting light onto the surface. These light reflections are then picked up by the cameras and fed through software to precisely calculate where the uneven areas are located on the surface of the body.

 “We consider the potential for using AI in our production systems to be very high” – Rudolf Hummelsberger, BMW

“The main benefit of deflectometry is that you have very good, accurate and objective data,” explains Wolfgang Berggold, business line manager, Isra Vision. “Additionally, high lateral resolution means that even extremely small defects can be detected, with a limitation of around 40 to 50 microns, which is very high.” In a recent webinar, Berggold and his colleagues outlined the benefits of AOI and described several use cases for using the technology during vehicle production, from paint shop to injection moulded B-pillars and even mirrors used for heads up display (HUD) units in the cabin. As the technology can be used across different areas of production, Berggold expects many carmakers to soon implement AOI in their factories for series production. 

BMW’s plant in Regensburg, in southeast Germany, became the first car factory in the world to start using AOI in March 2023 for series production. The entire process became digitalised and automated, from inspection through to the marking of painted vehicle surfaces for defect correction. This was achieved by leveraging AI. When the software receives data from cameras, it is analysed against data that has been previously collected to pinpoint any changes to the topcoat. The computer also saves the exact position, size and shape of the defect, and creates a digital 3D image that is entered into a catalogue and classified based on a specific criteria selection. The algorithm can instantly access this catalogue to quickly identify other defects, which Hummelsberger confidently says will ensure that no speck or bump, regardless of how small it may be, will escape detection.

“With the help of the continuous digital recording of defect characteristics by AOI, we derive trends and compile statistics that enable production to react more quickly to anomalies in the paintshop processes. In addition, AOI is the basis for the use of automated post-processing solutions, which we already use in series production with automated surface processing,” he adds.

Humans alongside AI
Despite accuracy and repeatability, AOI tools cannot currently process the edges of the body or the final millimetres next to a door and other joints. As a result, highly skilled human workers are still needed for surface inspection before the car can move along the production line, but instead of meticulously double-checking areas that the system has already cleared, a laser projector is used to digitally mark areas of the body that have not been looked over.

“We consider the potential for using AI in our production systems to be very high,” Hummelsberger muses. “At the moment, there is still a need for interaction between automated tools for surface inspection and humans, but we are gaining further experience with AI in production with some pilot installations.” 

At BMW’s Munich plant, AI is currently being used to compare live data from dust particle sensors in the paint booths to the data gathered from the AOI tools. At certain times of year, like dry and hot spells, dust particles may be more prominent. This can be detected by the algorithm, and if needed, filters in the air pumps can be replaced.

Although he cannot provide more information on the pilot installations, Hummelsberger hints at a greater reliance on AI in the near future. There are a huge number of data points currently in the paintshop, and he thinks that these can be used to further improve processes as production facilities evolve. Furthermore, robots in the paint shop may no longer need to be reprogrammed by humans and will instead be self-sufficient to ensure they are operating as effectively and efficiently as possible.

“The technology is constantly evolving, and the digitalisation of the finish areas opens up completely new possibilities for optimising our production systems,” he states. “We are working with our partners to enhance the systems in order to continuously improve our standards.”

Customisation presents challenges
But there may be a tricky stepping stone ahead for AI in the paintshop. Customisation is becoming increasingly common, as new car buyers look to attach unique characteristics to their vehicles, such as custom paint jobs with multi-tone topcoats. Furthermore, paint suppliers are developing new substrates that offer various benefits and are more environmentally friendly. With a different molecular make up, these new substrates can pose a challenge for an AI-controlled system that has already been gathering data.

“The use of different substrates or individualisation of the car body finish, such as multi-colour, poses challenges for the technology and must be technically evaluated on a case-by-case basis during implementation,” Hummelsberger stresses. He thinks that lessons could be learnt from the use of AOI in other areas of the vehicle, and even perhaps outside of the automotive industry.

Shelton Vision provides surface inspection machines for identifying the defects in textiles, car headliners, airbags, non-woven surfaces and composites. Speaking to AMS, Kiran Mohan, vision system engineer at the company, emphasises the complexity of using AI when considering various surfaces and materials, but is also keen to highlight improvements in the technology, making complex surface inspection possible. “The processing power of computers has greatly increased, and this has meant that we can analyse more intricate surfaces while simultaneously enabling faster inspections with more accuracy. The same can be said about cameras, lighting and optics,” he confirms.

Mohan admits that human inspection is still necessary, particularly in fields like automotive that demand exceptionally high quality across the entire product, from the surfaces on the dashboard to the paint finish on the roof. However, as inspection technology evolves and AI plays a greater role, he expects the robots to take more control: “Many companies are looking to automate the inspection process mainly to reduce labour and improve quality, so we certainly see a future where our systems are relied on for more areas of inspection.”

Part of a whole
Surface inspection technologies are a small but vital part of the paintshop, and Hummelsberger believes that through automation and the use of AI, the process is only improving. Furthermore, with the gathering of useful data, AOI is playing an even more crucial role in the digitalised production process, adding to information that could be used to reimagine paintshop design and other process areas.

For now, Hummelsberger expects trends like sustainability and the decarbonisation of factories to remain a top priority for BMW. At the start of 2024, one of the two base coat lines at the Regensburg plant was converted from wet paint to dry separation using limestone powder, with the second line due to be switched in 2025. This method will save a total of 17m litres of water and around 17,400 megawatt hours of energy per year at the company’s Regensburg and Dingolfing plants.

Smaller improvements will also be made, and Hummelsberger believes that AI could help make it all happen. “Simulations for product integration and analytical tools for ongoing process monitoring will be highly relevant. Paintshop processes will be further optimised using sensors combined with AI. This will have an effect on all key performance indicators in the medium and long term, and will form the backbone of our paint operations,” he predicts.