AMS reports on the most recent developments in the use of machine vision technology for quality control applications in automotive manufacturing
In response to rapid and sustained improvements in machine vision technologies, a growing number of automotive manufacturing OEMs and suppliers are now deploying such systems in a wider range of uses. According to Marc Benevento, managing director at Industrial Market Insight, machine vision is currently used across a number of automotive powertrain applications, where he stresses the cost of failure is “high in terms of warranty dollars and customer goodwill”.
“In addition, machine vision affords the opportunity to conduct 100% inspection on parts and assemblies that would not be practical or possible using human inspection. Potential applications exist anywhere it is necessary but impractical to validate proper assembly with visual inspection,” he says.
In terms of practical applications in existing automotive manufacturing plants, Benevento also reveals that machine vision is employed in both powertrain and body assembly operations – and is used widely “to verify that critical engine components are properly assembled and installed, which prevents costly engine repairs or replacements.” He adds: “Machine vision is also used to guarantee the proper placement of adhesives and sealants in engine and body applications.”
New automation solutionsOne of the early pioneers in the development of machine vision technologies is the Germany based MVTec Software, which has created a number of general purpose machine vision software products – including the Halcon and Merlic systems – used for imaging in a wide range of areas, such as the semi-conductor industry, inspection, optical quality control and metrology.
"The capabilities and ease of use have been improving consistently over the past 30 years when we first started to see widespread use of machine vision in automotive production" - David Dechow, Fanuc America
As Dr. Olaf Munkelt, managing director at MVTec, explains, the company’s software is geared towards enabling what he describes as “new automation solutions in settings of the industrial internet of things”. In particular, he says the Halcon system is “optimised for the needs of OEMs and system integrators and allows engineers to set up their own solutions for a specific machine vision task”.
Such tasks include 3D machine vision, matching to find unevenly rotated or partly occluded objects, blob analysis, high-accuracy measuring and optical character recognition and verification (OCR/OCV), and the detection of arbitrarily shaped regions of interest (ROIs). Meanwhile, Munkelt explains that the MERLIC system is an “all in one machine vision software product that enables users to build complete solutions without programming”.
As far as the automotive sector is concerned, Munkelt explains that both systems have already been used extensively for the inspection of many components produced at manufacturing facilities – from simple surface inspection up to complex 3D measurements or pose estimation applications, such as robotic guidance. To date, Munkelt says MVTec software has been used in many quality inspection applications within the automotive industry, for large manufacturers like Toyota and Kawasaki. T
[sam_ad id=17 codes='true']
The company has also recently collaborated with system integrator Ryf and camera vendor Kappa optronics to create an integrated image-processing system for Polydec, a leading micro-axle supplier, where it was used for the quality assurance of hardened-steel devices, a critical component of automotive step motors.
“Other machine vision technologies that are important for quality assurance within the automotive industry include those dedicated to matching, measuring, position detection (2D/3D), bar code/data code reading, also: OCR [Optical Character Recognition],” he says.
Elsewhere, David Dechow, staff engineer, Intelligent Robotics and Machine Vision, at Fanuc America, points out that machine vision is now a “very mature and well-incorporated technology” for automotive quality control, with an extremely wide-ranging application base covering “everything from assembly verification and measurement to body panel fit and paint colour verification”.
For Dechow, the interesting thing about the automotive industry with respect to machine vision is that it contains an extremely diverse set of applications. In some instances he believes the target applications have similarities to those found in the electronics industry, whereas in other cases “they can resemble machine vision applications in pharmaceutical, packaging, distribution, and other industries”.
Diverse applications“Overall, the application opportunities are extensive, and not all possible uses have been fully realised,” adds Dechow. “Machine vision is a key technology in achieving 100% on-line quality control. The capabilities of the technology and ease of use have been improving consistently over the past 30 years when we first started to see widespread use of machine vision in automotive production,” he says.
“Today’s systems are able to do many quality tasks, but design constraints due to the physical production line environment are often the largest limiting factor. New assembly and production systems that incorporate machine vision as an integral part of concept and design are producing more reliable and effective inspection results,” he states.
One of Fanuc’s developments is the iRVision system, a package that includes machine vision software and 2D and 3D imaging components, which Dechow says provides a configuration environment and set of machine vision tools “that make the configuration and implementation of complex inspection and VGR [Vision Guided Robotics] tasks faster and easier”.
“Like most machine vision systems, iRVision works by capturing images (in either 2D or 3D) of a part, feature, or groups of parts and features, then uses the image to perform either quality inspections or – most importantly – provide precise locations of the parts or features within the robot real world space,” says Dechow. “These locations are used in a variety of ways in VGR [Vision Guided Robotics]: for part picking and placement in assembly, machining, and handling; for path adjustment for applications including painting, sealing, and dispensing,” he adds.
Using a robotic platformWith iRVision, Dechow reveals that Fanuc looks at machine vision inspection or quality control from the point of view of robotics. By incorporating a robot, he believes that the company is capable of realising a “more flexible and easily-configurable inspection system for a variety of applications”.
“A robot-carried imaging system is not constrained by part presentation and location – or by the need to mechanically position multiple cameras in front of each feature to be evaluated,” he says.
"We predict deep learning technologies on classification of defects, deflectometry for reflecting surfaces, identification of objects and embedded vision" - Olaf Munkelt, MVTec
This flexibility has enabled iRVision to be incorporated in many quality inspection applications in the automotive industry, with typical applications involving part or feature identification and differentiation and presence or absence detection for quality and assembly verification, as well as measurement to detect out-of-tolerance parts, colour identification and verification, and code reading. One example is the use of robot-carried cameras to check engine assembly. Over multiple views, Dechow reveals that the camera has been able to check for the presence or absence and proper location of rocker arms.
“In another typical application, as part of an automated robotic assembly operation, the robot would pick randomly oriented bolts from a flexible feeder for insertion into an engine block. During the location process, iRVision would also verify that the bolt was the correct part based upon measurement,” he says.
Mixed material assembliesIn terms of ongoing trends, Benevento notes that OEMs are now increasingly using mixed material assemblies in vehicle bodies to reduce weight, and using structural adhesives in conjunction with other fastening techniques, such as riveting, when joining metals and polymer composites.
MVTec's car-door deformable matching system“Machine vision could be an enabling technology that gives OEMs greater confidence to rely on structural adhesives as the primary bond for body structures. This would allow them to reduce the number of spot welds and rivets. This enables them to reduce fixed and variable costs associated with fasteners and spot welds, and to shorten cycle times by reducing the number of joining operations,” he says.
Meanwhile, Munkelt singles out Industry 4.0 as a key future trend and one that he thinks will require a simultaneous improvement in the “intelligence” and connectivity of sensors. “On the technology side, we predict deep learning technologies on the production floor for the classification of defects, deflectometry for inspecting objects with reflecting surfaces, identification of objects and embedded vision,” he says.
Ultimately, Dechow believes that advances in imaging, particularly 3D imaging, over the next several years will “significantly enhance the capability of machine vision for quality inspection applications.”
“The ability of machine vision systems to more accurately identify features and provide better measurements will drive more plant floor applications,” he adds.
