Moves in Machine Vision

Ketan Mistry explains how some of the latest advances in machine vision technology are increasing the effectiveness of the automated inspection technique.

It would be difficult for a company to sustain its competitive advantage if it solely focused on its financial performance and ignored its social obligations. Some of the most successful global organizations have been able to meld their corporate and social strategies to develop true sustainable competitive advantages in the market, leading to sustainable long term financial performance andsocial performance.

Global warming, pollution in cities, water contamination, soil depletion, food shortages and many other problems have arisen due to our neglect of the environment. In addition, resource shortages are set to continue to be the norm for several more decades. In this situation, manufacturing industry will be constrained by scarcity of resources and increased costs of raw materials such as crude oil, metals, commodities, etc in their pursuit of productiveoutputs.

Only those manufacturing companies that have the capability to operate in an environmentally sustainable way are likely to survive. And this means embarking on systems and processes which minimizethe use of resources to produce more outputs. In layman terms, this is principled around zero wasteand zero defects management.

In the past, automated systems were implemented chiefly to improve production line productivity and reduce manufacturing costs. But, now automation is increasingly being applied to reduce scrap, eliminatedefects and to minimize resource consumption.

One of the forefront technologies delivering this value is “machine vision” – the technology that replaces human eyes and judgment by cameras andartificial intelligence.

For example, one of the world’s top biscuit manufacturing companies installed vision sensors in its baking line in order to detect the color of the biscuit after the baking process. Previously, the production inspector and operators controlled the line speed and oven temperature based on the color of the biscuits exiting the oven. (The color of a biscuitindicates if it has been baked properly or not.)

There were several problems arising from this arrangement, as different operators would view and interpret the colors differently and make judgement based on their own personal experience. In the company’s pursuit to maintain stringent quality, slight deviation of biscuit color was considered unacceptable. Since, there was no commonbenchmark, the scrap rate was high.

The company executives scouted for various options and solutions and selected a vision sensor with enhanced color capability technology. This type of vision sensor converts color data into numeric digits of R, G, B. Example: a color is registered as R=225, G=150, B=130, and each can be set todifferent threshold values.

With the introduction of the vision sensor, there were two immediate benefits. First, it was possible to set a clear reference for the color of the biscuit after baking, thus eliminating ambiguity arising from person to person judgment. Second, it was now possible to arrive at the required baking conditionspeedily.

Thee biscuit recipe now includes R, G and B data from the vision sensor, in addition to the oven temperature and conveyor speed. And the closed loop feedback control enables the required bakingcondition to be precisely achieved.

Being one of the largest food manufacturers inthe world, the company has an environmental policy in place that also includes efficient use of resources – i.e.minimizing usage of raw material, water and energy. As a result of itsapplication of machine vision, the company has managed to reduceits consumption of flour, sugar, dairy products, water and energyby a significant margin. This is not just bottom-line improvementbut also a contribution to the social responsibility philosophy of thebusiness.

Cost down

Despite its obvious benefits, machine vision has so far seen rather limited industry adoption. Some of the identified barriers aretechnology gap, cost, and lack of user friendliness.

As far as the cost of machine vision is concerned, the average price range of a complete vision system has come down dramatically with advancing technology. Similarly, user friendliness of machine vision has improved drastically after stand alone systems, i.e. vision sensors, entered the market. A vision sensor does not require the complicated integration of camera, frame grabbers and softwareprogramming required in first generation machine vision systems.

Even though price has reduced and user friendliness increased, many applications are turned down by machine vision specialists each year. Some of these applications, if solved, can realize significant reduction in waste and contribution to environment. This is enough motivation for those machine vision suppliers thatare equally passionate about environment.

While the demands for products to be inspected by machine vision have increased, the levels of challenges have also gone up. This is because of the gap between current technology and the userrequirements.

But while machine vision has certainly progressed – from monochrome to color, from 300K pixels to several mega pixel camera resolution, and a several fold increase in processor speed – such enhancements cannot make any difference if the processing software is not as powerful. For example in the application ofdetecting biscuit color, real color technology is the key.

Getting real

The first machine vision systems used monochrome processing. In this technology, color information is reduced to detecting brightness only at the time of image capture. After several years of development came color image processing, with the two mostcommon technologies being color pick up and colour grey.

In the color pick up technique, only the selected colour isextracted from the color image and processed, while the remaining colors from the image are blackened. Color pick allows up to eightcolours to be selected and processed simultaneously. In the colorgrey technique, one of the selected colors is converted to grey scale of256 levels of black and-white brightness and the contrasts of specificcolor are enhanced.

Both of these techniques have disadvantages: color pick up is highly sensitive to lighting condition variation; and color grey is limited to a single color and the processing efficiency is dependent on how the color filter is set. As a result, both the color image and color grey lose lots of information after the image is captured by the camera. Moreover, subtle changes in images with low contrastcannot be detected.

Based on the concept of the human eye where an image is processed with lots of information, “real color” technology (introduced by Omron) manages to break through the above constraints; there is noinformation loss either during image capturing or during processing.

Different colors are represented as different positions in the 3D RGB space, and subtle color variations can be recognised by representing them as distances between different colour pixelscomprising this space.

One of the advantages of real color sensing processing is stable measurements in different inspection environments. One drawback,however, is that it requires high-performance processing chips.

Currently, Omron is using a custom made Dual Mega ARCSEngine that can do multi-processing.

One example where real color technology can make a difference is inspecting foreign particles in connector moldings. Traditional color vision cannot detect defects the color of which cannot be defined, such as oil stains. Another example is inspecting wiring harnesses for color sequence. Traditional vision processing using color pick up can only see limited colors and each color sequence needs to be taught. And color grey would see only one color. But real color caninspect any color sequence by comparing it with the register model.

But while the technology works very well for color objects, it can do little when faced with a reflective shiny metallic object. Traditionally, vision specialists have been using lighting techniques and filters to get clear images of such objects, but success is not always guaranteed. However, a new technology called HDR is emerging which utilizes software power to eliminate the effects ofreflection and produce a clear image.

Higher range

One of the most difficult tasks in machine vision is the generation of a clear image for individual inspection. Image processing becomes easy if clear images are captured regardless of lighting variation,reflection and poor contrast.

The higher “dynamic range” the vision hardware scores, the clearer images it can generate when imaging objects with astrong contrast in luminosity. Dynamic range means the imaging hardware’s ability to detect differences in luminosity. But one of theproblems in machine vision has been the limited dynamic range ofcameras.

However, a machine vision processor featuring new “high dynamic range” (HDR) image generation technology takes two or more images of a work piece at different levels of luminosity by changing the shutter speed and synthesizes them into a single imagerapidly. This minimizes the effects of adverse lighting condition.

An image processor loaded with HDR technology can also enhance the contrast in the area to be inspected, by overlapping and synthesizing two or more images taken at the same shutter speed. After the synthesis, noise contents are suppressed while the area tobe inspected is amplified by integration.

HDR can make a significant difference in the inspection of metallic objects, such as punched or laser marked codes on automotive components. As depicted below, the top left-hand image is facing problem of overexposure while the bottom image is underexposed. As a result, both these images cannot be utilized for processing. The image on the right shows the same object captured using HDR technology. The surface of the object as well the 2D code is clearlyvisible, with enough contrast required for reliable processing.

Other examples where HDR technology can improve performance include the inspection of electrical components featuring shiny metallic pins and a black mold body under the same lighting conditions. Shiny and cylindrically curved objects are alsogood candidates for HDR.

Critical role

Yesterday’s businesses were oblivious to their negative impact on the environment. Today’s businesses are striving for zero impact on the environment while ensuring profitability. After all, sustainable manufacturing means meeting the needs of present without compromising the ability of future generationto meet their needs.

Factory automation suppliers play a critical role by providing products, services and technologies that can help manufacturing industries to realize sustainable environmentally-friendly operations. The result is not just new business opportunities forthe industry, it is also the social responsibility.

Ketan Mistry is Manager, Omron Asia Pacific(


Into a New Dimension

Poorly performing readers and the need to greatly increase the extent of encoded information led this electronics manufacturer into the realm of 2D bar codes. By Cognex.


Without a doubt, the electronic devices we see in the market today are shrinking in size. Be it an MP3 player, a DVD recorder or a laptop computer, small and slim is the way to go. As such, the printed circuit boards (PCBs) that go into these electronic gadgets have inevitably downsized in terms of overall design and form factor. Hence, it becomes increasingly challenging (and sometimes impossible) for manufacturers to adhere bar code identification labels onto these PCBs.

Beyonics Technology, a Singapore based provider of advanced contract manufacturing services to original equipment manufacturers (OEMs) in computer storage devices, medical devices and electronics communication products, was one company that faced such challenge, as it saw its customers’ product designs shrinking in size over the years and leaving little real estate for the bar code label.

Amplifying the problem was the increasing demand for product traceability requiring more and more manufacturing related information like lot code, vendor ID, product number and serial number to be encoded within a single and smaller bar code label.

Beyonics and its customers decided that migrating from on oneto a two-dimensional ID code could be a solution to this problem. And of the available 2D codes, the Data Matrix symbology appealed because of its high data capacity and integrity.

Pain before gain

Although Beyonics had identified a seemingly suitable approach, the real challenge lay with the actual implementation. Within the plant itself, Beyonics had many in-circuit and functional testers using different bar code reader configurations. These readers were in poor condition, could not read 2D codes, required custom cabling to handle various triggering inputs, and were configured to communicate with different protocols and set to output nonstandard types of string formatting.

It was thus necessary for Beyonics engineers to find a new reader that could handle the new demands, while ensuring that the corresponding migration effort be kept to a minimum in order not to take up too much time and resources.

“I do not wish to perform any re-cabling or re-programming on my current setup. Writing a set of new documentation would have been a nightmare. Hence, the new readers have to be plug-andplay,” said Parthiban Periayah, engineering specialist at Beyonics Technology.

Through a few sessions of on-site troubleshooting and testing, one-to-one direct replacement of Beyonics’ fixed position readers with new 2D capable devices (DataMan 100 from Cognex) was achieved without having to alter any software programming codes or hardware wiring configurations. This allowed Beyonics’ engineers to concentrate on their main tasks of maintaining the SMT machines and keep the production lines up and running.

Downtime down, yield up

“Maintaining a high manufacturing production quality has always been a priority and a belief at Beyonics. Our mission is to be a world-class manufacturer, delivering high quality products and services to our customers through an ongoing improvement programme,” said Chan Foong Yow, senior engineering process manager at Beyonics Technology.

And certainly, with more product designs being churned out from customers, Beyonics constantly has to look at ways to optimize its manufacturing process to drum up productivity and reduce downtime. To this end, the replacement of the old bar code readers has resulted in greater efficiency and more effective deployment of resources.

As Shanker Kaneson, another engineering specialist at Beyonics, remarks: “We were experiencing an unacceptable read rate with the old readers. Operators often had to reload the PCBs when a no-read situation happened. This caused the SMT machines to come to a halt. With the DataMan 100 readers installed, we witnessed a production throughput increase of about 10 percent – and also reducing the frustrations of the operators and eased the load of the line engineers.”

Beyond the deployment within the SMT Machines, Beyonics has started to use the same model of readers in its Diagnostics Operation, and has also replaced its handheld readers in the plant.

Shanker notes, “Compared with our existing handhelds, the DataMan 700 is an “electrostatic discharge safe” device, which conforms to the specifications published in the IEC 61340 standard. This translates to peace of mind when we use the readers in an ESD protected area. In the past, we had to wrap the handheld scanners with an ESD-safe jacket, which was awkward and non-IEC compliant.”

Cognex Corporation ( is a supplier of machine vision sensors and systems.


  • More About