Abstract

Complex coloration is important in human perception but it has presented serious obstacles to traditional machine vision approaches. Our proprietary statistical approach, minimum description, overcomes these problems. It also allows true training by example without the necessity of color space transformations, threshold setting, or other traditional operations requiring special technical knowledge or judgement.

Introduction

This image shows a classic example of such an application: an automobile junction box containing color coded fuses. The requirement is to verify that all fuses have been inserted in the correct locations. Note the non-uniformity of all colors and the very close similarity of the yellowish 20 and 25 amp fuse colors.

Modern automotive interiors often use multicolored fabrics, such as those shown here, for seating and interior panels. During production it may be necessary to determine which of several fabrics is being used in an assembly, and possibly even the orientation of this fabric. While easily handled by WAY-2C, fabrics like this may be extremely difficult to differentiate using traditional color machine vision methods .

One of the more interesting applications of color based recognition we have encountered is illustrated in this image. In this case a variety of slightly different sun visors are manufactured in the same work cell. The visors may be one of several different colors and may be for the left or right hand side of the vehicle. Some visors have vanity mirrors, some do not; some of the mirrors may be lighted. Finally (not shown) a visor may have one of several different labels depending on the country of destination. The inspection task is to determine all the above visor properties, and to print a bar code and label which correctly indicate these properties.
For more information on how such labels are created.

Another interesting application of color based recognition is illustrated here. In this case a company packages compact disks in transparent plastic "jewel cases". Each package must contain a variety of components including: a paper insert in the bottom of the case, a tray to hold the disk, the disk itself, a booklet and a spine label each of these can be in a variety of colors and designs. All of these are surrounded by a transparent wrapper. Finally one or more stickers may be applied to the outside of the wrapper. The inspection task is to determine that each of the listed components is present and correct.

Each of these inspection tasks is characterized by objects of complex shape and color distribution. Yet after a brief training session, few humans would have any difficulty in inspecting the assemblies based on viewing the colors at the relevant locations. No formal segmentation of the region to be inspected, or setting of thresholds would be required.

Human inspectors, of course, have reliability problems, not because the job is hard, more likely because it is so easy that it becomes boring. Attention wanders, and defects are missed. Other human disadvantages are the comparatively slow speed and high unit cost per inspection.

Reliability and speed are hallmarks of computers. What one needs to perform the above inspections is a reliable, fast, cheap color distribution sensing computer which one can train simply by example but which can still handle even the most complex color distributions.

The requirements, theoretical basis for, and implementation of such a system are discussed below.

Color Vision System Requirements

This requirement suggests in advance that methods which compute the mean, median, standard deviation, variance etc. of the test and reference data and then compare these statistics rather than the data themselves are likely to have limited success in comparing distributions. And indeed history has shown this to be the case.

A second requirement is the ability to train by showing. This makes it suitable for use by unskilled operators and removes the necessity of such sources of error and annoyance as threshold setting.

Other requirements are reliability, speed and low cost.

Theoretical Background

Consider a pair of histograms, one representing the reference distribution R and the other a test distribution T. There are obviously many functions which might be used to compare these histograms to decide which reference histogram a test histogram most resembles. Some of these functions are referred to as distance functions since they decrease in size as the similarity between distributions increases. Typical distance functions include:

• Absolute differences
• Sums of squares of differences

• normalized correlation

Over the years the above and similar functions have been tried with only limited success for color based inspection.

In addition to the problem of comparison functions there is problem of selection of the data on which to base the comparison. Most existing color machine vision systems which offer statistical matching must analyze the three color components separately. However both common sense and a rigorous analysis show that such separate analysis must inevitably introduce additional uncertainty. The deficiency may sometime be decreased somewhat when working with saturated colors and variable intensities by converting to HSI (hue, saturation, intensity) space while RGB space may be the better choice with unsaturated colors such as subtle tints. Most systems which must rely on separate component statistics offer the option of working in either space.

By replacing individual component analysis by a simultaneous analysis of all three color components, the advantages of one color space over another should become negligible. However, other problems have inevitably arisen when traditional statistical matching approaches have been applied to the complex color distributions associated with most scenes of interest. Too often they seem to work reasonably well when test and reference color distributions are almost identical, but require awkward, ad hoc, corrections or operator intervention to avoid bad decisions as the test and references begin to differ significantly. One way to interpret this is that the problems are inherently difficult, and will provide employment for "algorithm specialists" for the indefinate future. Another interpretation is that there is a better way to measure the similarity of the distributions. After all, birds, bees and other fauna with relatively small brains and little formal mathematical training seem to have little difficulty in performing this type of color recognition. Also, the most commonly used statistical matching methods are based on the assumption that the most likely color is very close to the mean color, and each color distribution has a single normal bell-shaped, Gaussian shape. The fact that the assumptions are valid in only a tiny fraction of the potential applications of the images of interest suggest a possible explanation of the difficulties.

Minimum Description Classification

The method has now proven itself in a wide variety of industrial, government, and military inspection and process control applications.

Some Challenging Examples

The first is an image pair illustrating the ability to interpret resistor color codes in the presence of glints, shadows and less than perfect markings and positioning. Each interpretation region is one pixel high and extends over the entire width of the region in which the component is expected. Note how classifications are not significantly affected by varying proportions of green and white in the background.

Practical Color Inspection Systems

This is particularly true in this modern era of flexible manufacturing, short production runs and high quality, flexible easily trained inspection systems are essential. When a product may change several times in a shift, inspection system setup and retraining times of the order of a couple of minutes or less may be required.

To address this problem we have developed several types of minimum description based color vision systems. The first of these is WAY-2C, an interactive script driven color machine vision system which can be programmed in a couple of hours to do most simple inspections such as checking and reporting or controlling a process based on which of several conditions prevails. A typical simple WAY-2C based system is shown in this image. It consists of a PC, frame grabber, color camera, and optional monitor and can accommodate most digital I/O boards.

For more complicated inspections, such as junction box inspection, more complex inspection systems, such as that shown here are necessary. In these cases the inspection scripts may have to handle a variety of different components and arrangements of these components using operators with only minimal computing skills. Training and inspection scripts are generated automatically from a database containing information on the location of the various fuses. Minor adjustments in position can be made simply by moving windows around with a mouse, and selecting actions from menus.

Our Assembly Color Inspection System and Canister Thermal Image Monitoring Systems are examples of the latter type of system. These start with some basic inspection scripts, but add a set of interface routines which can take the output of one operation to generate new scripts to be used in training, testing and on-line inspection. Such specialized applications make it possible to setup for a completely new assembly in a matter of several hours, and to switch between previously setup assemblies in minutes.

References