Abstract

Introduction

The advantages of having reliable, automated, full color inspection systems whose decisions are consistent with human perception have long been recognized. Now that inexpensive color cameras and color frame grabber boards are readily available, the major obstacle to widespread use of color has been the lack of suitable general classification methods.

To mimic human color recognition a machine vision system must be able to operate in a single three-dimensional color space, learn quickly, and handle multi-modal, overlapping, color distributions easily. At the same time it must remain sensitive to anomalous characteristics, even under non-uniform lighting conditions.

Typical Applications in Baking and Roasting

Figure 1a shows a group of four blueberry muffin images which serve as references of different degrees of doneness from a single batch. The numbers increase with cooking time and therefore darkness. Muffin #3 is undercooked, muffin #4 is optimum, muffin #5 is slightly overcooked and muffin #6 is substantially overcooked.

The fifth muffin (Figure 1b), to be graded by comparison with the references, is from a different batch.

Figure 3 illustrates a slightly different class of problem. It shows part of a tomato, cheese and mushroom pizza. Except for a small amount of tomato sauce exposed on the extreme right of the image, most of the tomato sauce is overlain by cheese which is in turn topped by scattered mushrooms. The objective was to map the pizza surface to determine which of the three ingredients was exposed at each location and the total amount of each appearing in the image.

Inspection Issues

All of the required classifications are easily performed by humans. The objective of this study was to see how the results from, WAY-2C, our color machine vision system based on the minimum description approach would compare. The hardware for a typical vision system (Fig. 4) is relatively simple: lighting, camera, and computer. The computer contains a frame grabber board with dual ported memory which captures the signal from the computer and makes it available to the classification software. A digital I/O capability accepts a trigger signal to tell the system when to begin inspection and outputs the results for process control purposes. The challenge is in the color classification software. To classify objects of relatively pure color a number of traditional methods work quite well. These include: table lookup, thresholding, and nearest neighbor classification. Unfortunately baked and roasted foods, like the muffins and peanuts shown here, may be far from uniform in color. In such cases the underlying assumptions of the traditional methods may be violated causing the traditional classification methods to fail (McConnell, Massa, and Blau, 1995).

Figure 5 shows muffin #4 from Fig. 1. The scene contains several clusters of pixels in color space. These include the cake cluster (mostly light browns), the blueberry cluster (dark), foil clusters (primarily grays and whites) and the shadow cluster (generally dark). The total number of pixels in the cake cluster is irrelevant but the relative number in each position in the color space is important. As the muffin is cooked the position of the pixels in the cake cluster tend to shift in color space. Meanwhile the colors of the blueberries, foil and shadows remain essentially constant providing no information on the degree of doneness.

Fig. 6. shows a closeup of one of the peanut images. Here again there are several basic color clusters: peanuts without skins, peanuts with skins, shadows and glints. For this application only the color distribution of the skinless peanuts was of interest. The muffin and peanut images, like most images, are multimodal: they have several distinct color clusters. In both sets of images one of the clusters varies in quality with the property of interest (doneness) but remains relatively constant in quantity. The other color clusters, which do not reflect the property of interest, are relatively constant in quality but may vary in quantity. Since the quantity of the irrelevant colors may vary from sample to sample, any method sensitive to changes in the quantity of these components, such as those based on mean values or thresholds, must isolate the irrelevant colors before performing the classification. Historically this has involved specialized algorithm development for classification thus effectively limiting use to a few high volume applications which can justify the development cost. Over the past several years we have developed a simple general classification method, which we refer to as minimum description which effectively handles the above problems without the necessity of any special algorithm development.

Minimum Description Analysis

The approach can be shown to be closely related to a Bayesian maximum likelihood classification and usually produces results in good agreement with those of human inspectors.

The system is trained much like a human by simply showing it examples of the various classes of interest from which it builds the reference histograms. Training time is typically of the order of a few seconds. The system can then be placed in classification mode where it performs the inspection and reports the results with little or no human intervention.

This classification method is now being successfully used in a wide variety of inspection and process control applications the electronic, automotive, food products and recycling industries. The results generally coincide well with those made by a human inspector.

Muffin Doneness Classification

Peanut Color Matching

For the first tests we poured approximately 200g of each sample into a 10x10cm rectangular plastic box. We then trained WAY-2C using four different images of each of samples #65 and #53. Between acquisition of each image the sample was rotated. At least once during the process it was mixed and then gently shaken to level it. Training time for the complete set was under 3 minutes.

When the training was complete we repeatedly classified each of the three samples, rotating, shaking, and stirring them from time to time to determine the effect of these actions.

The second set of tests used moving samples. For this set we used a rotating turntable containing approximately 500g of peanuts in a 22cm diameter pie plate. The training and inspection area was the same size as that used in the stationary tests.

Because of the color variations at scales of several inches, even though each of our measurements sampled an area of about 60 square centimeters, fluctuations were still large enough for occasional regions in sample #53 to most resemble sample #65 and vice versa.

In spite of the occasional fluctuations, the results of all the tests consistently indicated that the surface color distribution of the unknown sample more closely resembled that of sample #53 than sample #65.

After hearing that laboratory experiments by the sample provider based on colorimetry found the "unknown" sample more closely resembled Sample #65 we repeated the original tests, did some careful quantitative interpolation, and even had seven different unbiased human observers visually rank the samples in order of color under a variety of artificial and natural lighting conditions. All of these agreed with our original results: the "unknown" sample most resembles Sample #53. In other words the minimum description approach was in agreement with the human classification and in disagreement with a method which utilized the mean color of the peanuts.

Pizza Mapping

The third test demonstrates the use of the minimum description method for image interpretation. To map the pizza, the system was first trained by building reference histograms from representative regions of cheese, mushroom and tomato sauce. It was then instructed to classify this image at a 4x4 pixel resolution. The resulting map is shown. The derived statistics for surface exposure of the constituents are 67 percent cheese, 21 percent mushroom and 12 percent tomato sauce.

Conclusions

References