Traditional machine vision is based on analytical, rule-based algorithms to detect and parameterize errors that can be mathematically defined. In such applications, highly skilled developers and engineers evaluate each potential problem, apply a set of rules to accomplish the task, and then program the system. To simplify this process, many vendors have developed low-code and no-code solutions that make it easier to match pattern recognition, blob, edge or other vision tools to the requirements of the application. However, rule-based image processing reaches its limits when defects are difficult to define numerically or their appearance varies greatly. This makes the constant further development and maintenance of rule-based image processing applications, which is often unavoidable due to changes to parts and packaging, new components and modified processes, a challenge. Process changes result, for example, from variations in raw materials or components from different suppliers, technological advances or changes in lighting conditions in the production environment.

Development of deep learning

A decade ago, deep learning, a subcategory of machine learning and therefore artificial intelligence, was reserved for companies with specialized experts and large budgets. However, advances in theory, computer hardware and data availability have meant that this technology is now also being used in industrial image processing applications. Deep learning is particularly suitable for two areas:

• Situations where subjective decisions need to be made, such as those requiring human inspectors.
• Cluttered scenes where the identification of certain features in the image is difficult due to high complexity or extreme variability, for example in the presence of high background noise.

Unlike rule-based machine vision, where the development of new algorithms depends on experts, deep learning relies on the ability of operators, production line managers and other professionals to evaluate images as good or bad and classify the type of defects present in an image. This approach eliminates the need for highly skilled vision specialists and reduces the amount of technical staff required to deploy and maintain applications. When changes occur, the model can be easily retrained by capturing and labeling new images.

Challenges with deep learning

Deep learning toolkits can therefore make AI-based vision systems easier to use, but there are still obstacles. For example, most successful deep learning projects still require large budgets and specialized expertise from vision engineers and data specialists to initially set up the system. However, not all projects lead to a sufficient increase in efficiency to justify this investment.

As with any vision application, the image acquisition hardware plays a crucial role in the success of a deep learning solution. Image acquisition requires a well-designed imaging system. Reliable and repeatable imaging processes must be able to clearly distinguish features or objects of interest. The representation of parts, illumination methods and image resolution play an important role in recognizing the subtleties that allow differentiation into different classifications. And the processing used for image analysis must be robust and powerful enough to keep up with typical production rates and meet algorithmic requirements.

On the software side, model development can take a long time and require labeling hundreds or thousands of images. In addition, it can be difficult to obtain images of defective objects. This is especially true for prototype production lines that only produce a small number of parts, as well as consumer electronics and mobile device manufacturing, which often have very short production runs of one year or less. Such situations require frequent iterations. In addition, highly automated production lines usually produce good parts with few defects. Therefore, it can take several months to obtain a large enough sample to create a reliable model.

Closing the gap

Many applications are too complex for a rule-based vision solution, but do not justify the time and resources required to develop a full-fledged deep learning solution. To bridge this gap in vision application coverage between traditional rule-based and deep learning-based machine vision, hardware manufacturers have developed Edge AI that runs on top of smart camera software platforms.

This type of deep learning, also known as 'edge learning', uses a collection of pre-existing algorithms that enable model training and subsequent image analysis directly on the device. Edge learning is a machine learning approach that is specifically tailored to industrial automation. Training takes place in two steps: Pre-training is followed by training for specific use cases.

The first step is carried out by the edge learning provider using a large data set that is optimized for industrial automation. The pre-trained tool is then integrated into a smart camera and delivered to the customer, who carries out the second part of the training for their specific use case. This approach enables faster training that requires only a few images and does not require a computer or GPU. Setting up and capturing images also takes less time because smart cameras combine multiple elements such as sensor, optics, processor and sometimes even lighting. This approach reduces hardware integration issues, such as cabling to a PC and integrating the inference engine.