The manual harvesting of fruit and vegetables is a labor-intensive task that is suffering from the current shortage of labor in agriculture. To prevent the labor shortage from leading to a reduced supply of fruit and vegetables, robotic alternatives are being sought. In order for a robot to successfully harvest a crop, the fruit and vegetables must be recognized using computer vision methods. To date, however, most computer vision methods have not been able to function generically and reliably when applied to different fields. This still stands in the way of the commercial success of harvesting robots.

The aim of Pieter Blok's doctoral thesis was to research and develop new image processing methods that can help a harvesting robot to deal with variability and uncertainty. Blok is a researcher in Deep Learning and Computer Vision at Wageningen University and Research in the Netherlands with a Bachelor and Master of Sciences in Agricultural Engineering from Wageningen University. He obtained his PhD from Wageningen University with the thesis 'Perception models for selective harvesting robots in fruit and vegetable production' in December 2022.

The doctoral thesis focused on three tasks that every harvesting robot must perform: recognizing the crop, estimating the size of each crop and determining the quality of the crop. In the context of crop recognition, Blok's work focused on improving the generalization performance of Convolutional Neural Networks (CNNs) to deal with variations within the same crop. Usually, there are many variations within a crop, which can result in the trained CNN not being able to generalize sufficiently. Therefore, Blok's research focused on applying different types of data augmentation to optimize the training of a CNN. With geometric data augmentation (rotation, cropping and scaling of the image), the CNN was shown to generalize better to multiple plant variations.

Another research topic in Blok's dissertation dealt with the improvement of crop size estimation. Size estimation is important for a harvesting robot because it determines whether a plant should be harvested or left in the field to continue growing. However, one challenge in estimating the size of a plant is that it can be obscured by leaves, limiting the visibility of the plant. Blok introduced a new perception method to overcome this problem. His new method utilizes amodal perception, which is the ability to predict both the visible and occluded parts of objects in an image. By integrating this method into a CNN, the amodal shape of occluded plants could be accurately estimated for three-dimensional plant size estimation and robot positioning.

The third research topic in Blok's doctoral thesis dealt with determining crop quality. Specifically, it involved the use of active learning to automatically select and note sporadically occurring plant diseases. The newly developed active learning method used the output of the CNN to select unlabeled images where the network was most uncertain. These images were then labeled interactively in a semi-supervised manner and used to retrain the network. This active learning method significantly reduced the annotation effort by 1400 image annotations or 120 hours.

The integration of Blok's deep learning technologies into five harvesting robots has enabled the successful commercialization of these robots and is an example of a successful technology transfer from science to industry.

From 1 July 2023, Dr. Blok will continue his scientific career as an assistant professor at the Laboratory of Field Phenomics at the University of Tokyo in Japan, where he will focus on machine learning and image processing technologies for plant phenotyping.

The EMVA Young Professional Award

The EMVA Young Professional Award is an annual prize awarded at the EMVA Business Conference that honors the exceptional and innovative work of students or young professionals in machine vision. With this award, the EMVA would like to encourage students in particular to focus on the technical challenges of machine vision and to apply the latest research results in machine vision to the practical requirements of industry.

The European Machine Vision Association (EMVA) is a non-profit and non-commercial association founded in 2003 to represent the machine vision industry in Europe. It is open to all organizations involved in machine vision, computer vision, embedded vision or image processing technologies: Manufacturers, system and machine builders, integrators, distributors, consulting companies, research institutes and universities.