Research progress in globular fruit picking recognition algorithm based on deep learning

LI Hui; ZHANG Jun; YU Shuochen; LI Zhixin

doi:10.13925/j.cnki.gsxb.20240309

Abstract：China is a global leader in fruit production,and fruit picking mainly relies on manual labor,which helps to select fruits according to fruit size and quality to reduce loss in this way.Different tech-niques and tools can be adopted according to the characteristics and picking needs of each fruit crop.However,the present picking field is faced with the problem of decreasing human resources and aging problem.Meanwhile,the traditional manual picking method has become unable to meet the demand for fast and efficient picking.To solve the problem of labor shortage,the research and development of auto-mated fruit picking equipment with integrated computer vision have become the key to solve the prob-lem of labor shortage.It can effectively improve the efficiency and quality of fruit picking.Automatic picking equipment combined with computer vision often uses object detection algorithms to identify ob-jects,and object detection algorithms can be divided into both traditional algorithms and deep learning-based object detection algorithms.Traditional algorithms identify the position and bounding box of a specific object in an image or a video,usually by preprocessing the image(Scaling,grayscale or normal-ization),feature extraction(using traditional hand-designed features or automatic learning based on ma-chine learning),classification or regression(confirming object class and location),and non-maximum suppression to further optimize and filter detected objects.When traditional fruit detection algorithms process images in complex environments,their limited expression ability and robustness are easily af-fected by illumination,occlusion and other factors,resulting in a decline in recognition accuracy.Fur-thermore,with the increase of feature complexity and computation amount,the algorithm processing speed will be reduced.When changing scenes,adding fruit types and updating features,the feature ex-tractor needs to be redesigned and adjusted,and in special cases,the entire system needs to be retrained.Compared with traditional fruit detection algorithms,the fruit detection algorithm based on deep learn-ing can extract and learn rich features from a large amount of data,and has higher accuracy and robust-ness when processing noisy data.When changing new environments and adding new categories,the fruit detection algorithm based on deep learning can improve the recognition ability and recognition ac-curacy of the model through transfer learning,data enhancement,multi-model combination,feature fu-sion and multi-modal data.Fruit detection algorithms based on deep learning can be divided into two categories:one-stage target detection algorithm and two-stage target detection algorithm.The one-stage object detection algorithm achieves end-to-end detection by using a single convolutional neural network to directly predict the target location and category.This method achieves fast detection while maintain-ing high accuracy,transforms the problem of target detection into a regression problem,and completes the location and classification of the target directly.In the training and deployment phase of the algo-rithm,the first-stage object detection algorithm uses pruning and quantization techniques to reduce the model size,which is suitable for running in mobile devices or embedded systems with limited resourc-es.The two-stage target detection algorithm is called the target detection algorithm based on region of interest or region suggestion,which is usually divided into two stages:1)Generate a large number of candidate regions by selective search,regional suggestion network(RPN)and other methods;2)Through the network processing including classifiers and boundary box regressors,the candidate region is identified and accurately located.Traditional algorithms are effective in simple scenarios,but are of-ten limited by design features in complex environments.Algorithms based on deep learning are more suitable for automated fruit picking due to their high efficiency and accuracy.This paper summarizes the improvement and application of traditional object detection algorithm and deep learning-based ob-ject detection algorithm.Also,this paper summarizes the improvements and applications of traditional spherical fruit detection algorithms and deep learning-based spherical fruit detection algorithms,and an-alyzes the advantages and disadvantages of these algorithms in different use scenarios.This paper sum-marizes the fruit picking recognition algorithm and puts forward the future development trend of the al-gorithm.With model optimization and lightweight as the starting point,the efficient network architec-ture or model compression technology is adopted to reduce computational complexity and model size,improve model processing speed and adapt to mobile automatic picking equipment.It is required to en-hance data processing,improve model generalization by preprocessing and synthesizing data,and opti-mize model adaptability in changing environments.The accuracy and robustness of model recognition are improved by combining spectral,infrared,laser and other sensor data.The model adaptive adjust-ment algorithm should be developed to adjust strategies and parameters according to real-time feedback and adapt to different fruit picking operations and different picking environmental conditions.In the fruit picking recognition algorithm based on deep learning,YOLO can directly predict the boundary box and category probability of the target in a single forward propagation to achieve near real-time de-tection,which is very important for the fruit picking robot in the orchard that needs fast response.The end-to-end design of YOLO simplifies the training inspection process,reduces complexity,and enables faster deployment in picking robot systems.In the changeable environment of orchards and groves,YO-LO can effectively distinguish between fruit and background,improving the accuracy of detection.With the continuous research by domestic scholars,YOLO algorithm is also continuously iteratively opti-mized,and its ability to detect the objects of different sizes and shapes is significantly improved,which can adapt to the maturity degree,size and occlusion of fruits,and improve the detection performance in complex environments.