Target detection technology for coal mine gripper robot based on image and laser point cloud fusion

WANG Jingyang; ZHANG Shiyuan; WANG Lianfa; JI Chunlei; DIAO Xiuqiang; YU Xiao

doi:10.11799/ce202510020

Abstract：To achieve accurate 3D target identification in underground mining operation scenarios,this study focuses on target detection and control systems for auxiliary operation robots in coal mines.First,a fusion analysis method based on 3D LiDAR and video image data was developed,achieving spatiotemporal synchronization and positional alignment between LiDAR point cloud data and image data.Subsequently,an improved YOLOv8s image target detection algorithm model,incorporating a Slim-neck feature fusion network,and a 3D point cloud target detection algorithm model based on PointPillars were designed.These models reduce complexity while maintaining recognition accuracy.Building on this,an improved DS evidence theory based on Lance distance was proposed,leading to the development of YOPilaNet-a fused target detection model integrating video images and 3D point cloud data.Experimental validation using the KITTI dataset demonstrated that the proposed YOPilaNet fusion model achieved detection accuracies of 92.12%for Cars,64.68%for Pedestrians,and 72.71%for Cyclists,significantly outperforming single-modality target detection.Furthermore,in the auxiliary pipe-handling environment of an underground coal mine,YOPilaNet accurately identified pipes,metal pipe racks,and pipe connections while maintaining stable detection performance under complex working conditions,further validating its adaptability and engineering application value.