High speed target acquisition path planning for underwater unmanned vehicles based on deep reinforcement learning
PANG Zhou-qi
HAO Cheng-peng
LIN Xiao-bo
PAN Guang-shuai
Abstract：There are many challenges in high-speed underwater target acquisition.On the one hand,sonar detection data is delayed and uncertain due to the changeable underwater environment,which makes high-precision target acquisition tasks full of challenges;On the other hand,the intercepting vehicle is unable to capture in a pursuit attitude due to the high speed of the target,greatly reducing the number of interceptable trajectories.Based on this,this article proposed an improved twin delayed deep deterministic policy gradient algorithm(ITD3)to improve the acquisition efficiency and accuracy.Firstly,based on the dynamics of the intercepting vehicle,this paper proposed a"planner-controller"cascaded simulation method,which was more accurate than pure kinematic simulation and more in line with the actual situation compared to the IGC model;Secondly,in order to solve the problems of large action space and delayed underwater sensors,this paper proposed an action mask mechanism and exploring noise based on delayed messages;Thirdly,in order to make the reward function fit the characteristics of high-speed target acquisition task,this paper designed a new reward function to punish states which were not conducive to capture;Finally,in order to improve the convergence speed and stability of the algorithm,this paper combined priority experience replay and softmax operator with the TD3 algorithm.Simulation experiments and hardware-in-the-loop simulations showed that compared with traditional acquisition algorithms,the feasible ITD3 algorithm proposed in this paper had a shorter interception time and a lower miss rate.
Keywords：deep reinforcement learningdeterministic policy gradienthigh speed target acquisitionunderwater un-manned vehicleMarkov decision process
Publication Date：2025-10-30
Online Publishing Date：2025-11-13(First online date of this platform, not the publication date of the document)
Pages：13( 1968-1980 )