Memory-guided learning and decision-making of agent:A perspective from memory replay of hippocampus

ZHU Jin-biao; WU Yi-fan; WANG Dong-shu

doi:10.7641/CTA.2023.20709

Abstract：Memory replay plays an important role in improving learning and decision-making ability of organisms.Studies have shown that biology memory playback is mainly conducted by place cells in the hippocampus,on the playback activation sequence and specific activation positions diversity.Unfortunately,most of the existing researches of simulated hippocampus replay have single forms and only the replay in one direction or part of the case are simulated,which is difficult to well reproduce the hippocampus memory replay mechanism.Therefore,combining the memory playback mechanism of organisms,it is of great research value and application prospects to simulate and realize the memory playback of the hippocampal place cells,to improve the learning and decision-making performance of agents.For the static grid scenario,a combined reinforcement learning mechanism is used to simulate the diversity of the hippocampal reactivation.In this work,a bi-directional search model is designed to simulate the memory reactivation at different locations in the hippocampus by alternate use of the trajectory sampling and priority sweeping.Meanwhile,online and off-line learning is used to simulate the memory mechanism of the organism in awake and sleep statues respectively,so as to better reproduce the memory playback process of the hippocampus.Furthermore,a deep bi-directional search model with the function of"one set of parameters and two updates"is designed to enhance the learning and decision-making performance of agents in dynamic environments.Finally,agent navigation experiments in complex static and dynamic grid environments and performance comparison experiments with other reinforcement learning algorithms verify the effectiveness of the proposed model.