Adaptive signal control at a single intersection based on improved GBML-DQN algorithm
LI Qian
ZHOU Wencai
LI Yongqi
Abstract：To address the insufficient adaptability of fixed signal timing in dynamic traffic flow environ-ments,this study proposes an improved Gradient-Based Meta-Learning Deep Q-Network(GBML-DQN)algorithm integrating a meta-learning mechanism for adaptive signal control at isolated intersec-tions.First,the state space is constructed based on lane density,the signal phase action space is de-fined,and a multi-objective weighted reward function is designed.Second,using DQN as the base ar-chitecture,the discount factor γ is dynamically adjusted via a meta-gradient strategy.Third,a Dueling structure is used to decouple state values from action advantages,and NoisyNet is employed to replace the traditional ε-greedy strategy.Consequently,two improved algorithms are constructed:Improved GBML-DQN-Dueling(I-GD-D)and Improved GBML-DQN-Noisy(I-GD-N).Finally,experimen-tal validation is conducted on the SUMO simulation platform across high,medium,and low traffic vol-ume scenarios.The experimental results indicate that I-GD-N exhibits superior robustness and adapt-ability across different traffic scenarios.Specifically,under medium traffic conditions using the Sto-chastic Gradient Descent(SGD)optimizer,the average delay is reduced to 20.55 s,representing an improvement of approximately 20%compared to DQN.While DQN exhibits higher stability than GBML-DQN in medium and low traffic scenarios due to its simpler structure,it suffers from signifi-cant policy degradation in high traffic scenarios or when using the Root Mean Square Propagation(RM-Sprop)optimizer,performing worse than even fixed signal timing.The dynamic γ adjustment mecha-nism adaptively optimizes based on traffic intensity;in high traffic scenarios,it significantly outper-forms fixed γ strategies by reducing γ to rapidly respond to congestion.The research results provide a valuable reference for urban intersection signal control.
Keywords：traffic signal controlreinforcement learningintelligent transportationSUMOIm-proved GBML-DQN
Publication Date：2025-12-30
Online Publishing Date：2026-02-02(First online date of this platform, not the publication date of the document)
Pages：16( 110-125 )