A maximum principle for optimal control of discrete-time stochastic systems with Markov jump
LIN Xiang-yun
WANG Xin-rui
ZHANG Wei-hai
Abstract：The maximum principle(MP)of the discrete-time nonlinear stochastic optimal control problem is proved,in which the control systems are driven by both Markov jumps and multiplicative noise.Firstly,based on the adapted solutions of the backward stochastic difference equation,the linear functional with the constraint of a linear difference equation is represented.The Riesz theorem is used to prove the uniqueness of such representation.Secondly,the spike variation method is extend to the nonlinear stochastic difference equation with Markov jumps.The variation equation of such state equation is obtained.Thirdly,by introducing a Hamiltonian function,a necessary condition of the discrete-time nonlinear stochastic optimal control system with Markov jump is obtained.It is proved that the adjoint equation of the maximum principle of the system is a pair of backward stochastic difference equations.Moreover,a sufficient condition is also given and the corresponding Hamilton-Jacobi-Bellman equation is derived.Finally,a practical example is given to illustrate the practicability and feasibility of the proposed theory.
Keywords：maximum principleoptimal controlMarkov jumpbackward stochastic difference equationsHamilton-Jacobi-Bellman equations
Publication Date：2024-05-28
Online Publishing Date：2025-08-15(First online date of this platform, not the publication date of the document)
Pages：10( 895-904 )