Question 1. Consider the discrete-time dynamical system xk+1=xk+uk+wk,k=0,1,2,3, where the
state xk, the control uk, and the random parameter wk are all integers. The initial state is x0=5,
and the cost function is k=03(xk2+uk2) Apply the DP algorithm (show steps) and compute the
optimal policy (minimize the expected cost function) for the following case. For k=0,1,2,3, - State
space: Sk={0,1,2,3,4,5} - Control constraint set: Uk(xk)={u0xk+u5,u : integer } - If 0<xk+uk<5, wk={
11withprobability1/2withprobability1/2 otherwise (i.e. when xk+uk=0 or xk+uk=5)wk=0 with
probability 1..
Question 1 Consider the discretetime dynamical system xk+1.pdf
1. Question 1. Consider the discrete-time dynamical system xk+1=xk+uk+wk,k=0,1,2,3, where the
state xk, the control uk, and the random parameter wk are all integers. The initial state is x0=5,
and the cost function is k=03(xk2+uk2) Apply the DP algorithm (show steps) and compute the
optimal policy (minimize the expected cost function) for the following case. For k=0,1,2,3, - State
space: Sk={0,1,2,3,4,5} - Control constraint set: Uk(xk)={u0xk+u5,u : integer } - If 0<xk+uk<5, wk={
11withprobability1/2withprobability1/2 otherwise (i.e. when xk+uk=0 or xk+uk=5)wk=0 with
probability 1.
