Theoretical Q2.1

Theoretical Q2.1

Vered Zilberstein 26 Mar 2019 11:03

We are asked to build an MDP.
On the one hand the hint suggests that we "may incorporate the random d" (include it as part of the state space?)
On the other hand the random d is not a part of the MDP i.e. the MDP should only care which slots are empty/occupied.
What should we do?

Re: Theoretical Q2.1

Lee Cohen 26 Mar 2019 15:24

We always wish to minimize the number of states (that contains enough information for the optimal policy),
since the computation complexity of the Value Iteration depends on it.
You have to incorporate the randomness of the current new digit d in the states since the (optimal) policy should consider it.