HW2 (new threads)

couldn't connect to display "gate.tau.ac.il:0"

Sun, 07 Apr 2019 15:31:47 +0000

I recieved this error when attempting to run the code on nova remotely.
When I ran the code from Schreiber's lab and from my computer everything worked fine, what is this error?

Programming - not making in nova

Mon, 01 Apr 2019 17:36:02 +0000

Hi,
the line env = gym.make('FrozenLake-v0') does not work on the nova for us.
it runs and do not end.. (no error or something, all we know is that it gets to this line in the code and not further).

Any ideas to why it happens?

thanks

Programming Q1 - Duplicate states in mdp.P

Sun, 31 Mar 2019 18:23:27 +0000

In mdp.P, the following holds according to the comments:
for every (state,action), mdp.P[state][action] is a list of (probability,next_state,reward) tuples.
is it okay for this list to contain duplicate state, for example:
P[0][0] = [(0.3333333333333333, 0, 0.0), (0.3333333333333333, 0, 0.0), (0.3333333333333333, 4, 0.0)]

if so - what should we do with these duplicates?

Thanks

Theoretical Q2.1

Tue, 26 Mar 2019 11:03:24 +0000

We are asked to build an MDP.
On the one hand the hint suggests that we "may incorporate the random d" (include it as part of the state space?)
On the other hand the random d is not a part of the MDP i.e. the MDP should only care which slots are empty/occupied.
What should we do?

Theoretical part - Ex.2

Sun, 24 Mar 2019 18:42:36 +0000

I'm having some problems defining an MDP for this problem, as the valid actions change depending on the current state (putting a number in a taken cell is forbidden, and the contents of the cells is represented in the current state)
Can you elaborate as to how we should define an MDP in this case? Should we ignore the possibility of an invalid action?

Issues with "from gym.spaces import prng"

Wed, 20 Mar 2019 20:11:13 +0000

tl;dr: pip install gym==0.10.3

For those of you who use the latest version of gym (0.12.0), the above import will not work.
That is because openai removed it in version 0.11.0: https://github.com/openai/gym
My advice for you is to downgrade gym version to match the version installed on nova, which is 0.10.3.
See tl;dr for suggested way to downgrade.