<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wikidot="http://www.wikidot.com/rss-namespace">

	<channel>
		<title>Reinforcement  Learning - new forum posts</title>
		<link>https://rl-tau-2019.wikidot.com/forum/start</link>
		<description>Posts in forums of the site &quot;Reinforcement  Learning&quot; - Spring 2019, Tel-Aviv University</description>
				<copyright></copyright>
		<lastBuildDate>Tue, 10 Nov 2026 17:47:29 +0000</lastBuildDate>
		
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12225140#post-4306560</guid>
				<title>Exam solution</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12225140/exam-solution#post-4306560</link>
				<description></description>
				<pubDate>Mon, 15 Jul 2019 15:34:23 +0000</pubDate>
				<wikidot:authorName>guest</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hello,<br /> can you please upload a solution for the exam?</p> <p>Thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12225140/exam-solution">Exam solution</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12202168#post-4302144</guid>
				<title>Exam Moed A Solution</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12202168/exam-moed-a-solution#post-4302144</link>
				<description></description>
				<pubDate>Tue, 09 Jul 2019 18:11:51 +0000</pubDate>
				<wikidot:authorName>Guest</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hi,</p> <p>Can the Moed A exam and its solution be uploaded?</p> <p>Thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12202168/exam-moed-a-solution">Exam Moed A Solution</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12197422#post-4301281</guid>
				<title>Re: Controllability and stability</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12197422/controllability-and-stability#post-4301281</link>
				<description></description>
				<pubDate>Mon, 08 Jul 2019 17:10:47 +0000</pubDate>
				<wikidot:authorName>mansour</wikidot:authorName>				<wikidot:authorUserId>91461</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Controllability implies that we can reach any state, but does not guarantee finite cost.<br /> Stability, implies that the optimal control has finite cost, but does to guarantee that we can reach any state. (For example, we always move to x=0)<br /> We like to reach x=0, and should think that we &quot;normalize&quot; the system so that the origin (x=0) is the desired state of operation.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12197422/controllability-and-stability">Controllability and stability</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12195036#post-4301273</guid>
				<title>Re: Off/on policy evaluation in exam</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12195036/off-on-policy-evaluation-in-exam#post-4301273</link>
				<description></description>
				<pubDate>Mon, 08 Jul 2019 17:02:43 +0000</pubDate>
				<wikidot:authorName>mansour</wikidot:authorName>				<wikidot:authorUserId>91461</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>In question 2 Moed A you are asked about Q-learning, so it does not matter which policy generated them.<br /> In question 2 Moed B you are told that the trajectories are using pi, and asked to run Monte Carlo to learn V of pi.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12195036/off-on-policy-evaluation-in-exam">Off/on policy evaluation in exam</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12197949#post-4301099</guid>
				<title>Q4d moed a + b</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12197949/q4d-moed-a-b#post-4301099</link>
				<description></description>
				<pubDate>Mon, 08 Jul 2019 13:15:40 +0000</pubDate>
				<wikidot:authorName>guest</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Can someone please explain the answers for question 4d in last year's exams?<br /> Thanks!</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177834">Discussions / Past Exams</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12197949/q4d-moed-a-b">Q4d moed a + b</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12197726#post-4301044</guid>
				<title>HW4 Q1 solution</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12197726/hw4-q1-solution#post-4301044</link>
				<description></description>
				<pubDate>Mon, 08 Jul 2019 11:29:12 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Hi all,</p> <p>Question 1 in hw4 is (a slightly easier version of) question 5 here: <a href="https://ece.iisc.ac.in/~aditya/E1245_Online_Prediction_Learning_F2014/final_exam_full.pdf">https://ece.iisc.ac.in/~aditya/E1245_Online_Prediction_Learning_F2014/final_exam_full.pdf</a></p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177827">News / Course News</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12197726/hw4-q1-solution">HW4 Q1 solution</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12197422#post-4300936</guid>
				<title>Controllability and stability</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12197422/controllability-and-stability#post-4300936</link>
				<description></description>
				<pubDate>Mon, 08 Jul 2019 08:35:43 +0000</pubDate>
				<wikidot:authorName>asdf</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hi,</p> <p>In the LQR lecture we defined controllability as a sufficient condition for solving the ARE equations.<br /> Then we defined stability which basically tell us if our system will explode or not depending on the eigenvalues of the proposed optimal solution.<br /> Can someone explain how are the two related ?<br /> We can reach every state but then cannot stay there? we will try to reach it but the system will be very unstable?<br /> Also it says that a good system is a system where the eigenvalues are lower than 1 hence x_t goes to 0, why is it good?<br /> We want x_t to be a specific state and not zero.</p> <p>Thanks!</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12197422/controllability-and-stability">Controllability and stability</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12191649#post-4300407</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12191649/hw-solutions#post-4300407</link>
				<description></description>
				<pubDate>Sun, 07 Jul 2019 14:00:06 +0000</pubDate>
				<wikidot:authorName>gsdaf</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Can you post a solution from a student that received 100?</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12191649/hw-solutions">HW solutions</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12195036#post-4300406</guid>
				<title>Off/on policy evaluation in exam</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12195036/off-on-policy-evaluation-in-exam#post-4300406</link>
				<description></description>
				<pubDate>Sun, 07 Jul 2019 13:59:21 +0000</pubDate>
				<wikidot:authorName>gsdaf</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hi,</p> <p>In the exams you published there are questions that provide traces and ask us to compute the V or Q function via some method.<br /> My question is, how do we know if the traces were produced via on-policy or by off-policy?<br /> This changes dramatically the computation of the estimated Q/V function.</p> <p>Thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12195036/off-on-policy-evaluation-in-exam">Off/on policy evaluation in exam</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12194062#post-4300219</guid>
				<title>small question/clarification on recitation 5</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12194062/small-question-clarification-on-recitation-5#post-4300219</link>
				<description></description>
				<pubDate>Sun, 07 Jul 2019 07:38:46 +0000</pubDate>
				<wikidot:authorName>rafi levy</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>What does it mean &quot;s(1,1) - action is chosen at step 4 &amp; 5&quot; on page 5-6, rec. 5?</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12194062/small-question-clarification-on-recitation-5">small question/clarification on recitation 5</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12188079#post-4299731</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10#post-4299731</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 17:18:31 +0000</pubDate>
				<wikidot:authorName>rafi levy</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>but if alpha is positive but smaller than 1, wouldnt the integral diverge?</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10">recitation 10</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12191649#post-4299652</guid>
				<title>Re: HW solutions</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12191649/hw-solutions#post-4299652</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 15:27:06 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>We don't publish solutions. If there's something which is not clear to you after your exercises have been graded feel free to ask (you can send me an email if you prefer).</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12191649/hw-solutions">HW solutions</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12187375#post-4299651</guid>
				<title>Re: Recitation 8 - new weights vector, why?</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12187375/recitation-8-new-weights-vector-why#post-4299651</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 15:25:32 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>'wait' is an action, the state is a function of the chemicals' concentrations.<br /> In exercise 2 we handle the 'wait' action for the first time, so when we estimate the Q value we should consider different weights than the ones that were used for 'harvest' action.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12187375/recitation-8-new-weights-vector-why">Recitation 8 - new weights vector, why?</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12188079#post-4299650</guid>
				<title>Re: recitation 10</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10#post-4299650</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 15:24:31 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Thanks Dan.<br /> equation 22- it's not required to assume that alpha&gt;1.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10">recitation 10</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12190788#post-4299648</guid>
				<title>RE</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12190788/recitation-5-ex-1#post-4299648</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 15:24:20 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>In this question, room 5 loops back to itself with a reward of 100 (or you can go to rooms 1/4), the agent can start from room 5 as well.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12190788/recitation-5-ex-1">recitation 5, ex.1</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12190751#post-4299647</guid>
				<title>Re: UCB analysis exploration cost</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12190751/ucb-analysis-exploration-cost#post-4299647</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 15:24:09 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Technically you're right, but since sum i 1 to n of delta_i is a fixed term it doesn't increase the <strong>bound</strong> which is already logarithmic in T (i.e., O(logT+c)= O(logT) for any <span class="math-inline">$c\in \mathbb{R}$</span>)</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12190751/ucb-analysis-exploration-cost">UCB analysis exploration cost</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12191649#post-4299547</guid>
				<title>HW solutions</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12191649/hw-solutions#post-4299547</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 12:55:05 +0000</pubDate>
				<wikidot:authorName>adsavf</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Can we please get a solution to the homework.</p> <p>Thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12191649/hw-solutions">HW solutions</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12190788#post-4299405</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12190788/recitation-5-ex-1#post-4299405</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 08:37:44 +0000</pubDate>
				<wikidot:authorName>rafi levy</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>sorry, it is recitation 6</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12190788/recitation-5-ex-1">recitation 5, ex.1</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12190788#post-4299404</guid>
				<title>recitation 5, ex.1</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12190788/recitation-5-ex-1#post-4299404</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 08:36:38 +0000</pubDate>
				<wikidot:authorName>rafi levy</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Rec. 6, ex.1</p> <p>When do we update the entry Q(5,5)?<br /> Since it is the target (room 5), it seems it can be updated only when the episod is starting with that state?<br /> If it would have been stayed zero, we would have never reached Q values greater than 100</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12190788/recitation-5-ex-1">recitation 5, ex.1</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12190751#post-4299400</guid>
				<title>UCB analysis exploration cost</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12190751/ucb-analysis-exploration-cost#post-4299400</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 08:24:30 +0000</pubDate>
				<wikidot:authorName>recitation 10</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>In the analysis of the UCB bound there is an assumption that Ti is greater than 1.<br /> It holds since in the first round we start by pulling each arm one time.<br /> Shouldn't we add this to the regret?<br /> Hence the regret should have an extra term: sum i 1 to n of delta_i</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12190751/ucb-analysis-exploration-cost">UCB analysis exploration cost</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12188079#post-4299363</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10#post-4299363</link>
				<description></description>
				<pubDate>Sat, 06 Jul 2019 06:56:37 +0000</pubDate>
				<wikidot:authorName>rafi levy</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>thanks dan</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10">recitation 10</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12188079#post-4298946</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10#post-4298946</link>
				<description></description>
				<pubDate>Fri, 05 Jul 2019 16:17:04 +0000</pubDate>
				<wikidot:authorName>dan</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>regarding equation 8 - I think the usage in Hoeffding's inequality should be:</p> <span class="equation-number">(1)</span> <div class="math-equation" id="equation-499899-1">\begin{align} exp( \frac{-2 \epsilon ^2 m^2}{\sum ^m _{i=1} (b_i - a_i)^2} )=exp( \frac{-2 \epsilon ^2 m^2}{\sum ^m _{i=1} (1 - 0)^2} )=exp( \frac{-2 \epsilon ^2 m^2}{m}) \end{align}</div> <p>where <span class="math-inline">$m=T_i$</span> because we get the mean <span class="math-inline">$\hat \mu _{i,T_i}$</span> on <span class="math-inline">$T_i$</span> samples</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10">recitation 10</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12188079#post-4298783</guid>
				<title>recitation 10</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10#post-4298783</link>
				<description></description>
				<pubDate>Fri, 05 Jul 2019 12:30:10 +0000</pubDate>
				<wikidot:authorName>rafi levy</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Shalom Lee,</p> <p>In equation 8, shouldn't we write t instead of t square? indeed, in the last expression, the t square is replaced by t.</p> <p>in the explanation before (16), delta(i) = mu1 - mu2</p> <p>in equation 22, should we assume alpha is greather than 1?</p> <p>Toda,<br /> Rafi</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12188079/recitation-10">recitation 10</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12187353#post-4298600</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12187353/recitation-8-gradient-direction#post-4298600</link>
				<description></description>
				<pubDate>Fri, 05 Jul 2019 09:02:42 +0000</pubDate>
				<wikidot:authorName>Just asking</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>I take it back. I see that the two minus signs are canceled since you take f-f' and not f'-f.<br /> Thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12187353/recitation-8-gradient-direction">Recitation 8 - gradient direction</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12187375#post-4298562</guid>
				<title>Recitation 8 - new weights vector, why?</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12187375/recitation-8-new-weights-vector-why#post-4298562</link>
				<description></description>
				<pubDate>Fri, 05 Jul 2019 07:57:27 +0000</pubDate>
				<wikidot:authorName>Just asking</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>In recitation 8 in the second part we initialize a new weight vector for the state &quot;wait&quot;.<br /> The entire point of function approximation was to have a compact representation of the state space, hence if we initialize a weight vector for each state in the space<br /> then we are doomed, no?<br /> Why do we need a new weight vector? why is it reasonable?</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12187375/recitation-8-new-weights-vector-why">Recitation 8 - new weights vector, why?</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12187353#post-4298556</guid>
				<title>Recitation 8 - gradient direction</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12187353/recitation-8-gradient-direction#post-4298556</link>
				<description></description>
				<pubDate>Fri, 05 Jul 2019 07:49:37 +0000</pubDate>
				<wikidot:authorName>just asking</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Correct me if I am wrong but should the updates the weights of the Q function in the direction of the negative gradient.<br /> it seems that we add the gradient to the weights instead of subtracting it.</p> <p>Thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12187353/recitation-8-gradient-direction">Recitation 8 - gradient direction</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12185849#post-4298250</guid>
				<title>recitation 13 (exam solution) and recitation 11 minor fixes</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12185849/recitation-13-exam-solution-and-recitation-11-minor-fixes#post-4298250</link>
				<description></description>
				<pubDate>Thu, 04 Jul 2019 19:13:09 +0000</pubDate>
				<wikidot:authorName>rafi levy</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hello Lee,</p> <p>recitation 13, 13-1 part a - there is no 'return' action from state 1 to itself, index should start from 2.</p> <p>recitation 11, page 11-4 , the decreasing curve is open-left and the increasing curve is open right, the x-axis is b(s(l)) state (tiger on left door).</p> <p>page 11-5, the second equation : 0.15 + 0.7 * b(s(l))</p> <p>Toda<br /> Rafi</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12185849/recitation-13-exam-solution-and-recitation-11-minor-fixes">recitation 13 (exam solution) and recitation 11 minor fixes</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12185095#post-4298243</guid>
				<title>Re: Exam</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12185095/exam#post-4298243</link>
				<description></description>
				<pubDate>Thu, 04 Jul 2019 18:58:36 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Same as last year- you can bring <strong>one</strong> double-sided A4 sheet and a calculator.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12185095/exam">Exam</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12185095#post-4298069</guid>
				<title>Exam</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12185095/exam#post-4298069</link>
				<description></description>
				<pubDate>Thu, 04 Jul 2019 13:49:01 +0000</pubDate>
				<wikidot:authorName>Just asking</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hi,</p> <p>For those of us that did not attend the last class/recitation is there any special remarks we need to know?<br /> Can we bring formula pages ?</p> <p>Any information would be great,<br /> Thanks!</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12185095/exam">Exam</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12184037#post-4297825</guid>
				<title>minor fixes recitation 7</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12184037/minor-fixes-recitation-7#post-4297825</link>
				<description></description>
				<pubDate>Thu, 04 Jul 2019 07:24:15 +0000</pubDate>
				<wikidot:authorName>rafi levy</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Shalom Lee,</p> <p>in page 1-1, 1-2, 1-5 - mainly a matter of notation and being consistent with the class, the index of R/r should start from t and not from t + 1</p> <p>in page 1-3, last equation at the bottom, in the second expression there is i believe a missing gamma</p> <p>1-4, 1-5, perhaps it's worth mentioning as a footnote or something&#8230; V(lambda, t) is the same as G (lambda, t) (as presented in the first page) and V(n, t) is the same as G(n, t)</p> <p>thanks<br /> Rafi</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12184037/minor-fixes-recitation-7">minor fixes recitation 7</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12174112#post-4297081</guid>
				<title>Re: minor fixes in recitation 9</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12174112/minor-fixes-in-recitation-9#post-4297081</link>
				<description></description>
				<pubDate>Wed, 03 Jul 2019 09:17:32 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Fixed, thanks!</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12174112/minor-fixes-in-recitation-9">minor fixes in recitation 9</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12171007#post-4297072</guid>
				<title>Re: Lecture 9 - Finite Differences Methods</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12171007/lecture-9-finite-differences-methods#post-4297072</link>
				<description></description>
				<pubDate>Wed, 03 Jul 2019 08:57:41 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>We use the estimations, <span class="math-inline">$\hat{J}(\theta+\delta e_i)$</span> that are obtained by averaging samples.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12171007/lecture-9-finite-differences-methods">Lecture 9 - Finite Differences Methods</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12174112#post-4296610</guid>
				<title>minor fixes in recitation 9</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12174112/minor-fixes-in-recitation-9#post-4296610</link>
				<description></description>
				<pubDate>Tue, 02 Jul 2019 15:39:40 +0000</pubDate>
				<wikidot:authorName>rafi levy</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hi Lee,</p> <p>in 5-6 , mu is a fuction of teta and not of sigma ; in the second gradient, it should be written deviation (sigma) and not average</p> <p>thanks<br /> Rafi</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12174112/minor-fixes-in-recitation-9">minor fixes in recitation 9</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12171007#post-4296235</guid>
				<title>Lecture 9 - Finite Differences Methods</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12171007/lecture-9-finite-differences-methods#post-4296235</link>
				<description></description>
				<pubDate>Tue, 02 Jul 2019 04:21:28 +0000</pubDate>
				<wikidot:authorName>guest01</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hi,<br /> In the part about finite differences methods (scribe 9, page 4) it is said in the last paragraph that we do not have the value of J(theta).<br /> Can you please explain why it is possible to obtain the values of z which are J(theta + delta * u_i) but not the value of J(theta)?</p> <p>Thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12171007/lecture-9-finite-differences-methods">Lecture 9 - Finite Differences Methods</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12122817#post-4290110</guid>
				<title>Re: Policy iteration</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12122817/policy-iteration#post-4290110</link>
				<description></description>
				<pubDate>Mon, 24 Jun 2019 18:04:51 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Each iteration of the policy iteration has three parts:<br /> 1. (explicit) Policy Evaluation of <span class="math-inline">$V^{\pi_k}$</span> by using the formula <span class="math-inline">$V^{\pi_k}=(I-\gamma P^{\pi_k})^{-1}r^{\pi_k}$</span><br /> - where the main component here is to calculate the inverse which takes <span class="math-inline">$O(|S|^3)$</span> for a <span class="math-inline">$|S|\times|S|$</span> matrix (e.g. by Gauss–Jordan elimination).<br /> 2. Policy Improvement: per each state s (<span class="math-inline">$|S|$</span>),<br /> - we evaluate <span class="math-inline">$\pi_{k+1}(s)$</span> by choosing the action (<span class="math-inline">$|A|$</span>) the maximizes the bellman equation with <span class="math-inline">$V^{\pi_k}$</span>, which requires to go through every next state s' (<span class="math-inline">$|S|$</span>).<br /> - overall <span class="math-inline">$O(|A||S|^2)$</span>.<br /> 3. comparing <span class="math-inline">$V^{\pi_k}$</span> to <span class="math-inline">$V^{\pi_k}$</span> (and stop if they're equal)- which takes <span class="math-inline">$O(|S|)$</span></p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12122817/policy-iteration">Policy iteration</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-12122817#post-4288173</guid>
				<title>Policy iteration</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-12122817/policy-iteration#post-4288173</link>
				<description></description>
				<pubDate>Sat, 22 Jun 2019 12:48:22 +0000</pubDate>
				<wikidot:authorName>Amir</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hi,<br /> In lecture 4 - slide 57 it is said that policy iteration takes O(|A|*|S|^2 + |S|^3).<br /> Can you please explain why that is the case? what is the step that take |A|*|S|^2 and what is the part that takes |S|^3?</p> <p>Thanks!</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-12122817/policy-iteration">Policy iteration</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11658238#post-4260690</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues#post-4260690</link>
				<description></description>
				<pubDate>Sat, 25 May 2019 13:56:24 +0000</pubDate>
				<wikidot:authorName>Stav Bar-Sheshet</wikidot:authorName>				<wikidot:authorUserId>5254763</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Hi,</p> <p>About this issue, we've finished our project and now can't run it on NOVA.<br /> We see that all the project files and gym env is suitable for python 3.5 when the NOVA works on 2.7.<br /> In addition, we tried to open a virtualenv on NOVA using this link:<br /> <a href="https://www.cs.tau.ac.il/system/node/289">https://www.cs.tau.ac.il/system/node/289</a><br /> But still, only python 2.7 and 3.6 are available and gym 0.9.5 doesn't work.</p> <p>Are there any specific instructions on how to submit the project to work with NOVA like there were for the HW?</p> <p>Thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues">final project running issues</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11872026#post-4253228</guid>
				<title>Re: Alternative ways to work with GPU</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11872026/alternative-ways-to-work-with-gpu#post-4253228</link>
				<description></description>
				<pubDate>Sat, 18 May 2019 13:04:29 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Thank you, Vered.</p> <p>Normally you should have 1GB in your HOME_DIR.<br /> If you need more quota you can open a ticket here:<br /> <a href="https://www.cs.tau.ac.il/system/node/294">https://www.cs.tau.ac.il/system/node/294</a></p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11872026/alternative-ways-to-work-with-gpu">Alternative ways to work with GPU</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11872026#post-4253217</guid>
				<title>Alternative ways to work with GPU</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11872026/alternative-ways-to-work-with-gpu#post-4253217</link>
				<description></description>
				<pubDate>Sat, 18 May 2019 12:56:47 +0000</pubDate>
				<wikidot:authorName>Vered Zilberstein</wikidot:authorName>				<wikidot:authorUserId>5248283</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Hi all,<br /> I have 2 suggested ways for you:</p> <p><span style="text-decoration: underline;">1. Using google colab</span><br /> You can copy my colab notebook: <a href="http://www.example.com">https://colab.research.google.com/drive/1JRS6xTvBKGL74mJP6wyj1RlBhU7-4A9O</a> and transfer your code to the new notebook.<br /> It requires adjusting the code and takes some babysitting (it only lasts 12H) but it's free available GPU to get you started quickly.</p> <p><span style="text-decoration: underline;">2. Using virtual environment on another server</span><br /> Some of you may have access to more storage on the university's machine, this is for you (since it takes about 1.5gb and regular disk quota is 1gb).<br /> Just follow the instructions below:</p> <ul> <li>ssh savant/rack-gamir-g04/5/6 (from nova)</li> <li>bash</li> <li>cd &lt;directory for your env&gt;</li> <li># create the virtual environment once</li> <li>virtualenv -p /usr/local/lib/anaconda3-5.1.0/bin/python dqn_env</li> <li># NOTE: you need to activate venv each time before running</li> <li>source dqn_env/bin/activate</li> </ul> <ul> <li># install all of the requirements</li> <li>pip install &quot;gym[atari]&quot;==0.9.5</li> <li>pip install opencv-python</li> <li>pip install torch</li> <li>pip install matplotlib</li> </ul> <ul> <li># run project from within the environment</li> <li>python main.py</li> </ul> <ul> <li># sanity check from within the environment</li> <li>python</li> <li>import gym</li> <li>print(gym.<span style="text-decoration: underline;">version</span>) # make sure it says 0.9.5</li> <li>gym.make('PongNoFrameskip-v4') # make sure this command succeeds</li> </ul> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11872026/alternative-ways-to-work-with-gpu">Alternative ways to work with GPU</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11842786#post-4252297</guid>
				<title>Re: Failed running final project on university&#039;s server</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11842786/failed-running-final-project-on-university-s-server#post-4252297</link>
				<description></description>
				<pubDate>Fri, 17 May 2019 16:24:33 +0000</pubDate>
				<wikidot:authorName>Vered Zilberstein</wikidot:authorName>				<wikidot:authorUserId>5248283</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>You are not supposed to use 'pip install' on a public machine (this means installing the package for everyone).<br /> Use 'pip install' only as a sudoer on your own machine or within a virtual environment.<br /> Can you give an example of an error?</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11842786/failed-running-final-project-on-university-s-server">Failed running final project on university's server</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11842786#post-4251832</guid>
				<title>Failed running final project on university&#039;s server</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11842786/failed-running-final-project-on-university-s-server#post-4251832</link>
				<description></description>
				<pubDate>Fri, 17 May 2019 09:58:52 +0000</pubDate>
				<wikidot:authorName>lior</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Me and other students have encountered various problems running the main.py and ram.py on the university's servers. We have seen that on another thread you mentioned that the pip install command is not suitable for these servers (rack-gamir-g0[4,5,6] and savant), thus we would appreciate it if you could specify what commands we should run on these servers in order to run the project.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11842786/failed-running-final-project-on-university-s-server">Failed running final project on university's server</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11714391#post-4251246</guid>
				<title>Re: Final Project ffmpeg</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg#post-4251246</link>
				<description></description>
				<pubDate>Thu, 16 May 2019 17:30:19 +0000</pubDate>
				<wikidot:authorName>Vered Zilberstein</wikidot:authorName>				<wikidot:authorUserId>5248283</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>You can either follow the instruction here: (this worked for me)<br /> <a href="http://www.example.com">http://blog.gregzaal.com/how-to-install-ffmpeg-on-windows/</a><br /> Or install via conda: 'conda install -c conda-forge ffmpeg'</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg">Final Project ffmpeg</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11658238#post-4249677</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues#post-4249677</link>
				<description></description>
				<pubDate>Wed, 15 May 2019 11:04:29 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Since you're using university machines you can't use pip.<br /> Just enter python (anaconda) interpreter and do 'import gym'.<br /> You can verify gym version by 'print(gym.<span style="text-decoration: underline;">version</span>)'</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues">final project running issues</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11658238#post-4249640</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues#post-4249640</link>
				<description></description>
				<pubDate>Wed, 15 May 2019 09:34:20 +0000</pubDate>
				<wikidot:authorName>sapir</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>i tried to run different commands on different machines (savant and rack-gamir-g04/05/06) but it didn't work..<br /> perhaps i dont run it correctly</p> <p>what is the exact pip command we should use?<br /> and what is the exact command to run main.py?</p> <p>thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues">final project running issues</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11714391#post-4249579</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg#post-4249579</link>
				<description></description>
				<pubDate>Wed, 15 May 2019 08:05:27 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>As I've said in the beginning of the semester, the project was designed for linux,<br /> you can run it however you like as long as in the end, it is running on nova.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg">Final Project ffmpeg</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11658238#post-4249577</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues#post-4249577</link>
				<description></description>
				<pubDate>Wed, 15 May 2019 08:01:45 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Try again, perhaps on a different machine.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues">final project running issues</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11714391#post-4247241</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg#post-4247241</link>
				<description></description>
				<pubDate>Mon, 13 May 2019 10:15:09 +0000</pubDate>
				<wikidot:authorName>Amit</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Hi, Thanks.<br /> That means that the project cannot be done on windows?</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg">Final Project ffmpeg</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11714391#post-4247199</guid>
				<title>Re: Final Project ffmpeg</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg#post-4247199</link>
				<description></description>
				<pubDate>Mon, 13 May 2019 08:01:37 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>I recommend using Linux, you can use the university machines.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg">Final Project ffmpeg</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11658238#post-4246833</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues#post-4246833</link>
				<description></description>
				<pubDate>Sun, 12 May 2019 22:26:04 +0000</pubDate>
				<wikidot:authorName>sapir</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>thanks vered but it doesn't work for me.. :(<br /> in Nova, what is the exact pip command we should use?</p> <p>also, how should we run main.py in nova?<br /> I tried: /usr/local/lib/anaconda3-5.1.0/bin/python main.py</p> <p>which fails with:<br /> Traceback (most recent call last):<br /> File &quot;main.py&quot;, line 51, in &lt;module&gt;<br /> benchmark = gym.benchmark_spec('Atari40M')<br /> AttributeError: module 'gym' has no attribute 'benchmark_spec'</p> <p>thanks again</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues">final project running issues</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11714391#post-4246643</guid>
				<title>Final Project ffmpeg</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg#post-4246643</link>
				<description></description>
				<pubDate>Sun, 12 May 2019 16:57:17 +0000</pubDate>
				<wikidot:authorName>Amit</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>I am having problems with the ffmpeg (the atari does not recognize it).<br /> You wrote in the project that to install ffmpeg we can use homebrew or apt-get.<br /> the problem is that I use windows and both of those options (homebrew and apat-get) are only for linux (If I understood correctly).</p> <p>I tried unsuccessfully to install ffmpeg from some other places I found online.<br /> is there an option not to use ffmpeg? what would you recommend to do in my case with windows?</p> <p>Thanks</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11714391/final-project-ffmpeg">Final Project ffmpeg</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11658238#post-4244631</guid>
				<title>Re: final project running issues</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues#post-4244631</link>
				<description></description>
				<pubDate>Fri, 10 May 2019 16:05:32 +0000</pubDate>
				<wikidot:authorName>Vered Zilberstein</wikidot:authorName>				<wikidot:authorUserId>5248283</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>try running:<br /> pip install &#8212;no-index -f <a href="https://github.com/Kojoley/atari-py/releases">https://github.com/Kojoley/atari-py/releases</a> atari_py<br /> (don't forget to ensure pip runs with your desired python version)</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues">final project running issues</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11658238#post-4244619</guid>
				<title>Re: final project running issues</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues#post-4244619</link>
				<description></description>
				<pubDate>Fri, 10 May 2019 16:00:02 +0000</pubDate>
				<wikidot:authorName>Vered Zilberstein</wikidot:authorName>				<wikidot:authorUserId>5248283</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>You can do: 'pip &#8212;version' to see which python is being used and make sure it's the correct version.<br /> Are you installing in a virtualenv? Are you running locally or on one of the machines we were given? Windows/Linux?<br /> Maybe this will help: <a href="http://www.example.com">https://github.com/openai/gym#atari</a></p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues">final project running issues</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11658238#post-4244516</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues#post-4244516</link>
				<description></description>
				<pubDate>Fri, 10 May 2019 14:30:59 +0000</pubDate>
				<wikidot:authorName>sapir</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>רק אסכם בגדול:<br /> 1. לא הצלחנו להריץ את ה-pip install כפי שרשום בהוראות של התרגיל.<br /> 2. כנראה בגלל זה, אבל לא הצלחנו להריץ את ה-main.py וה-ram.py שנופלים בשלב הכנת הסביבה, בשורות:<br /> gym.make('Pong-ram-v0)<br /> benchmark = gym.benchmark_spec('Atari40M')</p> <p>3. לפי מה שאני מבינה התכנית אמורה לרוץ גם ללא שימוש ב-gpu, אבל בכל מקרה הוספתי את תמונת מצב ה-gpu</p> <p>שוב תודה!</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues">final project running issues</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11658238#post-4244503</guid>
				<title>final project running issues</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues#post-4244503</link>
				<description></description>
				<pubDate>Fri, 10 May 2019 14:19:43 +0000</pubDate>
				<wikidot:authorName>sspir</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>1. we tried to run pip install atari in a 3 different ways. It seems like we are having permission issues (although i might be wrong) when it comes to python3's pip.<br /> pip install &quot;gym[atari]&quot;==0.9.5<br /> this one probably fails since it's python2's pip</p> <p>pip3 install &quot;gym[atari]&quot;==0.9.5<br /> here the failure is:</p> <p>Exception:<br /> Traceback (most recent call last):<br /> File &quot;/usr/lib/python3.4/shutil.py&quot;, line 522, in move<br /> os.rename(src, real_dst)<br /> OSError: [Errno 18] Invalid cross-device link: '/usr/local/lib/python3.4/dist-packages/gym-0.9.1-py3.4.egg' -&gt; '/tmp/pip-25pnwkn1-uninstall/usr/local/lib/python3.4/dist-packages/gym-0.9.1-py3.4.egg'</p> <p>/usr/local/lib/anaconda3-5.1.0/bin/pip install &quot;gym[atari]&quot;==0.9.5<br /> here the failure is:</p> <p>Installing collected packages: atari-py, PyOpenGL, gym<br /> Exception:<br /> Traceback (most recent call last):<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pip/basecommand.py&quot;, line 215, in main<br /> status = self.run(options, args)<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pip/commands/install.py&quot;, line 342, in run<br /> prefix=options.prefix_path,<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pip/req/req_set.py&quot;, line 784, in install<br /> **kwargs<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pip/req/req_install.py&quot;, line 851, in install<br /> self.move_wheel_files(self.source_dir, root=root, prefix=prefix)<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pip/req/req_install.py&quot;, line 1064, in move_wheel_files<br /> isolated=self.isolated,<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pip/wheel.py&quot;, line 345, in move_wheel_files<br /> clobber(source, lib_dir, True)<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pip/wheel.py&quot;, line 316, in clobber<br /> ensure_dir(destdir)<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pip/utils/<span style="text-decoration: underline;">init</span>.py&quot;, line 83, in ensure_dir<br /> os.makedirs(path)<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/os.py&quot;, line 220, in makedirs<br /> mkdir(name, mode)<br /> OSError: [Errno 30] Read-only file system: '/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/atari_py'</p> <p>2. maybe because of the above, we are having trouble with running ram.py and main.py, they both fail in the stage of creating the environment, before it begins running our actual code:</p> <p>rack-gamir-g06&#160;14% /usr/local/lib/anaconda3-5.1.0/bin/python ram.py<br /> Traceback (most recent call last):<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/gym/envs/atari/atari_env.py&quot;, line 9, in &lt;module&gt;<br /> import atari_py<br /> ModuleNotFoundError: No module named 'atari_py'</p> <p>During handling of the above exception, another exception occurred:</p> <p>Traceback (most recent call last):<br /> File &quot;ram.py&quot;, line 51, in &lt;module&gt;<br /> env = gym.make('Pong-ram-v0')<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/gym/envs/registration.py&quot;, line 163, in make<br /> return registry.make(id)<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/gym/envs/registration.py&quot;, line 119, in make<br /> env = spec.make()<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/gym/envs/registration.py&quot;, line 85, in make<br /> cls = load(self._entry_point)<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/gym/envs/registration.py&quot;, line 14, in load<br /> result = entry_point.load(False)<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pkg_resources/<span style="text-decoration: underline;">init</span>.py&quot;, line 2408, in load<br /> return self.resolve()<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/pkg_resources/<span style="text-decoration: underline;">init</span>.py&quot;, line 2414, in resolve<br /> module = <span style="text-decoration: underline;">import</span>(self.module_name, fromlist=['<span style="text-decoration: underline;">name</span>'], level=0)<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/gym/envs/atari/<span style="text-decoration: underline;">init</span>.py&quot;, line 1, in &lt;module&gt;<br /> from gym.envs.atari.atari_env import AtariEnv<br /> File &quot;/usr/local/lib/anaconda3-5.1.0/lib/python3.6/site-packages/gym/envs/atari/atari_env.py&quot;, line 11, in &lt;module&gt;<br /> raise error.DependencyNotInstalled(&quot;{}. (HINT: you can install Atari dependencies by running 'pip install gym[atari]'.)&quot;.format(e))<br /> gym.error.DependencyNotInstalled: No module named 'atari_py'. (HINT: you can install Atari dependencies by running 'pip install gym[atari]'.)</p> <p>___</p> <p>rack-gamir-g06&#160;15% /usr/local/lib/anaconda3-5.1.0/bin/python main.py<br /> Traceback (most recent call last):<br /> File &quot;main.py&quot;, line 51, in &lt;module&gt;<br /> benchmark = gym.benchmark_spec('Atari40M')<br /> AttributeError: module 'gym' has no attribute 'benchmark_spec'</p> <p>and this was the gpu state:</p> <p>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">+<br /> | NVIDIA-SMI 418.43 Driver Version: 418.43 CUDA Version: 10.1 |<br /> |</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">&#8212;+</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">+<br /> | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |<br /> | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |<br /> |===============================+======================+======================|<br /> | 0 TITAN Xp Off | 00000000:3D:00.0 Off | N/A |<br /> | 49% 76C P2 251W / 250W | 10219MiB / 12196MiB | 99% Default |<br /> +</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">&#8212;+</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">+<br /> | 1 TITAN Xp Off | 00000000:3E:00.0 Off | N/A |<br /> | 45% 72C P2 258W / 250W | 8265MiB / 12196MiB | 99% Default |<br /> +</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">&#8212;+</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">+<br /> | 2 TITAN Xp Off | 00000000:60:00.0 Off | N/A |<br /> | 23% 33C P8 10W / 250W | 1481MiB / 12196MiB | 0% Default |<br /> +</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">&#8212;+</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">+<br /> | 3 TITAN Xp Off | 00000000:61:00.0 Off | N/A |<br /> | 31% 52C P2 69W / 250W | 11317MiB / 12196MiB | 0% Default |<br /> +</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">&#8212;+</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">+<br /> | 4 TITAN Xp Off | 00000000:B1:00.0 Off | N/A |<br /> | 23% 35C P2 74W / 250W | 11317MiB / 12196MiB | 0% Default |<br /> +</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">&#8212;+</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">+<br /> | 5 TITAN Xp Off | 00000000:B2:00.0 Off | N/A |<br /> | 23% 40C P2 68W / 250W | 11317MiB / 12196MiB | 0% Default |<br /> +</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">&#8212;+</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">+<br /> | 6 TITAN Xp Off | 00000000:DA:00.0 Off | N/A |<br /> | 23% 40C P2 68W / 250W | 11269MiB / 12196MiB | 0% Default |<br /> +</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">&#8212;+</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">+<br /> | 7 TITAN Xp Off | 00000000:DB:00.0 Off | N/A |<br /> | 24% 43C P2 71W / 250W | 11317MiB / 12196MiB | 0% Default |<br /> +</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">&#8212;+</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>+<span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span><span style="text-decoration: line-through;">-</span>&#8212;+</p> <p>are we doing something wrong?<br /> thanks for your help</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177828">Discussions / General</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11658238/final-project-running-issues">final project running issues</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11459137#post-4233807</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11459137/percentage-of-success-in-programming-question-2-section-b#post-4233807</link>
				<description></description>
				<pubDate>Tue, 30 Apr 2019 10:29:33 +0000</pubDate>
				<wikidot:authorName>Vered Zilberstein</wikidot:authorName>				<wikidot:authorUserId>5248283</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>Notice that score != percentage of success.<br /> After you have trained the network, you should evaluate its accuracy by running it several times and averaging the reward.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177830">Discussions / HW3</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11459137/percentage-of-success-in-programming-question-2-section-b">percentage of success in programming question 2 section b</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11459137#post-4233781</guid>
				<title></title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11459137/percentage-of-success-in-programming-question-2-section-b#post-4233781</link>
				<description></description>
				<pubDate>Tue, 30 Apr 2019 09:44:12 +0000</pubDate>
				<wikidot:authorName>ליאור</wikidot:authorName>								<content:encoded>
					<![CDATA[
						 <p>Can you give us a clue? We talked with a lot of people and we all implemented the learning according to the instructions and got about 0-5%</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177830">Discussions / HW3</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11459137/percentage-of-success-in-programming-question-2-section-b">percentage of success in programming question 2 section b</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11388938#post-4233714</guid>
				<title>No- they have the same transition probabilities</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11388938/q2-theoretical#post-4233714</link>
				<description></description>
				<pubDate>Tue, 30 Apr 2019 07:48:57 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p><a href="http://rl-tau-2019.wikidot.com/forum/t-11442950/theoretical-q2-clarification">http://rl-tau-2019.wikidot.com/forum/t-11442950/theoretical-q2-clarification</a></p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177830">Discussions / HW3</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11388938/q2-theoretical">Q2 theoretical</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11453520#post-4233713</guid>
				<title>Re: Performance in programming Q1</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11453520/performance-in-programming-q1#post-4233713</link>
				<description></description>
				<pubDate>Tue, 30 Apr 2019 07:48:04 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>You should aim for at least 100 steps.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177830">Discussions / HW3</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11453520/performance-in-programming-q1">Performance in programming Q1</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11459137#post-4233711</guid>
				<title>Re: percentage of success in programming question 2 section b</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11459137/percentage-of-success-in-programming-question-2-section-b#post-4233711</link>
				<description></description>
				<pubDate>Tue, 30 Apr 2019 07:47:12 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>70% should be enough.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177830">Discussions / HW3</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11459137/percentage-of-success-in-programming-question-2-section-b">percentage of success in programming question 2 section b</a>
				 	]]>
				</content:encoded>							</item>
					<item>
				<guid>http://rl-tau-2019.wikidot.com/forum/t-11251972#post-4233710</guid>
				<title>Re: Question 3ץ2</title>
				<link>https://rl-tau-2019.wikidot.com/forum/t-11251972/question-3-2#post-4233710</link>
				<description></description>
				<pubDate>Tue, 30 Apr 2019 07:46:07 +0000</pubDate>
				<wikidot:authorName>Lee Cohen</wikidot:authorName>				<wikidot:authorUserId>5195429</wikidot:authorUserId>				<content:encoded>
					<![CDATA[
						 <p>You may use any existing (efficient) algorithm from this course or a previous one, as long as you mention its name, explain why it solves it and it's complexity.</p> <br/>Forum category: <a href="https://rl-tau-2019.wikidot.com/forum/c-5177830">Discussions / HW3</a><br/>Forum thread: <a href="https://rl-tau-2019.wikidot.com/forum/t-11251972/question-3-2">Question 3ץ2</a>
				 	]]>
				</content:encoded>							</item>
				</channel>
</rss>