Q-Table learning in OpenAI grid world.

Q-Table Learning-Clean.ipynb

Thanks for the code. Question: Frozen lake returns r=1 when the agent reaches the last square and r=0 in all other cases. If rAll[] is meant to be the running sum of total reward shouldn't it be initialized before starting all the episodes? Otherwise it is redundant as you could just have rList.append[r]...

Thanks for the code. Question: Frozen lake returns r=1 when the agent reaches the last square and r=0 in all other cases. If rAll[] is meant to be the running sum of total reward shouldn't it be initialized before starting all the episodes? Otherwise it is redundant as you could just have rList.append[r]...

rAll[] used to calculate the average reward per episode at the end of training, which I don't think is helpful. It makes more sense to instead calculate moving averages to track the change in the rate of success over training episodes. This info is helpful when coupled with historical values of j (number of steps in each episode) to visualize how the number of steps changes during training. I don't know why that part was commented out in the code. Anyways, in order to calculate the moving averages of the model's success rate, you either need to keep track of j or add up the rewards in each episode (although its 0 or 1 in this case, other problems may be more complex--such as chess, where the episode reward may vary according to enemy pieces killed, draw/loss, etc) and be able to refer to the index of the said episode through rAll[].