Deep reinforcement learning using an asynchronous advantage actor-critic (A3C) model written in TensorFlow.
This AI does not rely on hand-engineered rules or features. Instead, it masters the environment by looking at raw pixels and learning from experience, just as humans do.
The Flappy Bird Gym environment was mastered in 48 hours of training using a CPU with 8 cores. Training and evaluation code is available at github.com/andreimuntean/a3c.
- OpenAI Gym 0.8
- TensorFlow 1.0
Uses environments provided by OpenAI Gym.
Each frame is transformed into a 47×47 grayscale image with 32-bit float values between 0 and 1. No image cropping is performed. Reward signals are restricted to -1, 0 and 1.
The input layer consists of a 47×47 grayscale image.
Four convolutional layers follow, each with 32 filters of size 3×3 and stride 2 and each applying the rectifier nonlinearity.
A recurrent layer follows, consisting of 256 LSTM units.
Lastly, the network diverges into two output layers – one is a probability distribution over actions (represented as logits), the other is a single linear output representing the value function.
Implementation inspired by the OpenAI Universe reference agent.
Heavily influenced by DeepMind's seminal paper 'Asynchronous Methods for Deep Reinforcement Learning' (Mnih et al., 2016).