Categorical DQN Implementation

Just wanted to share an attempt at implementing in Pytorch the Categorical DQN algorithm published by DeepMind last week in A distributional perspective on reinforcement learning.

It currently trains well on Catch, still running experiments on Atari. I am yet to find a good set of hyperparams for Atari games, will post updates about this.

You can find the repo here, let me know if you have any suggestions / comments.

1 Like

Wow, Rémi Munos is at Deepmind now :astonished:

I think this is his tenth paper or so at DeepMind :slight_smile: . Each one worth reading…

He was teaching RL in my undergrad school in France. This course changed my life.
The paper is amazing, now I’m looking forward seeing how it reacts with some parallelism boost.

Haha, I have some async boilerplate lying around, was thinking the same.

1 Like