Hey Checkout my repo here I implemented a3c with gpu use and hog wild training. I call it A3G and it’s works very well it’s actually even faster at training Atari games than mass parallelized evolutionary algorithms with also much higher overall performance.
update Just realized you had already visited repo and we spoke on issues on github. 
Guess you can disregard then lol