Implementation of decouple neural interface (DNI)

1

Hi guys, I implement the Decoupled Neural Interfaces using Synthetic Gradients in pytorch. The paper uses synthetic gradient to decouple the layers among the network, which is pretty interesting since we won’t suffer from update lock anymore. I test my model in mnist and almost achieve result as the paper claimed.

reference:

  1. Decoupled Neural Interfaces using Synthetic Gradients
  2. Understanding Synthetic Gradients and Decoupled Neural Interfaces
5 Likes
2

this is very nice Andrew. Thanks for sharing.

1 Like
3

working on the paper, thanks for the nice work!

1 Like
4

Awesome, thank you, Andrew!

1 Like