I am trying to understand the similarities/differences with the sonnet framwork of deepmind.
here are my conclusions:
Boths frameworks are trying to introduce some freedom degres in the forward/backward implementation of the layers.
Sonnet splits the layers implementation in two parts :
- Configuration of the layer
- Connection of the layer to the syntactic tree
While PyTorch introduce a new notion of syntactic tree: the dynamic syntactic tree;
While the intention is the same, in case of Sonnet, as soon as it is connected, you cannot change it again. A contrario, in the case of Pytorch, you change the syntactic tree all along the learning,
Am I correct ?