I have trained a graph attention network using Pytorch_geometric (although, I am pretty sure this question is Pytorch specific) - apologies if it is not.
I want to get node embeddings now from the first GATConv layer on the original but also test graphs that have been slightly modified. I have read about hooks etc, but am struggling to implement these as they do not make perfect sense to me (new to Pytorch framework)
Generally, forward hooks are used to obtain output from intermediate layer(s) when a forward pass is performed through the whole network. For instance, you can register a forward hook for all your conv layers. Then, if you perform a forward pass with model(input), the hook will store the outputs of your conv layers. In your case model.gat_conv1(data) works because it is the first layer of the network. However, you would have to perform another forward pass to get model(data). On the other hand, a forward hook will fetch you the output of both gat_conv1 and the model with one single forward pass. You can choose to use the forward hook depending on your use case.
I have not used Pytorch Geometric, so I do not know about why the embeddings are not normalized. Does GatConv output a normalized tensor?
got it! this makes perfect sense - I will stick to using model.gat_conv1(data) as this is all i require for a certain task i am doing!
on the second point I need to check (i don’t know, the short answer to your question) - but when using Tensorflow, taking the output of the first GATConv layer normally returns normalised constrained values. I will do some digging!