Is it possible to get topological relationships between layers in PyTorch?
I understand that we can do forward and backprop by calling forward(). What if I really need the model relationships? For example, I want to do network pruning. When I delete one Conv layer, I need to update all following layers connected to it accordingly.
Since graph is dynamic, i.e., you can change topology at each iteration or do things like
if random.random() < 0.01: add three linear layers, there is no general rule, unless you define a representation yourself and stick to it in your network definitions.
Thanks for your reply Simon!
I am curious how did PyTorch finish back propagation for each layer if it has no ideas of what layers are following itself, because to update weights of that layer we need to sum all gradients coming from all of its descendants?
A graph is constructed internally at each forward.
Can I get the graph if I want to get what layers are related to it?