<< extracting stats from activated layers - training >>

Yes, the tensor in your hook function will correspond to the specified layer (accessed by indexing or naming).

Yes, if the activations are created as modules. The alternative way would be to use the functional API for the activation functions, e.g. as done in DenseNet.
If you encounter such a model, you might want to override the forward method and store the activations manually or replace them with the corresponding modules.

Yes, your code looks fine to achieve this.

Haha, no need to apologize. Your post are perfectly clear! :wink:

1 Like