I have an RNN network model that is connected to a linear layer. My question is that if I use the softmax function outside the model(like outside the forward function) to calculate loss with cross-entropy loss, will it be included in the loss.backward() calculation?
Yes, it will be included. The backpropagation is not linked with the forward function. As much as I understand forward is just useful to make sure all the hooks and other essential nn.Module functionalities are maintained before the __call__() of the class.
1 Like