Loss gradient is None

1

Hi everyone,

I built my own loss function to calculate the log-likelihood for a Mixture Density Network, connected to LSTM. However, there seems to be a problem somewhere (the loss goes to infinity, and the whole thing collapses).
While I am debugging, I found that after I get the loss, and do the backward() step, the loss doesn’t have any grad information
loss.grad = None
Is this normal?

Thank you

2

Yes, only the gradients w.r.t. explicitly created Variables (called leaf Variables) are saved. If you want to get the grad w.r.t. some intermediate values (i.e. computed form leaf Variables) you need to use hooks (search for register_hook in the docs).

1 Like