Grad is none outside torch.no_grad

1

Hi, I’m facing an issue about the usage of torch.no_grad. I’m using something like this :

 with torch.no_grad():
                    logits ,_,_ = self.model(input_images)
                    logits = logits.detach()


_, loss_1, _ = self.model(images_second)

....

_, loss_2, _ = self.model(input_images, gt=logits)

total_loss = loss_1 + loss_2
total_loss.backward()

When I’m printing the gradient of my network, I get something like :

Grad is None :
...
params_X.weight
params_Y.weight
...


Grad is not None :
...
params_X.norm.weight
params_Y.norm.bias
....

I don’t understand why the grad of the parmaters are None, when I remove the torch.no_grad(), all params have a grad. Could anyone helps me on this ?

Thanks

2

If you could provide a small executable example (with a toy network and random inputs), then I can try figuring it out.