Hi , regarding your explanation that dOut/dInput backprop may continue while the other path is stopped. I did a little experiment , initialize a simple linear layered NN with some random input , output and target for loss computation. If I wrap the model in torch.nograd then backward() will show an error as no gradients can be passed . Now , suppose if I set requires_grad = False backprop is computed and I get some value a I get the same value even when I set requires_grad = True . Acc to the forums (No_grad() vs requires_grad) , requires_grad makes sense only for individual tensors and not for model as a whole as backprop will take place regardless . Thus , I am confused by your explanation that dOut/dparam will stop