Extracting grad tensors from a subgraph

[backpropper (Abhinav Gupta)]

Here is what I want to do:

Autograd through a sub branch with multiple loss functions

I am trying to take all the grad values of all the parameters in the model (after doing backward) and calculate the variance of it. Then I want to update only some of the parameters using this variance. baseline is a nn.Sequential() module. Do the following perform the desired task? optimizer = optim.Adam(model.parameters(), lr=opts['lr']) cv_optimizer = optim.Adam(model.baseline.parameters(), lr=10*opts['lr']) # # Training script # optimizer.zero_grad() cv_optimizer.zero_grad() loss.back…