# Adversarial network feature decorrelation: passing loss gradient

**URL:** <https://discuss.pytorch.org/t/adversarial-network-feature-decorrelation-passing-loss-gradient/151397>\
**Category:** autograd\
**Created:** [May 11, 2022, 11:36am UTC](https://discuss.pytorch.org/t/adversarial-network-feature-decorrelation-passing-loss-gradient/151397 "2022-05-11T11:36:34Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![damar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/damar/32/49493_2.png) [@damar](https://discuss.pytorch.org/u/damar)\
**Post date:** [May 11, 2022, 11:36am UTC](https://discuss.pytorch.org/t/adversarial-network-feature-decorrelation-passing-loss-gradient/151397/1 "2022-05-11T11:36:34Z")

</div>

Hi all! I’m trying to construct an ANN to decorrelate a classifier from a given feature. I think the best way to show is with a little sketch (I hope it renders):

 ![Screenshot from 2022-05-11 12-41-21](https://discuss.pytorch.org/uploads/default/original/3X/b/a/ba826971cccbf430703078a50e39bf4b07f94700.png)  
(the constant can be modified to balance classifying and feature decorellation)

The problem I have is how I pass the ANN loss to the classifier so it can calculate the gradient with respect to the classifier outputs, without modifying the ANN itself. What I have now (clearly still wrong) is something like this:

```auto
        #run number of batches
        for i, data in enumerate(trainloader, 0):
            #regtargets are the x feature values that the regressor tries to find
            inputs, targets, regtargets = data
            inputs, targets, regtargets = inputs.float(), targets.float(), regtargets.float()
            targets = targets.reshape((targets.shape[0], 2))
            #reset optimizers
            optimizer_reg.zero_grad()
            optimizer.zero_grad()

            #get classifier predictions
            outputs = mlp(inputs)
            
            #outputs are fed to ANN (detach, Error:args don't support auto diff, but one requires grad)
            ANN_in = outputs.detach()
            ANN_in.requires_grad = False
            #optimise ANN to new outputs
            ANN = TrainANN_hot(ANN, ANN_in, regtargets)
            ANNoutputs = ANN(ANN_in)
            loss_ANN = loss_function2(ANNoutputs, regtargets)
            loss_ANN.backward()
            optimizer_reg.step()
            optimizer_reg.zero_grad()
            #get new loss after optimization
            ANN_in.requires_grad = True
            ANNoutputs = ANN(ANN_in)
            loss_ANN = loss_function2(ANNoutputs, regtargets)
            
            #loss is calculated and complete model is retrained
            loss = loss_function(outputs, targets) - lda* loss_ANN
            loss.backward()

```

I’m new to pytorch so I hoped to get some insight on how to best implement gradient reversal.

---

<div class="post-metadata">

**Author:** ![damar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/damar/32/49493_2.png) [@damar](https://discuss.pytorch.org/u/damar)\
**Post date:** [May 20, 2022, 2:16pm UTC](https://discuss.pytorch.org/t/adversarial-network-feature-decorrelation-passing-loss-gradient/151397/2 "2022-05-20T14:16:36Z")

</div>

I eventually found a much more elegant solution here: [[Solved] Reverse gradients in backward pass](https://discuss.pytorch.org/t/solved-reverse-gradients-in-backward-pass/3589)