Hello,

I am trying to train a GAN in pytorch, and as training the “discriminator” will require doing based on the discriminator output on “real”, and “fake” samples. I tried doing this in two different ways that I expected to be equivalent, but seems to be only one of them works.

**First approach: (Didn’t work)** - summing the two criterions and backward the combined loss.

```
criterion = nn.BCELoss()
d_out_real = model_d(real_input)
real_loss = criterion(d_out_real, real_label)
d_out_fake = model_d(fake_input)
fake_loss = criterion(d_out_fake, fake_label)
total_loss = real_loss + fake_loss
d_optimizer.zero_grad() # Adam optimizer to optimize model_d parameters
total_loss.backward()
d_optimizer.step()
```

**Second approach: (Works fine):** backward each criterion loss individually, then taking an optimization step.

```
criterion = nn.BCELoss()
d_optimizer.zero_grad() # Adam optimizer to optimize model_d parameters
d_out_real = model_d(real_input)
real_loss = criterion(d_out_real, real_label)
real_loss.backward()
d_out_fake = model_d(fake_input)
fake_loss = criterion(d_out_fake, fake_label)
fake_loss.backward()
d_optimizer.step()
```

**Third approach: (Doesnot work):** Exactly like second approach, except for delaying the `real_loss.backward()`

until after I compute the model output and loss from fake data.

```
criterion = nn.BCELoss()
d_optimizer.zero_grad() # Adam optimizer to optimize model_d parameters
d_out_real = model_d(real_input)
real_loss = criterion(d_out_real, real_label)
d_out_fake = model_d(fake_input)
fake_loss = criterion(d_out_fake, fake_label)
real_loss.backward()
fake_loss.backward()
d_optimizer.step()
```

**Notes:**

- I tried using two
`BCELoss`

modules, but it didn’t help. - I also tried computing the loss using the functional APIs
`torch.nn.functional.binary_crossentropy_loss`

. Also this changed nothing.

Any explanations ?

Thanks,

Moustafa