Understanding backward in WGAN-GP

Hi, I’m having trouble understanding the code of WGAN with gradient penalty here.

The code defines two tensors:

one = torch.FloatTensor([1])
mone = one * -1

Later, it defines the loss and takes gradient like this:

D_real = netD(real_data_v).mean()
D_real.backward(mone)

D_real is an autograd.Variable, and backward is a method associated to D_fake. What is the argument “mone” for?

Same for the argument “one” in:

D_fake = netD(inputv).mean()
D_fake.backward(one)

Does it have something to do with the labels for fake and real? Wouldn’t we just need to call .backward() with no arguments? This seems to be the standard source for WGAN-GP. I see a similar code in here, but I guess it was modified from the previous.

The -1 is multiplied to the gradient, so the loss term contains it negatively. The 1 is probably not needed, but we all copied it from the DCGAN in the pytorch examples or the WGAN code.
For DCGAN and plain WGAN I can see the advantage over adding loss components (memory consumption), fir WGAN-GP its probably just copy-paste without thinking, at least that is the case for me.

Best regards

Thomas

2 Likes

In order to Update D network:
lossD = Expectation of D(fake data) - Expectation of D(real data) + gradient penalty
lossD ↓,D(real data) ↑

so you need to add minus one to the gradient process

Hi Tom,
I hope you are well. Sorry in this lines when we update the discriminator which cost is used for updating ?

errD_penalty = errD_fake - errD_real + grad_penalty
errD = errD_fake - errD_real
optimizerD.step()

or here

D_cost = D_fake - D_real + gradient_penalty
 Wasserstein_D = D_real - D_fake
optimizerD.step()
```