Questions on DCGAN Example in Repo

leeks · March 9, 2019, 5:54pm

I’ve several queries regarding how the DCGAN is implemented and trained in the official repository (https://github.com/pytorch/examples/blob/master/dcgan/main.py)

Updating Generator with Discriminator Frozen

Firstly, in the GAN updating of the G network, G is frozen to train D, and then D is frozen to train G sequentially. To do so, the detach() function is used to avoid backpropping to G as seen in

github.com

pytorch/examples/blob/master/dcgan/main.py#L225-L229


# train with fake
noise = torch.randn(batch_size, nz, 1, 1, device=device)
fake = netG(noise)
label.fill_(fake_label)
output = netD(fake.detach())

However, there doesn’t seem to be a need to implement detach() when getting the output from the discriminator, to prevent updates to the discriminator - why? This is seen in:

github.com

pytorch/examples/blob/master/dcgan/main.py#L241


errD_fake.backward()
D_G_z1 = output.mean().item()
errD = errD_real + errD_fake
optimizerD.step()


############################
# (2) Update G network: maximize log(D(G(z)))
###########################
netG.zero_grad()
label.fill_(real_label)  # fake labels are real for generator cost
output = netD(fake)
errG = criterion(output, label)
errG.backward()
D_G_z2 = output.mean().item()
optimizerG.step()


print('[%d/%d][%d/%d] Loss_D: %.4f Loss_G: %.4f D(x): %.4f D(G(z)): %.4f / %.4f'
      % (epoch, opt.niter, i, len(dataloader),
         errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))
if i % 100 == 0:
    vutils.save_image(real_cpu,

In this case, is netD frozen, while G is being updated?

DCGAN Generator Initial Projection Layer

Secondly, why is there not a need to project the vector into an FC layer and reshape it before performing the first conv2dtranspose, as seen in the original paper? The current implementation simply performs a conv2dtranspose on an input vector:

github.com

pytorch/examples/blob/master/dcgan/main.py#L122-L123


# input is Z, going into a convolution
nn.ConvTranspose2d(     nz, ngf * 8, 4, 1, 0, bias=False),

The projection layer is something that is quite commonly implemented in TensorFlow for DCGANs, for example:

github.com

aymericdamien/TensorFlow-Examples/blob/master/examples/3_NeuralNetworks/dcgan.py#L41-L56


def generator(x, reuse=False):
with tf.variable_scope('Generator', reuse=reuse):
    # TensorFlow Layers automatically create variables and calculate their
    # shape, based on the input.
    x = tf.layers.dense(x, units=6 * 6 * 128)
    x = tf.nn.tanh(x)
    # Reshape to a 4-D array of images: (batch, height, width, channels)
    # New shape: (batch, 6, 6, 128)
    x = tf.reshape(x, shape=[-1, 6, 6, 128])
    # Deconvolution, image shape: (batch, 14, 14, 64)
    x = tf.layers.conv2d_transpose(x, 64, 4, strides=2)
    # Deconvolution, image shape: (batch, 28, 28, 1)
    x = tf.layers.conv2d_transpose(x, 1, 2, strides=2)
    # Apply sigmoid to clip values between 0 and 1
    x = tf.nn.sigmoid(x)
    return x

Is there something about conv2dTranspose I’m missing?

Thanks a lot!