Hi,
I am still a little bit confused regarding the detach() method.
We do fake = netG(noise).detach() to prevent the backpropping through netG. Now, for the netG training, we do output = netD(fake). If we had lets say
fake = netG(inputs)
Loss1 = criterion(netD1(fake), real_label)
Loss2 = criterion(netD2(fake), real_label)
Loss3 = criterion(netD3(fake), real_label)
Loss_other = criterion_other(fake, target)
Loss = Loss1 + Loss2 + Loss3 + Loss_other
Loss.backward()
does this create the graphs for each of the netDs? Will be wrong if I did
Loss1 = criterion(netD1(fake).detach(), real_label)
Loss2 = criterion(netD2(fake).detach(), real_label)
Loss3 = criterion(netD3(fake).detach(), real_label)
Loss_other = criterion_other(fake, target)
Loss = Loss1 + Loss2 + Loss3 + Loss_other
Loss.backward()
to save some memory, since I don’t need to backprop through netD? Will there be any difference in backpropping?
Regards
Nabarun