I’m currently working on implementing a GAN, with a custom loss function which uses other Python modules and libraries. The input to the loss function is the output of GAN which is torch.tensor of size [8,3,256,256]
The forward path of the loss function is as follows:
Convert torch.tensor to PIL.Image and normalize it
Detect faces in the PIL.Image using face_detector and output landmarks
Use these landmarks to calculate torch.nn.MSELoss()
Once loss is calculated return it as loss to the original function
I’m able to implement the forward path with no particular issue. I don’t want to calculate gradients for this task but want to use this loss to back propagate GAN whose output I’m using to input to the loss function.
I’ve no particular priority for face_detector but would love to use any in general.
I’m trying to use almost nothing from python numpy, the only thing in numpy are the landmarks which are output of the face_detector.
If I add requires_grad=True while returning the loss, I’m able to perform a backward call but there are no gradients which are calculated.
Is there a way in which I’m able to accomplish the task of using this loss to back propagate through the GAN?
If I understand you correctly, even though you “don’t want to calculate
gradients for” the parameters used by the face detector in your loss
function, you do want to calculate gradients for the parameters of
the GAN used to generate the image fed to the face detector.
Once you start to pass the generated image through non-pytorch
calculations, you “break the computation graph,” and won’t be able
to backpropagate from one side of the non-pytorch calculations to
the other.
Your best bet will to be to implement the face detector in pytorch (if
it isn’t already) and figure out how to implement your landmark
calculations using differentiable pytorch operations. If the whole
end-to-end calculation – GAN image generation, face detection,
landmark calculations, and final loss function – is done with (differentiable)
pytorch tensor operations, pytorch’s autograd will work, you can call loss.backward(), and use the GAN-parameter gradients produced by
autograd to optimize your GAN.
(Other approaches are theoretically possible, but likely to be unworkable.)
Most probably I will implement or use some face-detector in PyTorch, cause any other method may not be implemented properly.
If that doesn’t work is there any other way where I can use the gradients calculated before the face-detector and implement those in the Custom Losses Backward Function?