both C extension and Numpy extention for pytorch, We have to write both forward and backward functions.
Here is a silly question but I have to ask, we have autograd which automatically calcuate gradients for us, and we only need to worry about forward(). but why here we have to write
What if I only know how to write a forward function, especially when dealing with complex layers like convolution and RNN? see below the official numpy extension example
What shall I do when I have no idea how to write complex backward function for convolution as below?
def backward(self, grad_output): numpy_go = grad_output.numpy() result = irfft2(numpy_go) return torch.FloatTensor(result) def backward(self, grad_output): input, filter = self.saved_tensors grad_input = convolve2d(grad_output.numpy(), filter.t().numpy(), mode='full') grad_filter = convolve2d(input.numpy(), grad_output.numpy(), mode='valid') return torch.FloatTensor(grad_input), torch.FloatTensor(grad_filter)
then how can I proceed? what should I do to move forward to create an extension?
Thanks a lot!