Backpropagation fails after moving tensor from GPU to CPU (numpy version)

autograd
#1

I would like to train two CNNs and then it has to be updating weights in both the networks. For better understanding, Please look at the architecture in attached file.

Capture2
Capture2.PNG1628Ă—503 248 KB

I give depth features as an input for first CNN and get a disparity map. I will warp the disparity map along with input images so I used Scipy(for interpolation operation:mapped_coordinates). Then I feed this warped images as feature tensor as an input for second CNN and calculate the mean square loss between predicted image and ground truth image.

Now the problem is that I moved a tensor outupt from first CNN to CPU(using tensor to numpy operation) and did that interpolation operation during forward propagation and created a warped image as an tensor (numpy to tensor). It works perfectly in a f/w path and loss.backwards() doesn’t calculate gradients w.r.to weights.

What could be the solution for this problem? Thanks in advance.

#2

Scipy operations are not differentiable from pytorch point of view.

#3

Can backpropagation work if we move data from GPU (tensor) to CPU (ndarray)? Or do we have to do interpolation using tensors itself ?

#4

no. when it is in numpy side, it is not a pytorch tensor anymore. python can not track its operation. and numpy doesn’t provide bwd methods.

#5

I think more to the point is that you must be taking it out of the Variable class and thus the automatic gradient calculations never occur.

#6

Yes, I understand.
I would like to do the below operation using torch tensor itself, Instead of converting into numpy operation.
Scipy Map coordinate operation
In both tensorflow and pytorch, only an image resize operation is available. But, I have never seen this bicubic interpolation using map_coordinates . (To my knowledge)If it is possible, then we can treat this problem as an end to end learning… Do you have any idea guys ?

1 Like