Use of torch.nn.functional.grid_sample()?

1

I want to use grid_sample(), and make a small example for test:

import torch.nn.functional as F
import torch.optim as optim
import torch
import torchvision
from PIL import Image
import numpy as np

img = Image.open('frame1.jpg')
img = np.array(img)
img = np.array([img])
b = torch.from_numpy(img)
b = b.permute(0, 3, 1, 2)
print(b.size())
# b has the size (1, 3, 360, 640)
flow = torch.rand(1, 360, 640 , 2)
b = Variable(b)
flow = Variable(flow)
ohno = F.grid_sample(b, flow)

but I got some error:
Traceback (most recent call last):
File “test.py”, line 25, in
ohno = F.grid_sample(b, flow)
File “/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py”, line 995, in grid_sample
return GridSampler.apply(input, grid)
File “/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/vision.py”, line 30, in forward
backend = type2backend[type(input)]
File “/usr/local/lib/python3.5/dist-packages/torch/_thnn/init.py”, line 15, in getitem
return self.backends[name].load()
KeyError: <class ‘torch.ByteTensor’>

Does any one know where I am wrong?

1 Like
2

b=b.float()

need to typecast tensor from byte to float

2 Likes
3

Hi @smth, I have a quick question here. So for the flow(which is the grid param in grid_sample), is there a limit on its range? I saw some people said that the grid should be in the range of [-1,1]. Is that true? Because the range of my optical flow is like [-20,20]. So should I divide the flow by 20 before I feed it to the grid_sample function for image warping? And is there anything else to notice about using the grid_sample() function? Thank you very much.

1 Like
4

yes you should keep it between [-1, 1].

5 Likes
5

@smth I have a question that does `torch.nn.functional.grid_sample propagate the grad of the output sampled map? or say, did it help me backwards the grad of the output to the input automatically?

6

yes it does @Zichun_Zhang

7

Can you please explain use of this function with example? Why we need this and
how we can use this?Thanks for help in advance