Bilinear sampler

alwynmathew · March 10, 2018, 1:15pm

I want to do something like this: https://github.com/mrharicot/monodepth/blob/master/bilinear_sampler.py

Inputs are input_image of size 1x1x256x512 and offset of size 1x1x256x512. I need to sample input_image and move each pixel in x direction according to offset.

Can I use torch.nn.functional.grid_sample? If yes, how? Any suggestion is highly appreciated.

aqib.mumtaz · May 19, 2018, 12:15pm

did you find your answer? so please share

sniklaus · May 20, 2018, 10:25pm

You could adjust the following code that performs backward-warping for optical flow.

github.com

sniklaus/pytorch-spynet/blob/master/run.py#L127


					torchHorizontal = torch.linspace(-1.0, 1.0, variableInput.size(3)).view(1, 1, 1, variableInput.size(3)).expand(variableInput.size(0), 1, variableInput.size(2), variableInput.size(3))
					torchVertical = torch.linspace(-1.0, 1.0, variableInput.size(2)).view(1, 1, variableInput.size(2), 1).expand(variableInput.size(0), 1, variableInput.size(2), variableInput.size(3))


					self.tensorGrid = torch.cat([ torchHorizontal, torchVertical ], 1).cuda()
				# end


				variableGrid = torch.autograd.Variable(data=self.tensorGrid, volatile=not self.training)


				variableFlow = torch.cat([ variableFlow[:, 0:1, :, :] / ((variableInput.size(3) - 1.0) / 2.0), variableFlow[:, 1:2, :, :] / ((variableInput.size(2) - 1.0) / 2.0) ], 1)


				return torch.nn.functional.grid_sample(input=variableInput, grid=(variableGrid + variableFlow).permute(0, 2, 3, 1), mode='bilinear', padding_mode='border')
			# end
		# end


		self.modulePreprocess = Preprocess()


		self.moduleBasic = torch.nn.ModuleList([ Basic(intLevel) for intLevel in range(6) ])


		self.moduleBackward = Backward()
	# end

Disparity is essentially just optical flow without vertical motion.

alwynmathew · June 17, 2018, 7:13am

@sniklaus solutiuon looks great.

Take a look at my repo too here