Variable -> 2D Line Select?

1

Hi,
Does anyone know how to differentiably ‘select’ indices that fall on a 2D line within a 2D tensor / rectangle ?

Something like this:

buffer = Variable( torch.FloatTensor( 128, 128 ) )
p0 = Variable( torch.FloatTensor( [2.4, 9.4] ) ) 
p1 = Variable( torch.FloatTensor( [5, 11.5] ) ) 

# magic happens here
buffer.lineSelect(p0, p1).fill( 1 )

Ideally lineSelect would automatically round float values to their proper discrete indices and would be differentiable.
My plan is to use this method to implement a differentiable 3D renderer in pytorch.

The nearest insight I found on the forum is this but it does not really show out to ‘select’ from a buffer. Indexing a variable with a variable

1 Like
2

you can use index_select, and yes it’s differentiable wrt buffer (but not differentiable wrt p0 or p1, because it’s not continuous):

buffer = Variable( torch.FloatTensor( 128, 128 ) , requires_grad=True)
p0 = Variable( torch.FloatTensor( [2.4, 9.4] ) )
p1 = Variable( torch.FloatTensor( [5, 11.5] ) )

buf2 = buffer.index_select(0, p0.long()).index_select(1, p1.long())
buf2.backward(torch.ones(2, 2))
2 Likes
3

Nice, thanks! I need wrt to p0 and p1 so I can train a network on the generation of these points. Still, I really appreciate the response!

4

wrt p0 is a non-differentiable operation, but you can use reinforcement learning, by using the reward(…) function on stochastic variables. so you will first get float indices, and you will stochastically sample from them over a multinomial or something, and you will give this sampled node a reward to propagate gradients back. see pytorch/examples or read on REINFORCE algorithm

1 Like