Using boolean logic for creating indices for index_select (possible bug?)

Hi,

I’m trying to use properties of one tensor to select out values from another tensor and I think I want to use index_select. I’m getting an error, however, that I don’t know how to get around.

Here’s a minimal working example:

import torch
from torch.autograd import Variable

blah = Variable( torch.Tensor( [[ 1.1,2.2,3.3],[4.4,5.5,6.6]] ), requires_grad=True )
all_above10 = Variable( torch.Tensor( [ 11, 12, 13, 14, 15, 16]), requires_grad=False)
inds_above10 = torch.nonzero( all_above10 > 10 )
print( "inds_above10", inds_above10 )
#foo = Variable( torch.LongTensor([0,0,0,1,1,1]), requires_grad=False)
foo = inds_above10 / 3
print( "foo", foo )
bar = torch.index_select( blah, 0, foo ) 
print(bar)

And I get this error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-fa2b5f6dcfe9> in <module>()
      9 foo = inds_above10 / 3
     10 print( "foo", foo )
---> 11 bar = torch.index_select( blah, 0, foo )
     12 print(bar)

RuntimeError: invalid argument 3: Index is supposed to be an empty tensor or a vector at /Users/andrew/rosetta/GIT/tmol/src/pytorch/aten/src/TH/generic/THTensorMath.c:261

If I just create my LongTensor foo manually (by uncommenting the other definition of foo and commenting out the current one), then there’s no problem.

How can I use logic (e.g. “all_above10 > 10” ) to get exactly the entries from one tensor into another tensor?

Edit: Im using version 0ef10385b2 of pytorch from 2018/2/13

Hi,

Are you sure you’re not looking for masked_select ? :smiley:

2 Likes

Thank you for your reply. This won’t work in my case, because I need to be able to copy some of the rows multiple times. E.g. if I used [ 0, 0, 0 , 1, 1, 1 ] for “foo” above, then I get three copies of row 0 and three copies of row 1.

Variable containing:
 1.1000  2.2000  3.3000
 1.1000  2.2000  3.3000
 1.1000  2.2000  3.3000
 4.4000  5.5000  6.6000
 4.4000  5.5000  6.6000
 4.4000  5.5000  6.6000
[torch.FloatTensor of size (6,3)]

Ho ok,
Then the error message is not super explicit, but the third argument to index_select should be LongTensor and I think it is a ByteTensor in your example.

I’ve been scratching my head on that, too. When you ask for foo’s type(), it reports LongTensor. Its type is indistinguishable when created the two different ways in my example code above, but one way works and one doesn’t.

print( "foo", foo, foo.type() )

it says

foo Variable containing:
 0
 0
 0
 1
 1
 1
[torch.LongTensor of size (6,1)]
 torch.LongTensor

e.g., I can update my MWE to use a non-binary value for the indices I’m looking for:

import torch
from torch.autograd import Variable

blah = Variable( torch.Tensor( [[ 1.1,2.2,3.3],[4.4,5.5,6.6],[7.7,8.8,9.9]] ), requires_grad=True )
some_above10 = Variable( torch.Tensor( [ 11, 12, 13, 2, 3, 4, 14, 15, 16]), requires_grad=False)
inds_above10 = torch.nonzero( some_above10 > 10 )
print( "inds_above10", inds_above10 )
#foo = Variable( torch.LongTensor([0,0,0,2,2,2]), requires_grad=False)
foo = inds_above10 / 3
print( "foo", foo, foo.type() )
bar = torch.index_select( blah, 0, foo ) 
print(bar)

which delivers this output:

inds_above10 Variable containing:
 0
 1
 2
 6
 7
 8
[torch.LongTensor of size (6,1)]

foo Variable containing:
 0
 0
 0
 2
 2
 2
[torch.LongTensor of size (6,1)]
 torch.LongTensor
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-5-544b552e6f58> in <module>()
      9 foo = inds_above10 / 3
     10 print( "foo", foo, foo.type() )
---> 11 bar = torch.index_select( blah, 0, foo )
     12 print(bar)

RuntimeError: invalid argument 3: Index is supposed to be an empty tensor or a vector at /Users/andrew/rosetta/GIT/tmol/src/pytorch/aten/src/TH/generic/THTensorMath.c:261

Would you be willing @albanD to confirm that the MWE fails for you, too?

Ah ha! There is a difference between the two versions of foo:

One is of size (6,1) and the other is of size (6,)

If I squeeze the inds_above10 tensor, then my case works.

Related to the error that someone was seeing here:

It would be super nice if the error message could say “You gave us what is essentially a row vector / column vector but we want it to be a rank-1 tensor.”

Also: If I give a rank-1 tensor to torch.nonzero, the output should probably be a rank-1 tensor, right?

Hi,

Sorry couldn’t answer earlier,
In the error message, vector means 1D tensor indeed.
torch.nonzero return a tensor of size nnz x ndim (number of non zero elements by number of dimensions in the original tensor), so it is expected to return a (6, 1) tensor in your case.

So it cannot be used easily as index? It seems very intuitive to do something like a[a.nonzero()] but if a is a 3 dimensional tensor this cannot work