How should I select random part of tensor?

1

I want to feed part of the model’s output to auxiliary loss.

With it Im getting the error

x = torch.rand(32, 128, 1000)
lens = torch.randint(0, 1000, (32,))
mask_start = (torch.rand(32) * (lens - 64)).long()
x[:,:,mask_start:mask_start+64]

TypeError: only integer tensors of a single element can be converted to an index
2

can you print the shape of mask_start

3

Shape is torch.Size([32])

4

Ok why are you making your mask start variable have a size of 32. You cannot do that for indexes. Is there a reason for 32.

5

Forum formatting eats first line, fixed it.
32 is batch size.

6

Ok you could just use a for loop like this:

for i in range(x.shape[0]):
   x[i, :, mask_start[i]: mask_start[i] + 64]
7

It should damage performance

8

Ya it would but that is the easiest option.

9

Well, i would not use loops in tensors.
Seems like torch.gather is thing i need.
But not sure how to generate tensor with all indeces i need.
Like if mask start at 10 and ends at 74, i should have tensor with 10,11,12… 73, 73.

10

Ya that should work.