I have a tensor of dimension (batch_size, Seq_len)
I want to mask out all values between two specific values of seq_len, for example 100, and 125. (including 100 and 125). There might be multiple such occurrences of them too.
I want to mask out with ‘-1’
how can I do it in a fast way in pytorch?
I’m confused about what you are asking. “Seq_len” is the second
dimension of your tensor, but then you have “two specific values of
seq_len.” I don’t understand what you mean. Is “seq_len” an index
into your tensor, or is it the value of an element of your tensor?
If “seq_len” is an index, then you can use slice notation. That is,
t is your tensor, then:
t[:, 100:126] = -1.0
will overwrite the elements of your tensor whose second index is in
the range [100, 125] (inclusive) with -1.0.
If “seq_len” is a value, then if you want to replace with -1 the elements
of your tensor whose values fall in a given range,
should do what you want. That is, if
t is your tensor, then:
t.masked_fill_ (t.ge (100.0) * t.le (125.0), -1.0)
will overwrite the elements of your tensor whose value fall in the
range [100.0, 125.0] (inclusive) with -1.0.
Thanks for your reply and thorough explanation.
However, values in the seq_len dimension are some integers which are basically word_indices.
let me clarify what I want to do:
Lets say seq_len = 128
consider X = myTensor[0,:]. # this would be of size [1, 128].
for example: X = [4,121,4,501,100, …,125, 90,92, 100, …,125]
I want all values between (including) 100 and 125 which I showed with …, set to -1. And I want to do that for all rows of myTensor (#row = batch_size)
I hope I could clarify this. Thanks again
Is there anyway I can use
masked_fill_() for this but not with respect to actual values?
Based on what I think you want, the
masked_fill_() code I posted
should work for you.
Could you post a short, self-contained script that uses
as I described, together with its output (including errors, if any), and
compare, concretely, what
masked_fill_() gives you with what you
The script you described set all elements whose values lies in between 100 and 125 to -1.
However, my elements are just some indices and I want the elements who are between specific start and end to be -1.
Just to clarify the confusion let me change the start and end to 41 and 7 respectively.
I want to change
x = torch.tensor([1,3,4,41,10,7,5,1,8,3,41,90,78,7,41]) to
With the code you sent
x.masked_fill_(x.ge(41) * x.le(4), -1.0) the output wouldn’t change.
I understand what you are asking now.
I do not see a way to do it using straightforward tensor operations;
I think you will have to loop over the elements in each row.