What wrong with mine masking?

Pytorch 1.2, tacotron gst model.
I wonna try to mask out gst outputs.
I have two 3d array with shape [32, 177, 512]

So i just wonna have zeros in first tensor like in second

Taking mask and expand shape
gst_mask = ~get_mask_from_lengths(text_lengths)
gst_mask = gst_mask.unsqueeze(-1).expand_as(gst_outputs)
Seems ok

Then i do
gst_outputs.data.masked_fill_(gst_mask, 0.0)

But get unexpected output:


How is gst_outputs created? Can your add a gst_outputs = gst_outputs.clone() before doing the masking?

Basically it is like this

will the gradient propagate if i copy tensor?

Yes it will :slight_smile:
As I was suspecting, gst_outputs is created with an expand. This means that this is not backed by actual memory and writing to one place of it will have effect on other places.
Using clone will force it to be backed by real memory and remove this problem !