Masked Fill can't differentiate the mask

When am trying to update zero values of a variable x[ x<=0.0] = 1.0 where x is a variable and is of floatTensor. I get this error, ‘MaskedFill can’t differentiate the mask’ and after debugging a bit, I found that in

def setitem(self, key, value):
if isinstance(key, Variable) and type( == ‘ByteTensor’:
if isinstance(value, Variable):
return MaskedScatter.apply(self, key, value, True)
return MaskedFill.apply(self, key, value, True)
return SetItem.apply(self, key, value)
It always go to the MaskedFill statement because they data of key is ByteTensor. Any suggestion as to why could this happen and how to fix it ? It would work fine with SetItem statement

I think torch.where on master will solve your problem.

Otherwise, you could try something like:

mask = x[x > 0.0].float()
x * mask + x[x <= 0.0].float()

but I’m not sure if that’s differentiable.

MaskFill is nondifferentiable almost everywhere wrt to the mask. So your x > 0.0 can’t require gradient. The filled tensor, x here however is differentiable almost everywhere so that’s fine.

I remember that we fixed variable comparison results (the byte tensor) requiring gradient. So you can try update, or just do x[(x <= 0.0).detach()] = 1

Thanks, that fixed it