Replace all values close to zero

1

I would like to know how to set values that are close to zero to a certain value. Let’s say I have a tensor T. During the forward pass, I would like to set all values in T in the range [-1e-4, 1e-4] to -1e-4 if the value is negative and 1e-4 in case of a small but positive value.

I wanted to use torch.where and torch.clamp but couldn’t manage to find a solution that works for me. Any ideas?

2

Something like this might work:

import torch
x = torch.randn(5,5)

# to handle 0 <= x <= 1e-4
x[torch.logical_and(x>=0, x<=1e-4)] = 1e-4 

# to handle -1e-4 <= x < 0
x[torch.logical_and(x<0, x>=-1e-4)] = -1e-4
3

Thank you. I didn’t know about torch.logical_and(). However, when I use this approach I get the following error in my forward() method:

RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.

4

I am not sure about your use case. But when you assign a hard-coded value for certain elements in a tensor, the gradients will be zero for those elements. I hope you are aware of this.

You can overcome the in-pace operation runtime error by cloning x before modifying it.

x = x.clone()

# to handle 0 <= x <= 1e-4
x[torch.logical_and(x>=0, x<=1e-4)] = 1e-4 

# to handle -1e-4 <= x < 0
x[torch.logical_and(x<0, x>=-1e-4)] = -1e-4
1 Like