How to deal with inf values when taking logs of tensors containing zeros

In my model, in the forward function, I have one tensor on which I first apply sigmoid and then log. However, the output of sigmoid might be considerably low. The return value of the forward function is :

def forward(dots, path, probs)
        apply_paths = dots * path 
        apply_sigmoid = torch.sigmoid(apply_paths)
        apply_log = -1*torch.log(apply_sigmoid)

       return torch.sum(probs*apply_log)

I’m getting a lot of time inf as a return value. The return value is the loss itself, and so the gradients are calculated based on this return value. I’m not sure if this is okay or not. Should I do anything of these inf values after taking log? Or do something about very small values from the sigmoid function?

You could clamp these invalid values or just use logsigmoid as seen here:

x = torch.tensor([-100., 100])
> tensor([-inf, 0.])
> tensor([-100.,   -0.])
1 Like