Why is forward pass generating nan values?

1 
def forward(self, x):

        hidden_inputs = self.hidden(x)

        hidden_outputs = self.sigmoid(hidden_inputs)

        final_inputs = self.output(hidden_outputs)

        final_outputs = self.softmax(final_inputs)

        return final_outputs


torch.isnan(self.features).any() # false
label_predict = self.forward(self.features)
torch.isnan(label_predict).any() # true

I checked, and it’s the nn.linear causing the output to be nan. Why is that and how can I fix that?

2

Could you check the stats of the input tensor as well as the parameters of the linear layer, which is causing this issue?
E.g. if your input contains Infs or very large values in their magnitude, the result might overflow and could be set to NaN in further operations.

3

Thanks. I had checked for NaN values in preprocessing but not for Inf values.