Pytorch save execution without NaN values

1

Hey,

there is the possibility to check and replace Nan values in Pytorch using torch.isnan and torch.nan_to_num.

For larger models, one could always loop over all parameters to replace the NaN (and Inf) values.

Is there an option to automate these operations and replace NaN values directly when they are created (e.g. in a matrix multiplication)?

2

I haven’t seen an automated approach replacing NaNs but it would also be interesting to learn about your use case. E.g. replacing NaNs in the forward would not fix the reason these were created.