SVD grad values are 'NaN'

Hi all,
when dealing with matrices with number of rows much greater than number of columns (X10)
I’m receiving NaN grads for SVD

when I use matrix size of 256X15 everything is fine
when I use 4096X100 I get NaN (this is the size I need)
any idea why?

I have a similar problem. I also get NaN in the gradient. My matrix has the size of [2028, 64]. Please post the solutions if you fixed it.

There are a few solutions

1- You can add small amount of noise to your matrix

2- Since nan are for numbers that are very large or small you can use torch.nan_to_num

3- you can use linalg.svd(X.cpu().detach().numpy(), full_matrices=False, lapack_driver=“gesvd”), which I believe still have problem but better than torch.svd

The gradient of an SVD decomposition is only defined if the singular values aren’t arbitrarily close together. There’s more detail about this shown in the documentation here.