Handling singular matrices when backprop

Hey guys!

I am currently running into the following problem:
I’ll get the
RuntimeError: inverse_cuda: For batch 0: U(1,1) is zero, singular U.
message, when computing lots of determinants during the forward pass. Do you know how to tackle this problem appropriately? I am aware of the fact that the matrices have to be non-singular in order to compute its gradients - but how to make sure that only non-singular matrices are considered when backwarding?

Thanks a lot in advance!

1 Like

I discovered that there are singular matrices when doing SVD decomposition - as part of doing PCA. I’m using torch.pca_lowrank(matrix,...) to do dimensional reduction. You can check whether the middle vector has zeros in it (singular values), e.g.

reduced, eigenvalues, _ = torch.pca_lowrank(matrix, q=4)
if not eigenvalues.prod().is_nonzero():
    raise Exception

Hope it helps.