I have a covariance matrix (symmetric positive definite) and hence I expected that symeig would give me unique eigen values. But it is giving repeated eigen values and this creates backward pass error in my custom loss function. I tried increasing the precision by changing from float32 to float64. However, the problem still persists. Can you please let me know how to solve the issue?
Out of curiosity, how do you ensure that the eigenvalues are unique? The positive definitiveness only ensure that they’re non-zero right?
Well. I am not an expert in linear algebra but I read that covariance matrices are diagonalisable and hence have unique eigen values. I might be wrong. But then again, even if the eigen values are not unique, why symeig should give such an error? Also, eig does not have a backward function.
What is the error you get from symeig? It should work fine for repeated eigenvalues.
I am using the autograd anomaly detector and it gave the following error “Function ‘SymeigBackward’ returned nan values in its 0th output.” Looking at the symeig function description, I saw that such nan values in the backward pass can happen when there are non-unique eigen values. " Extra care needs to be taken when backward through outputs. Such operation is really only stable when all eigenvalues are distinct. Otherwise,
NaN can appear as the gradients are not properly defined."
Ho thanks for the reference, I didn’t knew that.
I guess you’ll need to do some extra processing of your input matrix to ensure all the eigenvalues are distinct. But I’m not sure how to do that