# ReEig Layer
for i in range(features.shape):
for i in range(len(s_f)):
s_f_convert = torch.stack(s_f_convert)
result = torch.stack(result)
Gradients (“backward pass”) are not implemented for torch.eig(). If your features matrix is symmetric
(or can naturally be made symmetric), you can use torch.symeig(), otherwise you will have to rethink
your loss function.
Extra care needs to be taken when backward through
outputs. Such operation is really only stable when
all eigenvalues are distinct. Otherwise, NaN can
appear as the gradients are not properly defined.
Let me just say that there is nothing fundamentally improper
about using eigenvalues (or eigenvectors) as ingredients in
your loss function, but eigenvalues can be a little tricky, both
in terms of how they depend mathematically on your inputs,
and sometimes the numerical stability of the algorithms used
to calculate them. So you should think through carefully how
and why you’re using them.
Well, if you’re getting “none” gradients during training, there’s
almost certainly a bug somewhere, more likely in your code,
but possibly, though less likely, in pytorch.
If you don’t find the bug fairly quickly just looking through your
code, then more orderly debugging is called for. A common
approach is to break thing up into pieces to try and isolate the
As I suggested in my previous post, you might try just seeing
if you can get a gradient out of symeig. My suggestion would
be that you post a short, complete, runnable pytorch script
that pumps a tensor (with requires_grad set) through symeig and somehow calculates a scalar (single number)
from that. Then call backward() on that scalar and see if
your input tensor has good gradients or just “none”.
If you post the script and your results, forum participants can
not only look at it, but run it for themselves, and maybe they
will see something or have ideas about what is going on.