Hello everyone,
I have a model in which I do this operation A = A_1/ torch.abs(A_1 + epsilon)
A_1 is a complex matrix, and I figured out that the division by torch.abs(A_1 + epsilon) prevent the model from converging, otherwise when omitting this division the model converge without problem. same issue goes for another tensor B that has been multiplied like this (1/(n*s))*B, where ‘n’ and ‘s’ are two real numbers.
Hi @AMIRA_BEDOUI,
The derivative of the function torch.abs(x) is ill-defined when x = 0 as the derivative w.r.t x is x / torch.abs(x) whereas the derivative of x w.r.t x is just 1. So, check the value of A_1 + epsilon throughout the convergence.