Apply linalg.solve if matrix is not singular, return arbitrary vector otherwise

1

I’m working on a way to reverse my neural network given some output, and to do that I’m using T.linalg.solve. I have a pre-bias (Bv) and a post-bias (Bw), so the output of any layer is:
W = (V-Bv)M + Bw
The thing is, M can be completely zeros in some cases, and in that case I want to return Bv instead of the solution of M. The best solution I’ve come up with is:
V = T.where( is_matrix_singular(M), Bv, T.linalg.solve(M,W))
The issue is that solve throws an error even if the result is never used. All of my other options suck, so I’m hoping someone knows more about this.
The first sucky solution is to loop through each batch, and then conditionally call linalg.solve.
The second sucky solution is to use T.where to replace the zero matrices with identity matrices, subtract W, and then add Bv if the matrix is singular.
The third sucky solution is to concatenate Bv to each matrix, and concatenate a one to the end of each W, then solve that.

Any better ideas?

2

Hi Stingpie!

Consider using lstsq(). It will give you a result for singular systems. (Note, the documentation
suggests that the cuda version might not handle singular systems. If you need cuda, you
might have to use pinv() – which I believe does work with cuda – and use the pseudoinverse
to “solve” your singular system.)

Best.

K. Frank