PSD matrix inversion fails on GPU but not CPU

I have a square matrix that should be positive semi-definite (PSD). This is because, no matter what it’s value, I always modify it such that:

P = diag(diag(P)) + eye(I) * eps

Where eps is a small number. In other words, the matrix is both symmetric and has non-zero values along its diagonal. Then I try to invert this matrix. When I run it locally or even on my university’s head node machine, the code runs fine. But when I run it on GPUs, the code throws an error:

    beta = inv(mm(L, L.t()) + P)
RuntimeError: MAGMA getrf : U(2,2) is 0, U is singular at /opt/conda/conda-bld/pytorch_1524584710464/work/aten/src/THC/generic/THCTensorMathMagma.cu:413

So I can invert the matrix when running on a CPU but not on a GPU. I have no idea what might be causing this, and any help is appreciated.