An ambiguous error when converting from Pytorch tensor to numpy

vision
1

Hi,

I have a PyTorch tensor of size torch.Size([49, 49]):

sw_bar_tensor = tensor([[ 1.1895e+01, 4.9031e-01, 3.8824e-01, …, 1.0092e-01,
3.0263e-02, 9.5312e-03],
[ 4.9031e-01, 1.2864e+01, -1.0768e+00, …, -9.6299e-02,
-7.1840e-03, 2.4902e-02],
[ 3.8824e-01, -1.0768e+00, 1.3478e+01, …, -3.1151e-02,
2.2860e-02, -1.0342e-03],
…,
[ 1.0092e-01, -9.6299e-02, -3.1151e-02, …, 1.7947e-01,
-7.7783e-03, 2.0708e-03],
[ 3.0263e-02, -7.1840e-03, 2.2860e-02, …, -7.7783e-03,
1.2594e-01, 3.5290e-03],
[ 9.5312e-03, 2.4902e-02, -1.0342e-03, …, 2.0708e-03,
3.5290e-03, 6.8233e-02]], device=‘cuda:0’, dtype=torch.float64)

I’m calculating its null space using scipy.linalg.null_space function, so I converted the tensor to numpy array as follows:

arr_numpy= torch.Tensor.numpy(sw_bar_tensor.detach().cpu())
or
arr_numpy = sw_bar_tensor.detach().cpu().numpy()

Calculate nullspace

null_space(arr_numpy)

it gives me an empty array:
Out[544]: array([], shape=(49, 0), dtype=float64)

however, have the same values of the tensor but stored in a numpy array from another function, i.e, it was created as a numpy array at the first place. when I calculate its null space using the same function null_space(a_nump) it gives me a result (not an empty array), it behaves normally.

Does anyone know the problem or am I missing something or this is a bug in Pytorch?

2

This has to do with the rcond condition of the scipy null_space function. If you do not provide rcond it takes a weird default

if rcond is None:
        rcond = numpy.finfo(s.dtype).eps * max(M, N)

Here is a working code

import torch
from scipy.linalg import null_space
sw_bar_tensor=torch.FloatTensor(49, 49).to('cuda')
arr_numpy=sw_bar_tensor.detach().cpu().numpy()
null_space(arr_numpy,rcond=1.)

Result

array([[-1.74963553e-01,  0.00000000e+00],
       [ 4.51028104e-16, -9.12652367e-16],
       [ 2.33284737e-01,  1.45436634e-15],
       [ 1.38777878e-16, -1.54846174e-16],
       [ 1.16642369e-01,  9.48298648e-16],
       [ 5.55111512e-16, -1.01050836e-15],
       [ 7.07767178e-16,  4.64086501e-16],
       [ 2.26207941e-15, -3.16227766e-01],
       [ 1.74963553e-01,  7.21644966e-16],
1 Like
3

Thank you @anantguptadbl!
I know that I can use rcond but in this case it will give me all orthogonal basis which I don’t want.

I just figured the problem, in case of anyone experiences this problem, the tensor sw_bar_tensor was of type of float64 and for some reason when you convert it to lower type (e.g., float 32, float 16) it will work.

4

Thanks for the update @Abdelpakey