RuntimeError: svd_cuda: For batch 0: U(51,51) is zero, singular U

1

HI,
I want to apply PCA on model outputs. The code is:

def PCA_svd(x, k, center=True):
    n = x.size()[0]
    ones = torch.ones(n).view([n,1])
    h = ((1/n) * torch.mm(ones, ones.t()))# if center else torch.zeros(n*n).view([n,n])
    H = torch.eye(n) - h
    #H = H.cuda()
    X_center = torch.mm(H.double().cuda(), x.double().cuda())
    u, s, v = torch.svd(X_center)
    components = v[:k].t()
    #explained_variance = torch.mul(s[:k], s[:k])/(n-1)
    return components

The model output is composed of feature vectors of shape 512. So, we can assume the model output = torch.rand(50,512) where 50 is the batchsize.
Next I am using the function on these model outputs as:

feature = PCA_svd(model_output,128)

But, I am getting this error when I run the codes for a dataset of about 5000 images.

RuntimeError: svd_cuda: For batch 0: U(51,51) is zero, singular U.

Funny thing is that when I run it for a smaller dataset of lets say 3000 images, I don’t face this error.
I don’t understand it. Is it a bug? Or, is there something wrong in my PCA_svd function.

1 Like