I want to apply PCA on model outputs. The code is:
def PCA_svd(x, k, center=True): n = x.size() ones = torch.ones(n).view([n,1]) h = ((1/n) * torch.mm(ones, ones.t()))# if center else torch.zeros(n*n).view([n,n]) H = torch.eye(n) - h #H = H.cuda() X_center = torch.mm(H.double().cuda(), x.double().cuda()) u, s, v = torch.svd(X_center) components = v[:k].t() #explained_variance = torch.mul(s[:k], s[:k])/(n-1) return components
The model output is composed of feature vectors of shape 512. So, we can assume the model output = torch.rand(50,512) where 50 is the batchsize.
Next I am using the function on these model outputs as:
feature = PCA_svd(model_output,128)
But, I am getting this error when I run the codes for a dataset of about 5000 images.
RuntimeError: svd_cuda: For batch 0: U(51,51) is zero, singular U.
Funny thing is that when I run it for a smaller dataset of lets say 3000 images, I don’t face this error.
I don’t understand it. Is it a bug? Or, is there something wrong in my PCA_svd function.