PCA in Pytorch. I have written in lua

1

I have written a code in lua for PCA. Any one help to convert this code into Pytorch, please?

function unsup.pcacov(x)
local mean = torch.mean(x,1)
local xm = x - mean:expandAs(x)
local c = torch.mm(xm:t(),xm)
c:div(x:size(1)-1)
local ce,cv = torch.symeig(c,‘V’)
return ce,cv
end

2

This should translate pretty cleanly to python. Pytorch and regular torch have a very similar API.
In particular, if you’re not familiar with python syntax, here’s how you define a function:

def unsup.pacov(x):
    mean = torch.mean(x, 1)
    xm = x - mean.expand(x)
    # etc etc

Disclaimer: I haven’t tried the above code out but you can find the corresponding pytorch functions in the documentations page: http://pytorch.org/docs/master/torch.html

3

def pcacov(x)
mean = torch.mean(x,1)
xm = x - mean:expandAs(x)
c = torch.mm(xm:t(),xm)
c=c:div(x:size(1)-1)
ce,cv = torch.symeig(c,‘V’)
return ce,cv

But got syntax error

xm = xs - mean:expandAs(xs)
^
SyntaxError: invalid syntax

4

xm = xs - mean:expandAs(xs)

mean:expandAs is not working in pytorch

5

Another possible way to import numpy. But I am getting error as well. I am not sure how numpy is used with pytorch.
import numpy as np
def pca(x):
x = torch.from_numpy(np.cov(x))
ev_s , eig_s = torch.from_numpy(np.linalg.eig(x))
return ev_s , eig_s

7

def PCA(x):
x = x
xm = torch.mean(x)
xc = torch.matmul(torch.transpose(xm, 0, -1), xm)
es, vs = torch.symeig(xc, eigenvectors=True)
return es, vs

Getting the following error. Any idea, please. Thanks in advanced.

8

Haven’t checked the results for validity, but this might be a starter for you:

def PCA(x):
    xm = torch.mean(x, 1, keepdim=True)
    xc = torch.matmul(xm, torch.transpose(xm, 0, -1))
    es, vs = torch.symeig(xc, eigenvectors=True)
    return es, vs
1 Like
9

I am getting the following error.

TypeError: Type Variable doesn’t implement stateless method symeig

10

I think torch.symeig is not differentiable, thus it’s not available for Variables.
You could use x.data to get the tensor of the Variable.

1 Like
11

Is there any other functions in pytorch so that we can calculate eigenvectors?

12

Can I use numpy for calculating eigenvectors. But I think the problem is that x is not a numpy matrix. It’s a pytorch tensor. So how can I use numpy with pytorch?

import numpy as np
def pca(x):
x = torch.from_numpy(np.cov(x))
ev_s , eig_s = torch.from_numpy(np.linalg.eig(x))
return ev_s , eig_s

13

def pca(data, var_fraction):
""" principal components, retaining as many components as required to
retain var_fraction of the variance

Returns projected data, projection mapping, inverse mapping, mean"""
from numpy.linalg import eigh
u, v = eigh(numpy.cov(data, rowvar=1, bias=1))
v = v[:, numpy.argsort(u)[::-1]]
u.sort()
u = u[::-1]
u = u[u.cumsum()<u.sum()*var_fraction]
numprincomps = u.shape[0]
V = ((u**(-0.5))[:numprincomps][numpy.newaxis,:]*v[:,:numprincomps]).T
W = (u**0.5)[:numprincomps][numpy.newaxis,:]*v[:,:numprincomps]
return numpy.dot(V,data), V, W