PCA in Pytorch. I have written in lua

1

I have written a code in lua for PCA. Any one help to convert this code into Pytorch, please?

function unsup.pcacov(x)
local mean = torch.mean(x,1)
local xm = x - mean:expandAs(x)
local c = torch.mm(xm:t(),xm)
c:div(x:size(1)-1)
local ce,cv = torch.symeig(c,‘V’)
return ce,cv
end

2

This should translate pretty cleanly to python. Pytorch and regular torch have a very similar API.
In particular, if you’re not familiar with python syntax, here’s how you define a function:

def unsup.pacov(x):
    mean = torch.mean(x, 1)
    xm = x - mean.expand(x)
    # etc etc

Disclaimer: I haven’t tried the above code out but you can find the corresponding pytorch functions in the documentations page: http://pytorch.org/docs/master/torch.html

3

def pcacov(x)
mean = torch.mean(x,1)
xm = x - mean:expandAs(x)
c = torch.mm(xm:t(),xm)
c=c:div(x:size(1)-1)
ce,cv = torch.symeig(c,‘V’)
return ce,cv

But got syntax error

xm = xs - mean:expandAs(xs)
^
SyntaxError: invalid syntax

4

xm = xs - mean:expandAs(xs)

mean:expandAs is not working in pytorch

5

Another possible way to import numpy. But I am getting error as well. I am not sure how numpy is used with pytorch.
import numpy as np
def pca(x):
x = torch.from_numpy(np.cov(x))
ev_s , eig_s = torch.from_numpy(np.linalg.eig(x))
return ev_s , eig_s

7

def PCA(x):
x = x
xm = torch.mean(x)
xc = torch.matmul(torch.transpose(xm, 0, -1), xm)
es, vs = torch.symeig(xc, eigenvectors=True)
return es, vs

Getting the following error. Any idea, please. Thanks in advanced.

8

Haven’t checked the results for validity, but this might be a starter for you:

def PCA(x):
    xm = torch.mean(x, 1, keepdim=True)
    xc = torch.matmul(xm, torch.transpose(xm, 0, -1))
    es, vs = torch.symeig(xc, eigenvectors=True)
    return es, vs
9

I am getting the following error.

TypeError: Type Variable doesn’t implement stateless method symeig

10

I think torch.symeig is not differentiable, thus it’s not available for Variables.
You could use x.data to get the tensor of the Variable.

11

Is there any other functions in pytorch so that we can calculate eigenvectors?

12

Can I use numpy for calculating eigenvectors. But I think the problem is that x is not a numpy matrix. It’s a pytorch tensor. So how can I use numpy with pytorch?

import numpy as np
def pca(x):
x = torch.from_numpy(np.cov(x))
ev_s , eig_s = torch.from_numpy(np.linalg.eig(x))
return ev_s , eig_s

13

def pca(data, var_fraction):
""" principal components, retaining as many components as required to
retain var_fraction of the variance

Returns projected data, projection mapping, inverse mapping, mean"""
from numpy.linalg import eigh
u, v = eigh(numpy.cov(data, rowvar=1, bias=1))
v = v[:, numpy.argsort(u)[::-1]]
u.sort()
u = u[::-1]
u = u[u.cumsum()<u.sum()*var_fraction]
numprincomps = u.shape[0]
V = ((u**(-0.5))[:numprincomps][numpy.newaxis,:]*v[:,:numprincomps]).T
W = (u**0.5)[:numprincomps][numpy.newaxis,:]*v[:,:numprincomps]
return numpy.dot(V,data), V, W