Hallo. I’m just starting with PyTorch.
I have built the following program in python and I want to translate it to PyTorch so I can compare the result but I still haven’t found how to do it. I’m currently attenting a course for PyTorch but it’s really slow and I need some help. Can anyone tell me how to write this code using PyTorch?
It’s the code in the paper Deep Learning: An Introduction for Applied
Mathematicians - Catherine F. Higham, Desmond J. Higham but I have translated it from MATLAB to Python
P.S. Thank you everyone in advance for your time and sorry if it’s against the rules of the Forum. Please delete it if it is.
Also if you have any suggestion about Courses or YouTube videos/series that you found helpful while you were getting starting with PyTorch please let me know.
import numpy as np
from numpy.linalg import norm as norm
np.random.seed(5000)
def activate(x,W,b): #Evaluates sigmoid function. x is the input vector, y is the output vector W contains the weights, b contains the shifts. The i-th component of y is activate((Wx+b)_i) where activate(z) = 1/(1+exp(-z))
z=W@x+breturn 1/(1+np.exp(-z))
def cost(W2,W3,W4,b2,b3,b4):
costvec = np.zeros(10)
for i in range(10):
x=[x1[i],x2[i]]
a2=activate(x,W2,b2)
a3=activate(a2,W3,b3)
a4=activate(a3,W4,b4)
costvec[i] = norm(y[:,i]-a4,2)
costvec[i]
costval = norm(costvec,2)**2
return costval
x1 = np.array([0.1,0.3,0.1,0.6,0.4,0.6,0.5,0.9,0.4,0.7])
x2 = np.array([0.1,0.4,0.5,0.9,0.2,0.3,0.6,0.2,0.4,0.6])
y=np.array([ [1,1,1,1,1,0,0,0,0,0],[0,0,0,0,0,1,1,1,1,1] ])
W2=np.random.random((2,2))
W3=np.random.random((3,2))
W4=np.random.random((2,3))
b2=np.random.random(2)
b3=np.random.random(3)
b4=np.random.random(2)
#forward and back propagate
eta = 0.05 #learning rate
Niter = 10**6 #number of SG iterations
savecost = np.zeros(Niter) #Value of cost at each iteration
for counter in range(Niter):
k=np.random.randint(0,10) #choose a training point at random
x=np.array([x1[k],x2[k]])
#forward pass
a2 = activate(x,W2,b2)
a3 = activate(a2,W3,b3)
a4 = activate(a3,W4,b4)
#backward pass
delta4 = a4*(1-a4)*(a4-y[:,k])
delta3 = a3*(1-a3)*(W4.T@delta4)
delta2 = a2*(1-a2)*(W3.T@delta3)
#gradient step
W2 = W2 - eta*delta2@x
W3 = W3 - eta*delta3.reshape(3,1)@a2.reshape(1,2)
W4 = W4 - eta*delta4.reshape(2,1)@a3.reshape(1,3)
b2 = b2 - eta*delta2
b3 = b3 - eta*delta3
b4 = b4 - eta*delta4
newcost = cost(W2,W3,W4,b2,b3,b4)
savecost[counter]=newcost
print(newcost , 'i=',counter)
import matplotlib.pyplot as plt
iterr=[i for i in range(counter+1)]
plt.ylim([10**(-4),10])
plt.xlim([0,Niter])
plt.semilogy(iterr,savecost)
plt.show()
