Hallo. I’m just starting with PyTorch.
I have built the following program in python and I want to translate it to PyTorch so I can compare the result but I still haven’t found how to do it. I’m currently attenting a course for PyTorch but it’s really slow and I need some help. Can anyone tell me how to write this code using PyTorch?
It’s the code in the paper Deep Learning: An Introduction for Applied
Mathematicians - Catherine F. Higham, Desmond J. Higham but I have translated it from MATLAB to Python
P.S. Thank you everyone in advance for your time and sorry if it’s against the rules of the Forum. Please delete it if it is.
Also if you have any suggestion about Courses or YouTube videos/series that you found helpful while you were getting starting with PyTorch please let me know.
import numpy as np from numpy.linalg import norm as norm np.random.seed(5000) def activate(x,W,b): #Evaluates sigmoid function. x is the input vector, y is the output vector W contains the weights, b contains the shifts. The i-th component of y is activate((Wx+b)_i) where activate(z) = 1/(1+exp(-z)) z=W@x+breturn 1/(1+np.exp(-z)) def cost(W2,W3,W4,b2,b3,b4): costvec = np.zeros(10) for i in range(10): x=[x1[i],x2[i]] a2=activate(x,W2,b2) a3=activate(a2,W3,b3) a4=activate(a3,W4,b4) costvec[i] = norm(y[:,i]-a4,2) costvec[i] costval = norm(costvec,2)**2 return costval x1 = np.array([0.1,0.3,0.1,0.6,0.4,0.6,0.5,0.9,0.4,0.7]) x2 = np.array([0.1,0.4,0.5,0.9,0.2,0.3,0.6,0.2,0.4,0.6]) y=np.array([ [1,1,1,1,1,0,0,0,0,0],[0,0,0,0,0,1,1,1,1,1] ]) W2=np.random.random((2,2)) W3=np.random.random((3,2)) W4=np.random.random((2,3)) b2=np.random.random(2) b3=np.random.random(3) b4=np.random.random(2) #forward and back propagate eta = 0.05 #learning rate Niter = 10**6 #number of SG iterations savecost = np.zeros(Niter) #Value of cost at each iteration for counter in range(Niter): k=np.random.randint(0,10) #choose a training point at random x=np.array([x1[k],x2[k]]) #forward pass a2 = activate(x,W2,b2) a3 = activate(a2,W3,b3) a4 = activate(a3,W4,b4) #backward pass delta4 = a4*(1-a4)*(a4-y[:,k]) delta3 = a3*(1-a3)*(W4.T@delta4) delta2 = a2*(1-a2)*(W3.T@delta3) #gradient step W2 = W2 - eta*delta2@x W3 = W3 - eta*delta3.reshape(3,1)@a2.reshape(1,2) W4 = W4 - eta*delta4.reshape(2,1)@a3.reshape(1,3) b2 = b2 - eta*delta2 b3 = b3 - eta*delta3 b4 = b4 - eta*delta4 newcost = cost(W2,W3,W4,b2,b3,b4) savecost[counter]=newcost print(newcost , 'i=',counter) import matplotlib.pyplot as plt iterr=[i for i in range(counter+1)] plt.ylim([10**(-4),10]) plt.xlim([0,Niter]) plt.semilogy(iterr,savecost) plt.show()