Creating NN with pytorch and assign weights and biases manually

1

Hi,
I want to create a simple neural network using pytorch with one input neuron and two neurons in the hidden layer and one neuron in the output layer which activation function for the hidden layer and the output layer is f(x)=x^2 (we have 1 hidden layer). Take the number 5 as an input and the weights of the first layer are W1=0.06 and W2=2.03 and its bias is b1=1.87. The weights of the second layer are W3=0.06 and W4=2.03 and its bias is b2=3.29. The output should be 12.1002. I don’t want to train this NN. Can anyone guide me on this please?
Thank you very much.

2

You can create the modules and assign the custom weight and bias values in a with torch.no_grad() context.

3

Thank you for your reply.
I read your post about setting the weights and biases manually but i can’t apply this in my network. Could you please write the code?

4

Could you post your model definition and the approach you’ve used so far so I could add the missing parts?

5

I had generated the neural network with custom weights, biases and activation function in python before, and I wanted to calculate the value of the NN’s output when input equals 5.
Now I want to do same procedure in pytorch but I can’t write the code.
This is my python code which I generated before:
import numpy as np
def activation_func(t):
return t**2
num_hidden=2
t = np.array([
[0],
[1],
[1.5],
[5],
[0.7],
[0.2],
])
u = np.array([[0, 13, 15.5, 10.5, 11.5, 9]]).T
#defining weights and biases
b1 = 2
b2 = 3
hidden_weights = np.array([[0.2, 0.3]])
output_weights = np.array([[1.5], [2.5]])
hidden_layer = np.dot(t, hidden_weights)
hidden_layer_outputs = activation_func(hidden_layer + b1)
output_layer = np.dot(hidden_layer_outputs, output_weights)
u_hat = activation_func(output_layer + b2)
print(u_hat)

and this is the pytorch code I tried to write it with your guidance in previous posts but I can’t change the activation functions and I don’t know how can I apply the hidden_weights and output_weights and biases to the network.
import numpy as np
import torch
import torch.nn as nn
X = np.array([
[0],
[1],
[1.5],
[5],
[0.7],
[0.2],
])
y = np.array([[0, 13, 15.5, 10.5, 11.5, 9]]).T

X = torch.tensor(X, dtype=torch.float32)
y = torch.tensor(y, dtype=torch.float32)

class PimaClassifier(nn.Module):
def init(self):
super().init()
self.hidden1 = nn.Linear(1, 2)
self.act1 = nn.ReLU()
self.output = nn.Linear(2, 1)
self.act_output = nn.Sigmoid()

def forward(self, x):
    x = self.act1(self.hidden1(x))
    x = self.act_output(self.output(x))
    return x

model = PimaClassifier()
hidden_weights = np.array([[0.2, 0.3]])
output_weights = np.array([[1.5], [2.5]])
bias = np.array([2., 3.])
with torch.no_grad():
model.weight = nn.Parameter(torch.from_numpy(hidden_weights).float())
model.weight = nn.Parameter(torch.from_numpy(output_weights).float())

print(model(X)[3])

6

Hi. I want to create a neural network with 1 neuron in input layer, 2 neurons in hidden layer
and 1 neuron in output layer, the activation function for both hidden and output layer is
f(x)=x**2. the weights and bias of hidden layer are [0.2, 0.3] and 2 respectively.
the weights and bias of output layer are [1.5, 2.5] and 3 respectively. I want to calculate
the output of the neural network with input value of 5. it should be obtain 2220.765625.
I wrote this code but I can’t obtain true answer.
could you please correct this code? I don’t want to train the network.

import torch
import torch.nn as nn

input_size = 1

class NeuralNet(nn.Module):
def init(self):
super(NeuralNet, self).init()

    self.fc1 = nn.Linear(in_features=input_size, out_features=2)
    self.output = nn.Linear(in_features=2, out_features=1)

    self.weights_initialization()

def forward(self, x):        
    out = self.fc1(x)       
    out = out ** 2          
    output = self.output(out)            
    return output**2  

def weights_initialization(self):
    with torch.no_grad():
        self.fc1.weight.copy_(torch.tensor([[0.2, 0.3]], requires_grad=False))
        self.fc1.bias.copy_(torch.tensor([2], requires_grad=False))  
        self.output.weight.copy_(torch.tensor([[1.5, 2.5]], requires_grad=False))
        self.output.bias.copy_(torch.tensor([3], requires_grad=False))

net = NeuralNet()
input_data = torch.rand(5, input_size)
output = net(input_data)
print(output)

7

You are creating 5 random values in your current code as the input to your model while it seems you want to pass 5 as the actual value to it.

8

You are right but I don’t know how to implement 5 as input in neural network.

9

x = torch.tensor([[5.]]) should work.

10

thank you for your answer. i tried it but code doesn’t work. i received this error.
could you please check my code?
in weights_initialization
self.fc1.weight.copy_(torch.tensor([[0.2, 0.3]], requires_grad=False))
RuntimeError: output with shape [2, 1] doesn’t match the broadcast shape [2, 2]

11

Change to:

self.fc1.weight.copy_(torch.tensor([[0.2], [0.3]], requires_grad=False))
12

Thank you for your help, but it doesn’t work. The problem is due to output shape.
The output shape should be 11 but I don’t know why its shape becomes 12.
This is the error:
RuntimeError: output with shape [1, 2] doesn’t match the broadcast shape [2, 2]

13

Might be your fc1.bias only has 1 value and should have 2.

Anyways, it has something to do with how you’re setting the parameters. Before setting them, make sure you have the right shapes by printing their sizes out and again after you set them.

for param in model.parameters():
    print(param.size())
14

Thank you very much. I will try it.