Hi,

I have an NN model which is written from stratch.This model has 2 neurons input layer, 1 hidden layer with 5 neurons and 2 output neurons. In hidden layer there is tanh activation. Cross entropy loss is being used. Weight 2 regularization aplied with lambda 0.01. Also weights are initialized with Xavier initialization. Learning rate is: 0.001. After 1000 epoch, this model has gives a Loss value of 0.093, and accuracy of 0.970 with below dataset.

x,y = sklearn.datasets.make_moons(n_samples=200,noise=0.05)

I decided to build the same model using Pytorch. However, I got very poor results in Pytorch. It seemed to be quiet strange.

Can you please check my Pytorch code and comment on why did I got poor results?

input_neurons = 2

hidden_neurons = 5

output_neurons = 2

learning_rate = 0.001

lambda_reg = 0.01

x,y = sklearn.datasets.make_moons(n_samples=200,noise=0.05)

x = torch.FloatTensor(x)

y = torch.LongTensor(y)

class FeedForward(torch.nn.Module):

def **init**(self, input_neurons, hidden_neurons, output_neurons):

super(FeedForward,self).**init**()

self.hidden = nn.Linear(input_neurons, hidden_neurons)

self.out = nn.Linear(hidden_neurons,output_neurons)

def forward(self, x):

x = self.hidden(x)

x = F.tanh(x)

x = self.out(x)

return x

def init_weights(m):

if type(m) == nn.Linear:

torch.nn.init.xavier_uniform(m.weight)

#m.bias.data.fill_(0.01)

network = FeedForward(input_neurons = input_neurons, hidden_neurons = hidden_neurons, output_neurons = output_neurons)

network.apply(init_weights)

optimizer = torch.optim.SGD(network.parameters(), lr = learning_rate,weight_decay=lambda_reg)

loss_function = torch.nn.CrossEntropyLoss()

for epoch in range(1000):

out = network(x)

loss = loss_function(out, y)

optimizer.zero_grad()

loss.backward()

optimizer.step()

```
if epoch % 50 == 0:
max_value, prediction = torch.max(out, 1)
predicted_y = prediction.data.numpy()
target_y = y.data.numpy()
accuracy = (predicted_y == target_y).sum() / target_y.size
print('Accuracy = {:.2f} Loss = {:.2f}'.format(accuracy,loss))
```