I have a Question in my code. (MLP with binary classification

1

Hello.
I have a question in my code.
My code has two input parameter, two hidden layer and one output parameter.
I designed to output 0 or 1.
Is this a problem in my code?
Because, accuracy is very low.
Thank you, reading my code.

import torch
import torch.nn as nn
import torch.optim as optim
from torch.nn import functional as F
import numpy as np
import matplotlib.pyplot as plt
from torch.autograd import Variable
torch.manual_seed(1)

epoch = 10000
learning_rate = 0.1
input_dim = 2
hidden1_dim = 5
hidden2_dim = 3
output_dim = 1

data = np.loadtxt(‘Data_Set.csv’, delimiter=’,’)
x_data = data [:,:2]
y_data = data [:,2]

X_data = []
Y_data = []
for i in range(0, len(y_data)):
x = x_data[i]
y = y_data[i]
#print(x, “->”, y)
X_data.append(x)
Y_data.append(y)

X = torch.FloatTensor(np.array(X_data))
Y = torch.FloatTensor(np.array(Y_data))
fc1 = nn.Linear(input_dim, hidden1_dim)
fc2 = nn.Linear(hidden1_dim, hidden2_dim)
fc3 = nn.Linear(hidden2_dim, output_dim)
relu = nn.ReLU()
sigmoid = nn.Sigmoid()

model = torch.nn.Sequential(fc1, sigmoid, fc2, sigmoid, fc3, sigmoid)
criterion = torch.nn.BCELoss(size_average=True)
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

for step in range(epoch):
optimizer.zero_grad()
hypothesis = model(X)
cost = criterion(hypothesis, Y)
cost.backward()
optimizer.step()

if step % 100 == 0:
    print(step, cost.data.numpy())

predicted = (model(X).data >= 0.5).float()
accuracy = (predicted == Y.data).float().mean()
print("\nHypothesis: ", hypothesis.data.numpy(), "\nCorrect: ", predicted.numpy(), "\nAccuracy: ", accuracy*100, “%”)

2

Hello,

It is better to use ReLU activation function between hidden layers, and use sigmoid at last to generate probability map.

model = torch.nn.Sequential(fc1, relu, fc2, relu, fc3, sigmoid)

PS: you could format your code snippet with three `

3

Thanks for your advice.
And I tried “model = torch.nn.Sequential(fc1, relu, fc2, relu, fc3, sigmoid)”
However, accuracy is still low… :frowning:

4

Hi, I think your learning rate is too big and it’s better to decrease it to 1e-4.

5

Hello. Thanks for your advice.
but, I decreased learning rate low, my output_prediction is only 0… :joy: