Sklearn model to Pytorch model

1

Hi! I am new to pytorch and I am trying to recreate the same simple neural net model as scikit-learn.neural_network.

Sklearn model

mlp = MLPRegressor()
mlp.fit(x,y)

_out: MLPRegressor(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,_
_       beta_2=0.999, early_stopping=False, epsilon=1e-08,_
_       hidden_layer_sizes=(100,), learning_rate='constant',_
_       learning_rate_init=0.001, max_iter=200, momentum=0.9,_
_       nesterovs_momentum=True, power_t=0.5, random_state=None,_
_       shuffle=True, solver='adam', tol=0.0001, validation_fraction=0.1,_
_       verbose=False, warm_start=False)_

mlp.score(x,y)

_out: 0.950355451344164_

And here the code of the pytorch nn model which should be the same but gives me a score between 0.17 - 0.45:

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.utils.data as utils_data
from torch.autograd import Variable

inputs = Variable(torch.Tensor(x.values))
targets = Variable(torch.Tensor(y.values))

class MLP(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(MLP, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, output_size)
        
    def forward(self, x):
        out = self.fc2(F.relu(self.fc1(x)))
        return out

input_size = inputs.size()[1]
hidden_size = 100
output_size = 1
num_epoch = 200
learning_rate = 1e-3

model = MLP(input_size = input_size, hidden_size = hidden_size,
            output_size = output_size)

optimizer = torch.optim.Adam(model.parameters(), lr = learning_rate, weight_decay=1e-4)
loss_fct = nn.MSELoss()

training_samples = utils_data.TensorDataset(inputs, targets)
data_loader_trn = utils_data.DataLoader(training_samples, batch_size=200, drop_last=False, shuffle=True)

#train
for epoch in range(num_epoch):
        cum_loss = 0
        for batch_idx, (data, target) in enumerate(data_loader_trn):

            tr_x, tr_y = data.float(), target.float()

            pred = model(tr_x)
            loss = loss_fct(pred, tr_y)
        
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            cum_loss += loss.item()
            
        if epoch % 10 == 0:
            print ('Epoch [%d/%d], Loss: %.4f' 
                      %(epoch+1, num_epoch, cum_loss))


final_prediction = model(inputs)
final_pred_np = final_prediction.clone().detach().numpy()

np.corrcoef(final_pred_np.squeeze(), targets)[0,1]

_out: 0.17246591146678705_

additional informations: x.shape() = (18008, 2784)

Could you help me find my mistakes? Why this pytorch model doesn’t do a score of 0.95 as the sklearn model ?

Sorry if I have wrongly implemented this topic it is actually my first post on this website =/

Thanks a lot!

2

Please help ! =) I Don’t know what to do

3

You have a small mistake in your model, i.e. nn.MSELoss expects the model output and the target to have the same shape. If that’s not the case the tensors will be broadcasted if possible.
In your case tr_y is has only one dimension ([100]), while pred has two ([100, 1]).
Change the loss to loss = loss_fct(pred, tr_y.unsqueeze(1)) and you’ll get approx. the same performance (and coefficient of determination R^2) for both models.

PS: I’ve formatted your post to easily copy your code. You can add code snippets using three backticks ``` :wink:

4

Wow perfect answer =) Thanks a lot! it’s working just fine now.

And thanks also for the snippets tips, my bad :slight_smile:

1 Like