Pytorch Vs Keras - pytorch accuracy a lot lower

#1

I am trying to learn PyTorch just using a basic sequential model and comparing against keras - I am getting a widely different accuracy on the titanic dataset.

“”“model_2 = torch.nn.Sequential(torch.nn.Linear(7, 16),
torch.nn.ReLU(),
torch.nn.Linear(16, 16),
torch.nn.ReLU(),
torch.nn.Linear(16, 1),
torch.nn.Sigmoid())”""

Keras:
“”“model = Sequential()
model.add(Dense(16, activation=‘relu’, input_shape=(7,)))
model.add(Dense(16, activation=‘relu’))
model.add(Dense(1, activation=‘sigmoid’))”""

colab notebook:

Please be nice still learning!

#2

I can’t run the notebook, as the data is missing.
However, based on the code I’m not sure how the learning rate is set in Keras.
In PyTorch you are using lr=0.0001, while I guess Keras might be using their default of 0.001 as given here?
If so, change it in PyTorch to the same value.

Also, I would recommend to remove the sigmoid and use nn.BCEWithLogitsLoss for a better numerical stability.

#3

@ptrblck Thank you - I missed that I thought it was 0.0001 as a default and that has brought the accuracy up. Will test with Logits now! Thanks again!