How to implement Keras training in PyTorch

1

I just started learning PyTorch after using Keras exclusively for a couple of years. I really like how models in Keras are trained, and having nice qol features like model checking and early stopping. I was wondering if there’s a way to implement each of the following in PyTorch

  1. Reserving a portion of the training data as validation data so that the model predicts on the validation data at each epoch. Additionally, is there a way to randomize what data is put into the validation set

  2. Callbacks like early stopping, snapshot ensembling, and model checking

Also in the tutorial on training a classifier (https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html) and several other PyTorch implementations of nn architectures, there typically isn’t an activation at the end (i.e. sigmoid/softmax). Why is this the case? Is the activation function not needed at the end?

2

Have a look at Ignite. They have some utilities and abstractions which you may find useful.

You don’t need an activation function for certain loss functions.
nn.CrossEntropyLoss for example expects logits, so that you shouldn’t have any activation at the end of your model.
nn.NLLLoss on the other have expects log probabilities, so that you would have to add a nn.LogSoftmax as the last layer.
Have a look at the loss functions in the docs for more information.

This is mostly done for numerical stability reasons. E.g. calling log on softmax separately might not be stable.