Implementation of a Keras model doesn't converge

[ptrblck]

@Nbooo

As I understood, in Pytorch we need to set weight_decay parameter in our optimizer to achieve the same result, but I’m far from being confident here.

The weight_decay might be too aggressive in PyTorch, as it’ll add all parameters to the regularization term.
Have a look at this post to exclude batch norm params etc. (or just add the conv parameters to the regularization).