hi,all:
I meet a problem when I rewrite the network form keras
in keras, i use
optimizer = keras.optimizers.Adam(lr = 1e-4)
model.compile(optimizer=optimizer, loss=‘binary_crossentropy’, metrics=[‘accuracy’])
in keras,and use
optimizer = optim.Adam(model.parameters(),lr=0.0001)
in pytorch
but keras’s loss can keep decrease util 0.04,but in pytroch ,it will stuck at 0.2
I use same data in both and BCELoss in pytorch , how can I fix that problem?
thanks
Often the discrepancy is due to different parameter initializations.
Could you make sure to use the same torch.nn.init
methods for your layers as in your Keras model?
I use he_normal to initialize the layers in keras,so what should I use in pytorch?