Are you trying to have a larger effective learning rate for the decoder of your model? In that case multiple param groups would be more suitable?
opt = Adam([{'params' : model.encoder.parameters(), 'lr' : encoder_lr}, {'params' : model.decoder.parameters(), 'lr' : decoder_lr}])