Train decoder only

Hello;

I am new to PyTorch and wondering how can you freeze an encoder and train a decoder only for seq-to-seq.

Thanks!!!

You could set requires_grad(False) on the encoder or detach the tensor before passing it to the decoder.
Assuming your model has members named encoder and decoder you could try the following:

for param in model.encoder.parameters():
    param.requires_grad_(False)
1 Like