Training when encoder-decoder model

1

Hi,
This is my simplified model structure

class Encoder(nn.Module):
    def __init__(self, ...):
    ....

encoder = Encoder()

class Decoder(nn.Module):
    def __init__(self, encoder, ...):
        ...

Decoder net has encode model in it.
Then when i train this model, should I optimize both models like

encoder_optimizer = optim.SGD(encoder.parameters(), lr=learning_rate)
decoder_optimizer = optim.SGD(decoder.parameters(), lr=learning_rate)

or just decoder?

2

Print decoder.parameters() shapes to see what parameters, the Decoder contains. If it contains Encoder parameters as well, you may only optimize the Decoder.

1 Like
3

if you call the encoder in the decoder’s forward function, one optim is enough.