How do I optimize specific model of parameters?


(Km4342) #1

I train Autoencoder model, and I want to train only encoder model.

class AE(nn.Module):
    def __init__(self, indim, zdim):
        super(AE, self).__init__()
        self.encoder = nn.Sequential(
            nn.Linear(indim, zdim),
            nn.ReLU(True)
        )
        self.decoder = nn.Sequential(            
            nn.Linear(zdim, indim),
            nn.ReLU(True)
        )
        # initialize weights
        self.apply(init_weights)

    def forward(self, x):
        z = self.encoder(x)
        x_tilde = self.decoder(z)

        return x_tilde

in optimizer, I take follows,

optimizer = optim.Adam(model.encoder.parameters(), lr=1e-3)

but I can’t optimize. Loss function become bigger… How do I only train encoder?


(Vahid Mirjalili) #2

How is your loss function defined? If the parameters of the decoder are fixed from scratch, it will be very difficult to reconstruct input data solely based on encoder.

Also, you may consider removing the last ReLU activation in the decoder. The ReLU is removing all the negative values in the output, and setting them to zero, so that automatically adds a constant value to the loss value.