Can I include an auto encoder training in my model and only use the bottleneck?

1

I have a problem where I want to get a compressed representation of some high-dimensional data. So normally, I would train an autoencoder to get that representation. But I don’t actually need the decoder, so is it possible for me to have the autoencoder training in my main loop?

For my AE, I’d want to use MSE loss and for my main model, I have a different loss function.

Is it possible to do it all at once or do I have to train my autoencoder separately?

2

You could split your autoencoder into the encoder and decoder part and then train all modules together.
Here is a dummy example:

encoder = MyEncoder()
decoder = MyDecoder()

z = encoder(data)
out = decoder(z)

loss_z = criterion_z(z, target_z)
loss_out = criterion_out(out, target_out)
loss = loss_z + loss_out
loss.backward()

Let me know, if that would work for you.

1 Like