I want to use latent vectors from my autoencoder (which provides with good reconstruction results) for classification problem, but I have troubles classifying my data so far. I tried following:

method №1

I have saved latent vectors as `.npy`

files and was running this classifier on them, but my model started to overfit very fast (giving me 80-90% accuracy on a train set, but 5-10% on validation), though I have 50k latent vectors. But at least it was good on the train set

method №2

I was trying to use saved autoencoder model with the same classifier for the same purpose (because to keep all the vectors as `.npy`

takes too much storage). So i loaded encoder part (and froze all layers with `param.requires_grad = False`

as I only need to produce latent vectors but not to train autoencoder) and put only classifier to optimization: `optim.SGD(clf.classifier.parameters(), lr=0.001, momentum=0.9)`

, I also added `clf.classifier.train()`

and `clf.encoder.eval()`

. But this model simply wouldn’t work - it just predicts mainly one class for all vectors in a batch that gives maximum 16% accuracy on train set

Where possibly did I do wrong? And what are the ways to make a classifier work?