Hi, I am wondering if there is anything wrong with my training as it achieves very high accuracy from the starting itself? The logs for training on CIFAR10 looks like this as below-
Files already downloaded and verified Files already downloaded and verified Epoch 000: Training Loss: 64.2905438385933 | Training Accuracy: 91.71 Total Test Loss: 2.600952295586467 | Test Accuracy: 99.82 Saving... Epoch 001: Training Loss: 1.0865276760887355 | Training Accuracy: 99.81 Total Test Loss: 3.7665871270000935 | Test Accuracy: 99.92 Epoch 002: Training Loss: 0.18417388796842715 | Training Accuracy: 99.986 Total Test Loss: 1.7079168632626534 | Test Accuracy: 99.96 Epoch 003: Training Loss: 0.09753012470901012 | Training Accuracy: 99.994 Total Test Loss: 1.410150108858943 | Test Accuracy: 99.96
The good part is, after downloading the model and testing on examples it still gives the same accuracy. Still, I suspect if there is anything basic that I might be missing and all this accuracy is because of an error.
The model is stacked model of an autoencoder and a classifier and involves a custom loss for autoencoder including MSE loss and Cross-Entropy loss for Classifier.
I am looking for any suggestions or checks you may have to advise to understand the correctness of my pipeline.
for epoch in range(EPOCH+1): global best_train_acc running_loss, train_loss, correct, total = 0.0, 0, 0, 0 autoencoder.train() for i, (inputs, labels) in enumerate(trainloader): inputs, labels = inputs.to(device), labels.to(device) encoded, decoded, _, _, _ = autoencoder(inputs, labels) loss_ae = custom_loss_fn() output_classifier = classifier(encoded) loss_classifier = ce(output_classifier, labels) # classifier loss optimizer.zero_grad() optimizer_classifier.zero_grad() loss_ae.backward(retain_graph=True) loss_classifier.backward() optimizer.step() optimizer_classifier.step() train_loss += loss_classifier.item() _, predicted = output_classifier.max(1) total += labels.size(0) correct += predicted.eq(labels).sum().item()
and testing in the same way just I add;