VGG16 for CIFAR10

Bjorn_Lindqvist · August 21, 2024, 12:27am

Hello fellow deep learners,

To learn more about image classification I have implemented VGG16 for CIFAR10 in PyTorch. My code works and the training converges. But the model is capable of ~95% accuracy whereas mine only reaches ~89%. So I wonder if anyone could take a loot at my training code to see what could be improved?

gist.github.com

https://gist.github.com/bjourne/855e22acc36ca6a7ffb958c6f2bb3eb0

vgg16_cifar.py

from itertools import islice
from pathlib import Path
from torch.nn import *
from torch.nn.functional import cross_entropy, mse_loss, one_hot
from torch.nn.init import constant_, kaiming_normal_, normal_
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
from torchinfo import summary
from torchtoolbox.transform import Cutout

This file has been truncated. show original

TIA!

wilhelm · August 21, 2024, 12:02pm

A learning rate of 0.1 is IMHO too much.
You should start with something like 1-e4.
Secondly you need a scheduler for your training.

Ok, I saw right now, that you are using already a CosineAnnealingLR.
I would something more conventional like a LinerLR or StepLR.