VGG16 for CIFAR10

Bjorn_Lindqvist · August 21, 2024, 12:27am

Hello fellow deep learners,

To learn more about image classification I have implemented VGG16 for CIFAR10 in PyTorch. My code works and the training converges. But the model is capable of ~95% accuracy whereas mine only reaches ~89%. So I wonder if anyone could take a loot at my training code to see what could be improved?

gist.github.com

https://gist.github.com/bjourne/855e22acc36ca6a7ffb958c6f2bb3eb0

vgg16_cifar.py

from itertools import islice
from pathlib import Path
from torch.nn import *
from torch.nn.functional import cross_entropy, mse_loss, one_hot
from torch.nn.init import constant_, kaiming_normal_, normal_
from torch.optim import SGD
from torch.optim.lr_scheduler import CosineAnnealingLR
from torch.utils.data import DataLoader
from torchinfo import summary
from torchtoolbox.transform import Cutout

This file has been truncated. show original

TIA!

wilhelm · August 21, 2024, 12:02pm

A learning rate of 0.1 is IMHO too much.
You should start with something like 1-e4.
Secondly you need a scheduler for your training.

Ok, I saw right now, that you are using already a CosineAnnealingLR.
I would something more conventional like a LinerLR or StepLR.

Bjorn_Lindqvist · September 26, 2024, 11:11am

After tweaking a bit I got 93.6%. Better, but still not very impressive.

aavraham · November 26, 2024, 3:28pm

Hi Bjorn, I would like to take part in the efforts of improving the accuracy. I would appreciate if you could provide me with access to your github repo and maybe collaborate together.

Bjorn_Lindqvist · January 24, 2025, 10:43am

Hello Aviv. My low performance was due to insufficient data augmentation. With better augmentation I’m getting way better results. All my code is here: GitHub - bjourne/ml-stuff