How to increase Macro f1 score?

vision
1

I’m working on a 6-class animal footprint/paw classification task. The dataset has 3,000 training images and 300 validation images. I’m using ResNet-18 with ImageNet pretrained weights, ImageNet normalization, Adam (lr = 1e-3), batch size 32, 20 epochs, and Cross-Entropy loss.

The issue: by around epoch 10 I reach 97% training accuracy, but on validation both accuracy and macro F1 stay around 77% and don’t really improve.

Here’s the model head:

model_ft = models.resnet18(weights=‘IMAGENET1K_V1’)
num_ftrs = model_ft.fc.in_features
model_ft.fc = nn.Sequential(
nn.Linear(num_ftrs, 256),
nn.ReLU(),
nn.Linear(256, num_classes)
)

Here are the augmentations:

train

transforms.RandomResizedCrop(224, scale=(0.7, 1.0)),
transforms.RandomHorizontalFlip(0.5),
transforms.RandomRotation(15),
transforms.ColorJitter(0.3, 0.3, 0.3, 0.1),
transforms.RandomAffine(degrees=0, translate=(0.1, 0.1), scale=(0.9, 1.1))

val

transforms.Resize(256),
transforms.CenterCrop(224)

I’m honestly not sure what to try next in this situation—any advice would be appreciated!