AlexNet for cifar from " Understanding deep learning requires rethinking generalization"

Brando_Miranda · July 24, 2018, 3:59am

I wanted to train an AlexNet model on cifar with the architecture from:

“Understanding deep learning requires rethinking generalization”

Is the following the recommended way to do it:

bearpaw/pytorch-classification/blob/master/models/cifar/alexnet.py

'''AlexNet for CIFAR10. FC layers are removed. Paddings are adjusted.
Without BN, the start learning rate should be 0.01
(c) YANG, Wei 
'''
import torch.nn as nn


__all__ = ['alexnet']


class AlexNet(nn.Module):

    def __init__(self, num_classes=10):
        super(AlexNet, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=5),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=2, stride=2),
            nn.Conv2d(64, 192, kernel_size=5, padding=2),
            nn.ReLU(inplace=True),

This file has been truncated. show original

or is there a standard way to do this in pytorch for cifar?

Brando_Miranda · July 24, 2018, 4:00am

(note I don’t need pre-trained models)

aplassard · July 24, 2018, 10:37am

That looks like a reasonable way to initialize it, though your model is not exactly the same as AlexNet both in the layer definitions and the fully connected layers.

There is also torchvision which has a pre-setup version of the model - https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py

Brando_Miranda · July 24, 2019, 11:10pm

is the alexnet on that paper the alex net on Cifar 10 or Imagenet? (I want one for Cifar 10, I know the original one was on imagenet)

Brando_Miranda · July 24, 2019, 11:17pm

Crossposted:

https://qr.ae/TWnjL4