CNN Computationally expensive

class ConvNet(nn.Module):
    def __init__(self, num_classes=2):
        super(ConvNet, self).__init__()
        self.layer1 = nn.Sequential(
            nn.Conv2d(1, 16, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2))
        self.layer2 = nn.Sequential(
            nn.Conv2d(16, 32, kernel_size=5, stride=1, padding=2),
            #nn.BatchNorm2d(64),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=5, stride=0))
        
        self.drop_out = nn.Dropout()

        self.fc1 = nn.Linear(15488, 152)
        self.fc2 = nn.Linear(152,2)
        
    def forward(self, x):
        x.float()
        out = self.layer1(x)
        out = self.layer2(out)
        out = out.reshape(out.size(0), -1)
        out = self.drop_out(out)
        out = self.fc1(out)
        out = self.fc2(out)
       
        return out

This architecture is so Computationally expensive and requires much memory , any recommendation to Minimize this computation and save the performance.
[ my model is take 224*224 picture and return 2 classes : P , N ]
thanks in advance

Dont use a fc 15000-200 but get a single 1d vector of fetures (32x1x1)using a adaptative max pool

sorry but i dont understand you , can you write your changes in the architecture and post it
thanks in advance

class ConvNet(nn.Module):
    def __init__(self, num_classes=2):
        super(ConvNet, self).__init__()
        self.layer1 = nn.Sequential(
            nn.Conv2d(1, 16, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2))
        self.layer2 = nn.Sequential(
            nn.Conv2d(16, 32, kernel_size=5, stride=1, padding=2),
            #nn.BatchNorm2d(64),
            nn.ReLU(),
            # nn.MaxPool2d(kernel_size=5, stride=0)) # Too much parameters after this to use a relu
            nn.AdaptiveMaxPool2d(1)
        
        self.drop_out = nn.Dropout()

        self.fc1 = nn.Linear(32, 16)
        self.relu = nn.ReLu()
        self.fc2 = nn.Linear(16,2)
        
    def forward(self, x):
        x.float()
        out = self.layer1(x)
        out = self.layer2(out)
        out = out.reshape(out.size(0), -1)
        out = self.drop_out(out)
        out = self.fc1(out)
        out = self.relu(out)
        out = self.fc2(out)
       
        return out

is this right syntax not a typo ?

Yep, Basically you are making a pooling to get spatial size = 1. This way instead of having 15k parameters you will have just 1 parameter per channel. Then, fully connected is feasible. Besides you need a non-linearity between both FC. Otherwise it’s like having a single FC (as FC are linear layers)

ok i will try it and i will tell you about the performance
thanks

Ypu can try adaptative average pooling instead of max pooling, depending on the final size of your tensor.

can you explain more details about that ?

Hmmm. Max pooling propagates the maximum activation of each channel. Average pooling propagate a mean channel.
You can Google advantages and disadvantages for example this qa
https://www.quora.com/What-is-the-benefit-of-using-average-pooling-rather-than-max-pooling

1 Like

can you give me a syntax for average pooling or a source to it ?
thanks for your interest