Want to use support vector regression in the last fullyconnected layer of my CNN model

1

Hello Friends,

I am trying to build a CNN model

that takes image as input and produces a 6x1 vector of continuous values. Here is what I have wrote

class MultiLabelNN(nn.Module):
def init(self):
super(MultiLabelNN, self).init()
self.conv1 = nn.Conv2d(3,64, 5)
self.pool = nn.MaxPool2d(2,2)
self.conv2 = nn.Conv2d(64, 128, 5)
self.conv3 = nn.Conv2d(128, 256, 5)
self.conv4 = nn.Conv2d(256,320,5)
self.fc1 = nn.Linear(250880,2048)
self.fc2 = nn.Linear(2048, 1024)
self.fc3 = nn.Linear(1024, 512)
self.fc4 = nn.Linear(512, 6)
def forward(self, x):
x = self.conv1(x)
x = nn.ReLU(x)
x = self.pool(x)
x = self.conv2(x)
x = nn.ReLU(x)
x = self.pool(x)
x = self.conv3(x)
x = nn.ReLU(x)
x = self.pool(x)
x = self.conv4(x)
x = nn.ReLU(x)
x = self.pool(x)
x = x.view(-1, 250880)
x = self.fc1(x)
x = self.fc2(x)
x = self.fc3(x)
x = self.fc4(x)
return x

Here self.fc4 = nn.Linear(512, 6) this is my final layer.I have run this network but the results are good but not much convincing. I want to use support vector regression instead of nn.Linear but I do not find any help. Please help me in this regard. I hope I have clearly mentioned my problem.

2

Hi,

Using an SVM as the loss function for a cnn is not necessarily good as it will give sparse gradients and won’t improve training.
You can also consider all other layers fixed, extract the feature for the last layer and use a regular svm solver to train the last layer as postprocessing of your training.
Finally you can check this paper that tries to extend the use of svms for cnn.

3

Thanks for your response brother, I will check the paper.