How to use SVM for classifying Image

1

I am making an image classifier and I have already used CNN and Transfer Learning to classify the images. Support Vector Machine gives a very good boundary with a solid margin, so now I would like to try the SVM into my project. Now I am using PyTorch for all my models. How can I make this model now?
I got this code for making an SVM Classifier -

import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable

class SVM(nn.Module):
    def __init__(self):
        super().__init__()  # Call the init function of nn.Module
        
        self.fc1 = nn.Linear(25088, 12544)
        self.fc2 = nn.Linear(12544, 6272)
        self.fc3 = nn.Linear(6272, 3136)
        self.fc4 = nn.Linear(3136, 1568)
        self.fc5 = nn.Linear(1568, 4)
                
    def forward(self, x):
        
        #Flatten the Image
        x = x.view(x.size(0), -1)

        # add dropout layer
        x = self.dropout(x)
        
        # add 1st hidden layer, with relu activation function
        x = F.leaky_relu(self.fc1(x))
        # add dropout layer
        x = self.dropout(x)
        
        # add 2nd hidden layer, with relu activation function
        x = F.leaky_relu(self.fc2(x))
        # add dropout layer
        x = self.dropout(x)
        
        # add 3rd hidden layer, with relu activation function
        x = F.leaky_relu(self.fc3(x))
        # add dropout layer
        x = self.dropout(x)
        
        # add 4th hidden layer, with relu activation function
        x = F.leaky_relu(self.fc4(x))
        # add dropout layer
        x = self.dropout(x)
        
        # add 5th hidden layer, with relu activation function
        x = self.fc5(x)
       
        
        # add 2nd hidden layer, with relu activation function
        #x = self.fc2(x)
        return x

Is it the code for SVM? I am not sure about this, becasue this is only just some linear layers. Can you guys please throw some light in it? Thanks!

2

I’m not sure to understand the question completely, but you’ve created a neural network in your code snippet.
If you would like to use a SVM, I would just use sklearn.svm.

3

Okay, so here I am making a classifier of 4 classes and now I want to use SVM, for that I got this reference - SVM using PyTorch in Github. I have seen this scikit learn SVM, but I am not able to find out how to use this and print the loss and accuracy per epoch. I want to do it in PyTorch. This is the code after printing the model of SVM -

learning_rate = 0.1  # Learning rate
epoch = 10  # Number of epochs
batch_size = 32  # Batch size

#X = torch.FloatTensor(X)  # Convert X and Y to FloatTensors
#Y = torch.FloatTensor(Y)
#N = len(Y)  # Number of samples, 500

#model = SVM()  # Our model
optimizer = optim.SGD(model.parameters(), lr=learning_rate)  # Our optimizer

target = torch.randint(0, n_classes, (batch_size,)) # labels

for epoch in range(1, epoch+1):
  
  train_loss = 0
  model.train()
  for images, labels in dataloader_train:
    
    #steps += 1
    images, labels = images.to(device), labels.to(device)
    
    optimizer.zero_grad()
    
    output = model.forward(images)
    conf_matrix = confusion_matrix(output, labels, conf_matrix)
    p = torch.nn.functional.softmax(output, dim=1)
    prediction = torch.argmax(p, dim=1)
    #loss = torch.nn.functional.nll_loss(torch.log(p), y)
    loss = criterion(output, labels)
    loss.backward()
    optimizer.step()
    
    train_loss += loss.item()*images.size(0)
    
    #vis.line(X=torch.ones((1,1)).cpu()*epoch, Y=torch.Tensor([train_loss]).unsqueeze(0).cpu(),win=loss_window,update='append')    
    
  #if steps % print_every == 0:
  valid_loss = 0
  accuracy = 0
  model.eval()
  for images, labels in dataloader_test:
    optimizer.zero_grad()
    with torch.no_grad():
       
      images, labels = images.to(device), labels.to(device)
      #if train_on_gpu:
      #      images, labels = data.cuda(), target.cuda()

      output = model.forward(images)
      conf_matrix = confusion_matrix(output, labels, conf_matrix)
      p = torch.nn.functional.softmax(output, dim=1)
      prediction = torch.argmax(p, dim=1)
      loss = criterion(output, labels)
          
      valid_loss += loss.item()*images.size(0)
      
      ps = torch.exp(output)
         
      top_p, top_class = ps.topk(1, dim = 1)
      equals = top_class == labels.view(*top_class.shape)
      accuracy += torch.mean(equals.type(torch.FloatTensor))
      
  # calculate average losses
  train_loss = train_loss/len(dataloader_train.dataset)
  valid_loss = valid_loss/len(dataloader_test.dataset)

How to proceed with SVM, there is almost no clue in the net, or I am not able to find out. Thanks.

4

I would probably start with Support Vector Networks by Cortes and Vapnik.
It’s a good introduction to SVMs and explains how and why e.g. the different kernel functions work for SVMs.

2 Likes