Hi everyone, for a project I find myself having to recreate the experimets of this paper( https://arxiv.org/abs/1306.0239), where they integrate the svm in the structure of the network in order to fine tune even the first layers with the svm objective.
My problem is that I have little to no idea on how implement this feature. In particular I am using the implementation of resnet of this repository( https://github.com/akamaster/pytorch_resnet_cifar10) in the other parts of the project, so I need to use it also here for consistency
another thing that I would like to ask you is how can I use the pretrained network as a feature extractor for the svm in order to compare the results with the other mothod.
thank you in advance
I’m not familiar with the paper, but have a look at MultiLabelMarginLoss, which implements the multi-class hinge loss and might thus be seen as the “SVM loss”.
thank you for answering, I am a little bit confused about the whole thing…
do you think that the result of implementing hinge loss is going to be the equivalent of replacing the fc layer with a support vector classifier?
And another thing, how can I use the net that I alread trained on the dataset as a feature extractor and then use those features to train an svm classifier?
I’m not sure for the first question, but have seen use cases which use the proposed loss instead of an SVM.
If you want to use a “real” SVM, you could store all extracted features as numpy arrays (with their target), and train e.g. an SVM from sklearn on it. This would of course not retrain the feature extractor anymore, but might be a valid use case, if you just want to train the SVM.
I would lilke to do both things.
First I would like to use an svm classifier instead of the fc layer, in order to train the network with it, and I was wondering if it was enough to use a different loss in order to do it, or if I had to do something else.
I also would like to try and use the sklearn implementation of the svm using the in_features of the fc layer as its inputs. In this case my question regards how to retrieve these features from the trained net
Hello,
Did you find out how to integrate SVM in the neural network structure? I’m currently trying to implement similar model.