Question about Trained models


(Rob) #1

Hi everyone!
Just learning and have a few questions.

Fist what is meant by using a ‘pre trained’ model? If I have my own image set that I train with how is this model pretrained or how does this affect my model training? For example I’m using

model = models.vgg16(pretrained=True)

The doc’s say - Parameters: pretrained (bool) – If True, returns a model pre-trained on ImageNet

Looking at code examples when someone uses the pretrained = true in their code but has a training function on their own image set. What is the point of downloading a trained model when they are training it anyway?


(Nicolò Savioli) #2

Means that all layers are updated with those of a network trained on another data-set, instead of having random weights you have well initialise weights. Then, pretrained=True you activate it.


(Rob) #3

Ok, so it would be used to help speed up training then? When would one want to use a non pretrained model - is there a situation where this would be required?


(Nicolò Savioli) #4

It serves to have a better generalisation of the model and works well with very deep nets. It turns out to be a good practice to pre-initialise the weights when you can.


#5

If you use a pre-trained model, you are more or less bound to the architecture, which is fine in most use cases.
Sometimes you would like to define a specific model architecture and cannot use pre-trained weights, since they are not matching.
It really depends on the use case. Often you can replace a part of your overall model with a pre-trained model, which might help the training as @Nicolo_Savioli explained.

On the other hand, there are not many pre-trained models working with e.g. accelerometer data for activity recognition.