Best practice on transfer learning

vision
1

HI all,
I would like some advice on best practice on transfer learning.
Supposing i have a dataset A, and i have done some training with that dataset to produce optimum weights W1, If i have an additional dataset B, and i decide to start training from weights W1, in an effort to produce optimum weights W2 that best maps my data. is it best to train with just the additional dataset B or should i resume training from W1 with dataset A+B.

2

How similar are the two datasets? Do they have the same labels?

3

@ptrmcl thank you for taking the time out to reply. and yes assuming dataset A and dataset B have the same labels and are relatively similar. I would like to know when applying transfer learning with a learned set of weights that was trained on dataset A. In order to get good results on dataset B, do i need to feed my model just dataset B alone or dataset A +B

4

So are the data in both datasets something you may see when using your model? In that case, I think I’d use both if you have the computational resources.

Would be good to get others’ opinions too.

1 Like
5

Hey, @temi92!
I have applied transfer learning once, and did it that way:

  • I used a pretrained VGG on Imagenet (your dataset A with weights W1)
  • Then, I used this VGG to train on Imagenette (a similar dataset, your dataset B)
  • So in my case, it was best to just continue training with dataset B, however, note that the VGG already converged on the dataset A.

Let me know if you have any questions.
BR, David

1 Like
6

So yeah frankly in the case of Imagenet it makes sense since it is a large dataset. If both datasets are small I think it may be better to train on A+B?

7

Yes, might be. But depending on the architecture used, and the dataset size, I would also worry about potential overfitting. Why not try both ways, using a validation set during training and testing what works best.