I built a custom model that is similar to MobileNet-v2 in its structure, but has less layers overall. The problem is that my model is useless, as I cannot afford fully training it on ImageNet.
I work on a PC with a GeForce GTX 1060 GPU with 6GB, so I cannot use the settings from the torchvision training reference. All the workarounds I tried, using different kinds of subsets of the ImageNet dataset with different learning-rate strategies and different batch sizes have failed to produce a usable model that can be transferred for other tasks.
Are there any recommendations what I can do to have the model pretrained?
Specifically, I’d be glad to know:
- whether you think Knowledge Distillation should make a difference
- whether there exists any service that lets you just upload a Python source file with a PyTorch network class and that automatically trains it for you on ImageNet, so you can later download its trained weights?