Transfer Learning with imbalanced dataset

One approach would be to use a WeightedRandomSampler (here is a small example of its usage) or use weights in your loss function (e.g. nn.CrossEntropyLoss accepts a weight argument).