Inject dropout into resnet (or any other network)

So i want to inject dropout into a (pretrained) resnet, as i get pretty bad over-fitting. (for example add a dropout layer after each residual step)

I guess that i could simply monkey patch the resnet base class. However i wonder if there is easy way to achieve this?

I think the cleanest approach would be to create your custom resnet class by deriving from the torchvision implementation and overriding the forward method.

With Res net in particular the derivation is even easier as the actual layers are already implemented in a separate class that gets passed to a basic resnet.

I was hopping that there is a general approach, that i could apply to multiple models. The standard models typically don’t contain drop out, as they are usually trained with big datasets. But for small Training datasets which are pretty common in practice, dropout helps a lot. So having a function that would adds dropout before/after each relu would be very useful.

model_with_dropout = add_dropout(model, after=“relu”)

Alternatively to my proposed approach you could also use forward hooks and add dropout at some layers. I’m not sure, if it’s the best approach, but it might be similar to your mentioned approach.