Model parallelism for pretrained model

I’m trying to follow for model parallelism, is there a way to make this process easy for pretrained models? For example, in this post, I can create a paralleled ModelParallelResNet50 from scratch. However, if I have a pretrained ResNet50, I cannot directly load model state into ModelParallelResNet50, since the latter has a different structure.

Hi @wasabi,

Perhaps using the strict=False argument might be a workaround for this error?