Convert pytorch 1.0 model to pytorch 1.1

1

When trying to load a model I trained back in pytorch 1.0 in pytorch 1.1, I get the following warning :

/home/user/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.modules.conv.Conv2d' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.
  warnings.warn(msg, SourceChangeWarning)
/home/user/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.modules.batchnorm.BatchNorm2d' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.
  warnings.warn(msg, SourceChangeWarning)
/home/user/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.modules.activation.ReLU' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.
  warnings.warn(msg, SourceChangeWarning)
/home/user/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.modules.container.ModuleList' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.
  warnings.warn(msg, SourceChangeWarning)
/home/user/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torchvision.models.resnet.BasicBlock' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.
  warnings.warn(msg, SourceChangeWarning)
/home/user/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.modules.pixelshuffle.PixelShuffle' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.
  warnings.warn(msg, SourceChangeWarning)
/home/user/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.modules.padding.ReplicationPad2d' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.
  warnings.warn(msg, SourceChangeWarning)

That warning translate into an error when trying to use the model :

~/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

~/Documents/fastai/fastai/layers.py in forward(self, x)
    153         for l in self.layers:
    154             res.orig = x
--> 155             nres = l(res)
    156             # We have to remove res.orig to avoid hanging refs and therefore memory leaks
    157             res.orig = None

~/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

~/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/nn/modules/container.py in forward(self, input)
     90     def forward(self, input):
     91         for module in self._modules.values():
---> 92             input = module(input)
     93         return input
     94 

~/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

~/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/nn/modules/conv.py in forward(self, input)
    329     @weak_script_method
    330     def forward(self, input):
--> 331         if self.padding_mode == 'circular':
    332             expanded_padding = ((self.padding[1] + 1) // 2, self.padding[1] // 2,
    333                                 (self.padding[0] + 1) // 2, self.padding[0] // 2)

~/miniconda3/envs/crystal_clear/lib/python3.6/site-packages/torch/nn/modules/module.py in __getattr__(self, name)
    537                 return modules[name]
    538         raise AttributeError("'{}' object has no attribute '{}'".format(
--> 539             type(self).__name__, name))
    540 
    541     def __setattr__(self, name, value):

AttributeError: 'Conv2d' object has no attribute 'padding_mode'

From what I understand, I can revert the source with the patch tool to be able to use my old model. Is there a way to convert the model to pytorch 1.1?

2

While doing something else, I noticed that I managed to load an old pytorch model by loading the weights and updating the state dicts by doing model.load_state_dict(new_weights). I’m guessing those warning are coming from fastai that must pickle the model object. I’m guessing that I should be able to update my old model by creating a new model on pytorch 1.1 and loading the weights from the old model. Then I’ll create a new Learner object in fastai and I’ll save that new Learner. I’ll let you know how it goes.

3

Hey @StatisticDean,
any updates on your success?

4

Hi @faib, the steps I mentionned hold. The “issue” when loading is that your model is not the same kind of object, but it has the same architecture and can load the same weights, so you just have to create a new model object in pytorch 1.1 and load the weights from your old model in pytorch 1.0 and you’re good to go :slight_smile:

5

So when you are in you notebook with pytorch 1.0.1, using fastai, after defining the model your are doing: torch.save(model.state_dict(),'pretrained_svhn_ufpr_stage9.pth')
then you switch to a virtualenv or you update to pytorch 1.1, define your model and use:
model.load_state_dict(torch.load('pretrained_svhn_ufpr_stage9.pth'))
to load the model weights?

I tried it and it doesn’t work for me…
Best
Fabian

6

What kind of error did you get? Could you please provide more information on the model you’re trying to save and load, and how you created the model in both notebooks?

7

I converted the model to the newer version as below.

loading the model saved in pytorch 1.0.*
model = torch.load(model_out_path+ 'Saved_model.pth')

Now save the model again using model.state_dict(), it will save model in version 1.1
torch.save(model.state_dict(), "model_new.pth")

make a new network class’ object and load the recently saved model:

model = myNet()
model.load_state_dict(torch.load("model_new.pth"))
model.eval()

Now no errors :slightly_smiling_face:

1 Like