Trying to load a model in pytorch 1.6 results in _pickle.UnpicklingError: invalid load key, '\xad'

#1

Hello everyone,
I’m trying to load a model that was trained and saved in Pytorch1.5.1 in Pytorch1.6. the model loads just fine in 1.5.1 but it fails with the following error when I try to load it in Pytorch 1.6:

  File "d:\codes\fac_ver\python\FV\F_V.py", line 563, in _init_model
    checkpoint = torch.load(self.model_checkpoint_path, map_location=torch.device('cpu'))
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\serialization.py", line 585, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\serialization.py", line 755, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\xad'.

Whats wrong here? how should I be loading this in Pytorch 1.6?
Thanks a lot in advance

#2

The error seems to be raised by pickle so you might need to change the encoding via:

import pickle
pickle.load = partial(pickle.load, encoding="latin1")
pickle.Unpickler = partial(pickle.Unpickler, encoding="latin1")
model = torch.load(model_file, map_location=lambda storage, loc: storage, pickle_module=pickle)

This would be necessary, if you’ve stored e.g. numpy arrays in Python2 and try to load it in Python3.
However, PyTorch 1.5.1 didn’t support Python2.7 anymore, so I’m not sure if that’s the root cause of this issue.

1 Like
#3

Thanks a lot really appreciate your kind help. The model was trained in 1.4 and not 1.5.1 and I have always used Python3.
I’ll give that a try and see how it goes