Deserialization error

deployment
1

I saved the object of my trainer class using the torch save method. However, when loading the saved file, the depickle process can’t find “__iterator”, which is a private method inside my “Trainer” class. Because of that, the whole loading process fails. Any idea how to solve or bypass it? I just need to recover the data saved into the attributes of that class.

Traceback (most recent call last):
  File ".conda/envs/py37/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3331, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-a5666d77c70f>", line 1, in <module>
    torch.load("snapshots/model.pth", map_location='cpu')
  File ".conda/envs/py37/lib/python3.7/site-packages/torch/serialization.py", line 529, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File ".conda/envs/py37/lib/python3.7/site-packages/torch/serialization.py", line 702, in _legacy_load
    result = unpickler.load()
AttributeError: 'Trainer' object has no attribute '__iterator'
2

Did you store the Trainer instance directly? Also, could you post its definition?

As explained in the serialization docs it’s recommended to store the state_dicts instead of the objects, as changes in the source files might yield to such errors and you would also have to make sure to recreate the same folder structure.

3

Unfortunately, I saved the entire object, and that is what caused the whole problem. Here there is a piece of code that will generate the problem I’m facing right now.

import torch

class Test:
	def __init__(self):
		self.a = min
		self.b = max
		self.c = self.__private  # buggy

	def __private(self):
		return None

test = Test()

torch.save({"test": test}, "file.pkl")
torch.load("file.pkl")

However, if you remove the private attribute from the method, you won’t get any error.

import torch

class Test:
	def __init__(self):
		self.a = min
		self.b = max
		self.c = self.private  # not buggy

	def private(self):
		return None

test = Test()

torch.save({"test": test}, "file.pkl")
torch.load("file.pkl")