Problem with my checkpoint file when using torch.load()

Hi, I have a problem loading my checkpoint file(.pth). It’s all right when I load my other checkpoint files but not with this. Here’s how I save the model:

    def save_networks(self, epoch):
        """Save all the networks to the disk.

        Parameters:
            epoch (int) -- current epoch; used in the file name '%s_net_%s.pth' % (epoch, name)
        """
        for name in self.model_names:
            if isinstance(name, str):
                save_filename = '%s_net_%s.pth' % (epoch, name)
                save_path = os.path.join(self.save_dir, save_filename)
                net = getattr(self, 'net' + name)

                if len(self.gpu_ids) > 0 and torch.cuda.is_available():
                    if name == 'Rgr':
                        torch.save(net.state_dict(), save_path)
                    else:
                        torch.save(net.module.cpu().state_dict(), save_path)
                        net.cuda(self.gpu_ids[0])
                else:
                    if name == 'Rgr':
                        torch.save(net.state_dict(), save_path)
                    else:
                        torch.save(net.cpu().state_dict(), save_path)

And when I load the file:

torch.load('./latest_net_G.pth', map_location='cpu')

I got the following error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/opt/conda/lib/python3.7/tarfile.py in nti(s)
    186             s = nts(s, "ascii", "strict")
--> 187             n = int(s.strip() or "0", 8)
    188         except ValueError:

ValueError: invalid literal for int() with base 8: '_v2\nq\x03(('

During handling of the above exception, another exception occurred:

InvalidHeaderError                        Traceback (most recent call last)
/opt/conda/lib/python3.7/tarfile.py in next(self)
   2288             try:
-> 2289                 tarinfo = self.tarinfo.fromtarfile(self)
   2290             except EOFHeaderError as e:

/opt/conda/lib/python3.7/tarfile.py in fromtarfile(cls, tarfile)
   1094         buf = tarfile.fileobj.read(BLOCKSIZE)
-> 1095         obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)
   1096         obj.offset = tarfile.fileobj.tell() - BLOCKSIZE

/opt/conda/lib/python3.7/tarfile.py in frombuf(cls, buf, encoding, errors)
   1036 
-> 1037         chksum = nti(buf[148:156])
   1038         if chksum not in calc_chksums(buf):

/opt/conda/lib/python3.7/tarfile.py in nti(s)
    188         except ValueError:
--> 189             raise InvalidHeaderError("invalid header")
    190     return n

InvalidHeaderError: invalid header

During handling of the above exception, another exception occurred:

ReadError                                 Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/torch/serialization.py in _load(f, map_location, pickle_module, **pickle_load_args)
    555         try:
--> 556             return legacy_load(f)
    557         except tarfile.TarError:

/opt/conda/lib/python3.7/site-packages/torch/serialization.py in legacy_load(f)
    466 
--> 467         with closing(tarfile.open(fileobj=f, mode='r:', format=tarfile.PAX_FORMAT)) as tar, \
    468                 mkdtemp() as tmpdir:

/opt/conda/lib/python3.7/tarfile.py in open(cls, name, mode, fileobj, bufsize, **kwargs)
   1590                 raise CompressionError("unknown compression type %r" % comptype)
-> 1591             return func(name, filemode, fileobj, **kwargs)
   1592 

/opt/conda/lib/python3.7/tarfile.py in taropen(cls, name, mode, fileobj, **kwargs)
   1620             raise ValueError("mode must be 'r', 'a', 'w' or 'x'")
-> 1621         return cls(name, mode, fileobj, **kwargs)
   1622 

/opt/conda/lib/python3.7/tarfile.py in __init__(self, name, mode, fileobj, format, tarinfo, dereference, ignore_zeros, encoding, errors, pax_headers, debug, errorlevel, copybufsize)
   1483                 self.firstmember = None
-> 1484                 self.firstmember = self.next()
   1485 

/opt/conda/lib/python3.7/tarfile.py in next(self)
   2300                 elif self.offset == 0:
-> 2301                     raise ReadError(str(e))
   2302             except EmptyHeaderError:

ReadError: invalid header

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
<ipython-input-10-c2a19cc36006> in <module>
----> 1 torch.load('multi_task/checkpoints/latest_pet/latest_net_G.pth', map_location=torch.device('cpu'))

/opt/conda/lib/python3.7/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
    385         f = f.open('rb')
    386     try:
--> 387         return _load(f, map_location, pickle_module, **pickle_load_args)
    388     finally:
    389         if new_fd:

/opt/conda/lib/python3.7/site-packages/torch/serialization.py in _load(f, map_location, pickle_module, **pickle_load_args)
    558             if zipfile.is_zipfile(f):
    559                 # .zip is used for torch.jit.save and will throw an un-pickling error here
--> 560                 raise RuntimeError("{} is a zip archive (did you mean to use torch.jit.load()?)".format(f.name))
    561             # if not a tarfile, reset file offset and proceed
    562             f.seek(0)

RuntimeError: multi_task/checkpoints/latest_pet/latest_net_G.pth is a zip archive (did you mean to use torch.jit.load()?)

I don’t know what mistakes I have made when saving the model. Could you help me with this?

Are you using the same PyTorch version to save and load the state_dict or are you using an older one to load it?

5 Likes

Thanks @ptrblck. I was facing a similar kind of a problem. I realized that the pytorch versions while saving and loading were different.

3 Likes

@Chandan_Agrawal @ptrblck thanks, that’s the problem. The version of pytorch for saving and loading should be the same~

1 Like

When you have to inference with a pytorch version below 1.6, try code below to convert your model, because pytorch changed the model saving format after version 1.6.

torch.save(model.state_dict(), path, _use_new_zipfile_serialization=False)
4 Likes

Hi @Eric_Hou, nice what about loading the model from a newer version of pytorch (1.8.1) to and older version of pytorch (1.3.0) what would you need to add? I receive the error:

RuntimeError: test.pth is a zip archive (did you mean to use torch.jit.load()?)

Then when adding torch.jit.load, I receive the error message:

RuntimeError: version_number <= kMaxSupportedFileFormatVersion INTERNAL ASSERT FAILED at ../caffe2/serialize/inline_container.cc.......

Regards,
Abu

HI Ptrblck,

I face this problem when I want to load the “csv” file. What can I do to solve the problem. I save my dataset and want to reuse them but gave me the same error. Should I save the result from begining?

Since you are working with a csv file, I assume it wasn’t stored using PyTorch, so you could load it via e.g. pandas or any other library which handles these files.

I saved the variable with: torch.save(Path+’.csv’)

Would you please guide me for saving the variable “.csv” or “.pt” does have different? I most of the time save the variable and results with torch.sava(.csv)

PyTorch doesn’t support storing the data in human-readable csv format, so the file ending won’t matter.
Both files, the *.pt and *.csv will be stored in PyTorch’s binary format.
If you want to store tensor data as a csv file, you would have to use another library, e.g. np.savetxt or pandas.DataFrame.to_csv.

Really nice post, it’s already helping me.