Torch model downloaded as string to bytes

1

Hello, I am downloading a model via Dropbox url and saving the data as a system string. In order to load the model into Pytorch I have to convert the data from string to byte data. When I use .encode() with ‘UTF-8’ codec to convert the string data to byte data and try to load it into BytesIO I get UnpicklingError: invalid load key, ‘<’

I wish I could just use Python request and use the method .content to get the raw byte info but I am stuck with string data on the platform I am using.

Any other ideas on how to convert string data to byte?

data_str = self.Download("https://www.dropbox.com/s/examplepath/checkpoint.pth?dl=1")

data_byte = data_str.encode()

self.agent.local.load_state_dict(torch.load(BytesIO(data_byte), map_location=lambda storage, loc: storage))

UnpicklingError: invalid load key, ‘<’

2

There are plenty of examples and utility functions for downloading and opening models in the utils for the torch model hub. The key likely is to download them as an octet stream into bytes. You are probably best off using the api rather than reinventing the implementation.

Best regards

Thomas