Getting an error “UnpicklingError: invalid load key, 'v'.” in Pytorch model deploying in Streamlit

1

I am trying to deploy my wheat detection model of Pytorch in Streamlight. But, I am getting an error as

UnpicklingError: invalid load key, 'v'.
Traceback:
File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)
File "/app/streamlit/streamlit.py", line 84, in <module>
    model.load_state_dict(torch.load(weights_file, map_location=device))
File "/home/appuser/.local/lib/python3.7/site-packages/torch/serialization.py", line 529, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/appuser/.local/lib/python3.7/site-packages/torch/serialization.py", line 692, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)

My Github repo is in GitHub - Anubhav1107/streamlit

Please tell if you want something, Thanks for helping !

2

I’m not sure how you are trying to load the state_dict, but did you make sure that it was properly downloaded before trying to open it?

3

hey @ptrblck, thanks for replying.
Indeed the problem is related to improper download of the torch file, When I try to access the .pth file(168 MB) through git lfs, it just gives a log file of 12 kb, that is why the error occurs.

So, I was thinking of what is another way to download a weights file, either drive or something else

4

I’m unsure if this question is related to PyTorch or to e.g. Google Colab / Github.
Are you generally having trouble downloading these files?

5

For future visitors- this might be happening because transformer model sharing makes use of git lfs system. So the pytorch_model.bin, that got downloaded is just a pointer.

As @Anubhav1107 pointed out, you can download the file instead of pointer, and you should not get this error.

3 Likes
6

Thanks, this worked like a charm!

7

Thanks, i’m having the same issue.
and how can i do to download the entire file and not pointer?

i’m working on colab and this is my code for download
!wget https://huggingface.co/BoccheseGiacomo/longformer-mail/raw/main/longformer-mail.pth