urllib.error.HTTPError: HTTP Error 403: rate limit exceeded

vision
1

I am not sure why I get rate limit error.

(fashcomp) [jalal@goku fashion-compatibility]$ python main.py --test --l2_embed --resume runs/nondisjoint_l2norm/model_best.pth.tar --datadir ../../../data/fashion
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn("The use of the transforms.Scale transform is deprecated, " +
Traceback (most recent call last):
  File "main.py", line 313, in <module>
    main()    
  File "main.py", line 105, in main
    model = torch.hub.load('pytorch/vision:v1.9.0', 'resnet18', pretrained=True)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/hub.py", line 362, in load
    repo_or_dir = _get_cache_or_reload(repo_or_dir, force_reload, verbose)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/hub.py", line 162, in _get_cache_or_reload
    _validate_not_a_forked_repo(repo_owner, repo_name, branch)
  File "/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/hub.py", line 124, in _validate_not_a_forked_repo
    with urlopen(url) as r:
  File "/usr/local/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/local/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/local/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: rate limit exceeded

I have:

$ pip freeze
h5py==3.3.0
joblib==1.0.1
numpy==1.21.2
Pillow==8.3.1
scikit-learn==0.24.2
scipy==1.7.1
sklearn==0.0
threadpoolctl==2.2.0
torch==1.9.0
torchaudio==0.9.0
torchvision==0.10.0
typing-extensions==3.10.0.0

and I followed this:

Screen Shot 2021-08-23 at 10.50.55 PM
Screen Shot 2021-08-23 at 10.50.55 PM1626×492 64.6 KB

2

This error message indicates that you have exceeded the rate limit for the API or website that you are accessing with the urllib library. This could happen if you are making too many requests in a short amount of time.

To resolve this issue, you can try the following:

Check the API or website documentation to see what the rate limit is and adjust your request frequency accordingly.

If you are accessing an API, consider using an API key or authentication token to increase your rate limit.

Implement a backoff mechanism in your code that waits for a certain amount of time before making another request. This will prevent your code from making too many requests too quickly.

If the website or API has a paid subscription plan, consider upgrading to increase your rate limit.

If none of the above solutions work, you may need to contact the API or website support team for further assistance.

It’s also worth noting that some websites and APIs may have strict usage policies or terms of service that prohibit scraping or excessive use. Be sure to read and comply with these policies to avoid being blocked or banned from the service.

Regards,
Rachel Gomez