Help for "HTTP Error 403: rate limit exceeded"

1

hi, everyone.
I´m try to use the ResNet in pytorch hub to my project in google colab, but the following error appeared:

/usr/lib/python3.7/urllib/request.py in http_error_default(self, req, fp, code, msg, hdrs)
    647 class HTTPDefaultErrorHandler(BaseHandler):
    648     def http_error_default(self, req, fp, code, msg, hdrs):
--> 649         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    650 
    651 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 403: rate limit exceeded

I am new in this area and I thank you in advance for all the help.

2

It is probably because the site you are trying to reach has a request limit for security reasons. For example, if you use the Reddit API, you are only allowed to do one request every two seconds if I remember correctly. You could maybe investigate the rules at the Google Colab API and see if it mentions this.

You could try to make an exception handling on HTTPError and make your program sleep for a number of seconds before requesting again.

Maybe these links are helpful:

3

Hi Simon,
Thanks for answering. I don’t understand how the reboot works? this message has been appearing for a few days.
How do I reset the time? :frowning:

Thanks a lot.

4

I think in order for people to help you better you have to more clearly explain what exactly you’re trying to do.

What I am hinting at is that if you’re programatically trying to request an URL, you may want to do something like

import time

try:
 # do something
except(HTTPError):
 time.sleep(60)
 # do it again
5

Hi again.
Yes, It’s just that I’m new to this forum. I’m sorry.
I’m learning about CNN and I created a ResNet by calling the Pytorch hub:

net = torch.hub.load('pytorch/vision:v0.9.0','resnet18',pretrained=False)

This call is made by the training code. “Net” is the model I’m using, which in this case will be an untrained ResNet.
That’s when the error I mentioned appears.
Sorry if it’s a rookie question, but I really am. :smiley: :smiley:
Thanks a lot.

6

I can reproduce the issue so thanks for posting it here.
Would you mind updating this issue, too?

7

Hi everyone,
Thanks ptrblck for answer. Yes, i´ll post there too.
The paleative I’m using to solve this problem is copying the ResNet code from github and pasting it into the colab code. And there I run the code.
I followed the recommendation of user D_L in this post.
The same error happens for other networks used by pytorch hub. I am currently using shufflenet which had the same problem.

Thanks a lot.

8

I ran into the same issue today when using a conda environment with pytorch 1.9.0 installed.

When my python script hits the following line:
model = torch.hub.load(“pytorch/vision:v0.6.0”, “inception_v3”, pretrained=True)
i run into this error File “web.py”, line 26, in classify_product
product_h1 = Product_h1().getmodel()
File “/home/frontier/01_Programs/00_code/fabric_tag/models.py”, line 97, in init
model = torch.hub.load(“pytorch/vision:v0.6.0”, “inception_v3”, pretrained=True)
File “/home/frontier/anaconda3/lib/python3.8/site-packages/torch/hub.py”, line 362, in load
repo_or_dir = _get_cache_or_reload(repo_or_dir, force_reload, verbose)
File “/home/frontier/anaconda3/lib/python3.8/site-packages/torch/hub.py”, line 162, in _get_cache_or_reload
_validate_not_a_forked_repo(repo_owner, repo_name, branch)
File “/home/frontier/anaconda3/lib/python3.8/site-packages/torch/hub.py”, line 124, in _validate_not_a_forked_repo
with urlopen(url) as r:
File “/home/frontier/anaconda3/lib/python3.8/urllib/request.py”, line 222, in urlopen
return opener.open(url, data, timeout)
File “/home/frontier/anaconda3/lib/python3.8/urllib/request.py”, line 531, in open
response = meth(req, response)
File “/home/frontier/anaconda3/lib/python3.8/urllib/request.py”, line 640, in http_response
response = self.parent.error(
File “/home/frontier/anaconda3/lib/python3.8/urllib/request.py”, line 569, in error
return self._call_chain(*args)
File “/home/frontier/anaconda3/lib/python3.8/urllib/request.py”, line 502, in _call_chain
result = func(*args)
File “/home/frontier/anaconda3/lib/python3.8/urllib/request.py”, line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: rate limit exceeded

Is there any solution workaround? I saw some people installed another pytorch version…

9

In the linked issue this workaround was posted, so you could try it out and check if it’s working.

10

Thank you! This worked