Page MenuHomeSoftware Heritage

cgit: handle "429 Too Many Requests" errors
Closed, MigratedEdits Locked

Description

For example:

HTTPError: 429 Client Error: Too Many Requests for url: https://git.alpinelinux.org/?ofs=10
  File "swh/lister/cgit/tasks.py", line 20, in list_cgit
    return lister.run().dict()
  File "swh/lister/pattern.py", line 127, in run
    for page in self.get_pages():
  File "swh/lister/cgit/lister.py", line 90, in get_pages
    bs_idx = self._get_and_parse(next_page)
  File "swh/lister/cgit/lister.py", line 80, in _get_and_parse
    response.raise_for_status()
  File "requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

https://sentry.softwareheritage.org/share/issue/9316020c589242e587f9a71c607cb31a/