Changeset View
Changeset View
Standalone View
Standalone View
docs/tutorial.rst
Show First 20 Lines • Show All 311 Lines • ▼ Show 20 Lines | |||||
Transport error handling | Transport error handling | ||||
^^^^^^^^^^^^^^^^^^^^^^^^ | ^^^^^^^^^^^^^^^^^^^^^^^^ | ||||
We generally recommend logging every unhandleable error with the response content and | We generally recommend logging every unhandleable error with the response content and | ||||
then immediately stop the listing by doing an equivalent of | then immediately stop the listing by doing an equivalent of | ||||
:py:meth:`Response.raise_for_status` from the ``requests`` library. As for rate-limiting | :py:meth:`Response.raise_for_status` from the ``requests`` library. As for rate-limiting | ||||
errors, we have a strategy of using a flexible decorator to handle the retrying for us. | errors, we have a strategy of using a flexible decorator to handle the retrying for us. | ||||
It is based on the ``tenacity`` library and accessible as :py:func:`throttling_retry` from | It is based on the ``tenacity`` library and accessible as :py:func:`http_retry` from | ||||
:py:mod:`swh.lister.utils`. | :py:mod:`swh.lister.utils`. | ||||
Pagination | Pagination | ||||
^^^^^^^^^^ | ^^^^^^^^^^ | ||||
This one is a moving target. You have to understand how the pagination mechanics of the | This one is a moving target. You have to understand how the pagination mechanics of the | ||||
particular service works. Some guidelines though. The identifier may be minimal (an id | particular service works. Some guidelines though. The identifier may be minimal (an id | ||||
to pass as query parameter), compound (a set of such parameters) or complete (a whole | to pass as query parameter), compound (a set of such parameters) or complete (a whole | ||||
▲ Show 20 Lines • Show All 52 Lines • Show Last 20 Lines |