diff --git a/docs/run_a_new_lister.rst b/docs/run_a_new_lister.rst --- a/docs/run_a_new_lister.rst +++ b/docs/run_a_new_lister.rst @@ -9,8 +9,8 @@ Here are the steps you need to follow to run a lister within your local environment. -1. You must edit the docker-compose override file (`docker-compose.override.yml`). - following the sample provided :: +1. You must edit the docker-compose override file (:file:`docker-compose.override.yml`). + following the sample provided:: version: '2' @@ -19,7 +19,7 @@ volumes: - "$SWH_ENVIRONMENT_HOME/swh-lister:/src/swh-lister" - The file named `docker-compose.override.yml` will automatically be loaded by + The file named :file:`docker-compose.override.yml` will automatically be loaded by ``docker-compose``.Having an override makes it possible to run a docker container with some swh packages installed from sources instead of using the latest published packages from pypi. For more details, you may refer to README.md @@ -45,7 +45,7 @@ ~/swh-environment/swh-docker-dev$ docker-compose up -d 5. Add the lister task-type in the scheduler. For example, if you want to - add pypi lister task-type :: + add pypi lister task-type:: ~/swh-environment$ swh scheduler task-type add list-gnu-full \ "swh.lister.gnu.tasks.GNUListerTask" "Full GNU lister" \ diff --git a/docs/tutorial.rst b/docs/tutorial.rst --- a/docs/tutorial.rst +++ b/docs/tutorial.rst @@ -172,7 +172,7 @@ :ref:`handling-specific-topics`. In any case, we want the data we return to be usefully filtered and structured. The -easiest way to create an iterator is to use the `yield` keyword. Yield each data page +easiest way to create an iterator is to use the ``yield`` keyword. Yield each data page you have structured in accordance with the page type you have declared. The page type exists only for static type checking of data passed from :py:meth:`get_pages` to :py:meth:`get_origins_from_page`; you can choose whatever fits the bill. @@ -315,9 +315,9 @@ We generally recommend logging every unhandleable error with the response content and then immediately stop the listing by doing an equivalent of -:py:meth:`Response.raise_for_status` from the `requests` library. As for rate-limiting +:py:meth:`Response.raise_for_status` from the ``requests`` library. As for rate-limiting errors, we have a strategy of using a flexible decorator to handle the retrying for us. -It is based on the `tenacity` library and accessible as :py:func:`throttling_retry` from +It is based on the ``tenacity`` library and accessible as :py:func:`throttling_retry` from :py:mod:`swh.lister.utils`. Pagination @@ -345,7 +345,7 @@ There are also a few files that need to be modified outside of the lister directory, namely: -* `/setup.py` to add your lister to the end of the list in the *setup* section: +* :file:`/setup.py` to add your lister to the end of the list in the *setup* section: entry_points=""" [swh.cli.subcommands] @@ -355,16 +355,16 @@ lister.cgit=swh.lister.cgit:register ...""" -* `/swh/lister/tests/test_cli.py` to get a default set of parameters in scheduler-related tests. -* `/README.md` to reference the new lister. -* `/CONTRIBUTORS` to add your name. +* :file:`/swh/lister/tests/test_cli.py` to get a default set of parameters in scheduler-related tests. +* :file:`/README.md` to reference the new lister. +* :file:`/CONTRIBUTORS` to add your name. Testing your lister ------------------- When developing a new lister, it's important to test. For this, add the tests -(check `swh/lister/*/tests/`) and register the celery tasks in the main -conftest.py (`swh/lister/core/tests/conftest.py`). +(check :file:`swh/lister/*/tests/`) and register the celery tasks in the main +conftest.py (:file:`swh/lister/core/tests/conftest.py`). Another important step is to actually run it within the docker-dev (:ref:`run-lister-tutorial`).