Changeset View
Changeset View
Standalone View
Standalone View
docs/journal-clients.rst
| .. _journal_clients: | .. _journal_clients: | ||||
| Software Heritage Journal clients | Software Heritage Journal clients | ||||
| ================================= | ================================= | ||||
| Journal client are processes that read data from the |swh| Journal, | Journal client are processes that read data from the |swh| Journal, | ||||
| in order to efficiently process all existing objects, and process new objects | in order to efficiently process all existing objects, and process new objects | ||||
| as they come. | as they come. | ||||
| Some journal clients, such as :ref:`swh-dataset <swh-dataset>` only read | Some journal clients, such as :ref:`swh-dataset <swh-dataset>` only read | ||||
| existing objects and stop when they are done. | existing objects and stop when they are done. | ||||
| They can run in parallel, and the :mod:`swh.journal.client` module | Other journal clients, such as the :ref:`mirror <swh-storage>` are expected to | ||||
| provides an abstraction handling all the setup, so actual clients are actually | read constantly from the journal. | ||||
| a single function that takes :mod:`model objects <swh.model.model>` as parameters. | |||||
| They can run in parallel, and the :mod:`swh.journal.client` module provides an | |||||
| abstraction handling all the setup, so actual clients only consists in a single | |||||
| function that takes :mod:`model objects <swh.model.model>` as parameters. | |||||
| For example, a very simple journal client that prints all revisions and releases | For example, a very simple journal client that prints all revisions and releases | ||||
| to the console can be implemented like this: | to the console can be implemented like this: | ||||
| .. literalinclude:: example-journal-client.py | .. literalinclude:: example-journal-client.py | ||||
| Parallelization | Parallelization | ||||
| ▲ Show 20 Lines • Show All 54 Lines • Show Last 20 Lines | |||||