Updating 2e997f3..6e5bd0c
Fast-forward
 mypy.ini                                       |   3 +
 swh/provenance/__init__.py                     |  14 +-
 swh/provenance/api/client.py                   | 528 ++++++++++++++++-
 swh/provenance/api/server.py                   | 784 ++++++++++++++++++++++---
 swh/provenance/cli.py                          |  29 +-
 swh/provenance/interface.py                    |   4 +-
 swh/provenance/mongo/backend.py                |   4 +-
 swh/provenance/postgresql/archive.py           |  13 +
 swh/provenance/postgresql/provenance.py        |  10 +-
 swh/provenance/storage/archive.py              |  13 +
 swh/provenance/tests/conftest.py               |  31 +-
 swh/provenance/tests/test_archive_interface.py |  16 +-
 12 files changed, 1313 insertions(+), 136 deletions(-)

Changes applied before test

commit 6e5bd0c00d9778edc412779f75fa2f591b9dd838
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Sep 14 15:33:33 2021 +0200

    Add StatsD support on `ArchiveInterface` implementations

commit 3bde39bfe1560cff91f4cd07a8cb37bf8d899091
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Sep 14 15:21:55 2021 +0200

    Improve timeout while waiting for response handling on client side

commit 3aaa272b2e540de50d864979cade5538d0d30ca1
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Sep 14 14:50:42 2021 +0200

    Switch to use a topic exchange instead of a direct one on remote backend
    
    This allows to have less working processes (16 per entity and 16 extra for locations),
    each one listening in several queues, depending on the requested insertion method.
    All pending conflict resolutions is solved now.

commit 7f1b31b6975a350ad336f89564b32e4adaa79e09
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Sep 3 16:06:18 2021 +0200

    Improve connection error handling on both client and server side
    
    Change the RabbitMQ scheme to have 10 exchange (one per set method) and
    16 routing keys per exchange, instead of the original 160 exchanges with
    1 routing key each. Still 160 worker processes though.

commit f306903a932648f4837cd6365919fc22078c4dab
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Sep 1 11:27:02 2021 +0200

    Remove old client/server storage based on `swh.core.api.RPCClient`

commit 1b856a23946860665341804a0eba97b461cbc97d
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 31 13:36:34 2021 +0200

    Rework `ProvenanceStorageRabbitMQWorker` to handle connection loss
    
    The workers now use `pika.SelectConnection` and make and explicit handle
    of its life-cycle

commit a26fa8c35224f8157ad54e6ceabd5014086e60ce
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Sep 7 15:45:34 2021 +0200

    Refactor server to use multiple sub-processes instead of threads

commit 17e67552ddf718e7c3652bce96fa76f17fbb5ce7
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Aug 25 13:52:31 2021 +0200

    Add support for `relation_add` to the RabbitMQ server

commit 5623696fd976cc34d5ae57da3da64f42e3ab3ce4
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 24 15:09:02 2021 +0200

    Remove get queues and have client read from `ProvenanceStorage` directly
    
    Add a new queue to the server for clients to retrieve storage configuration

commit 95e7f73c9d52755cae98abbf07f93abbdf19909d
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 24 11:45:47 2021 +0200

    Split set method's requests into several queues on server side

commit 081303596f3e4daa5759f8dfafa28022b0d76a79
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Aug 20 12:21:27 2021 +0200

    Add new RabbitMQ-based client/server API

commit 3383cae57ef741859c33300045bc8a149dfc0d35
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Sep 14 09:58:20 2021 +0200

    Generalize types for `content_add` and `directory_add` in the storage interface

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/340/ for more details.

Harbormaster completed remote builds in B23589: Diff 22653.Sep 14 2021, 3:39 PM

aeviso requested review of this revision.Sep 14 2021, 3:39 PM

aeviso added a parent revision: D6165: Add new RabbitMQ-based client/server API.Sep 14 2021, 3:40 PM

rebase

aeviso added a child revision: D6266: Add StatsD support to the main algorithms of the module.Sep 15 2021, 1:43 PM

aeviso retitled this revision from Add StatsD support on `ArchiveInterface` implementations to Add StatsD support to `ArchiveInterface` implementations.Sep 15 2021, 1:44 PM

aeviso removed a parent revision: D6165: Add new RabbitMQ-based client/server API.

Build is green

Patch application report for D6256 (id=22681)

Rebasing onto 3383cae57e...

Current branch diff-target is up to date.

Changes applied before test

commit a8009be206dc1e414a69a708f826266b750b74cd
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Sep 14 15:33:33 2021 +0200

    Add StatsD support to `ArchiveInterface` implementations

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/343/ for more details.

Harbormaster completed remote builds in B23614: Diff 22681.Sep 15 2021, 1:49 PM

aeviso added a reviewer: jayeshv.Sep 15 2021, 2:01 PM

jayeshv accepted this revision.Sep 16 2021, 3:21 PM

This revision is now accepted and ready to land.Sep 16 2021, 3:21 PM

Closed by commit rDPROVa8009be206dc: Add StatsD support to `ArchiveInterface` implementations (authored by aeviso). · Explain WhySep 16 2021, 3:40 PM

This revision was automatically updated to reflect the committed changes.

aeviso added a commit: rDPROVa8009be206dc: Add StatsD support to `ArchiveInterface` implementations.