Details
Details
- Reviewers
douardda - Group Reviewers
Reviewers - Commits
- rDPROVe2a1843d5ebe: Add StatsD support to provenance backend
Diff Detail
Diff Detail
- Repository
- rDPROV Provenance database
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
Build is green
Patch application report for D6351 (id=23086)
Could not rebase; Attempt merge onto 4c087ea0ec...
Updating 4c087ea..f7e98b0 Fast-forward .gitignore | 4 +- mypy.ini | 3 + pytest.ini | 5 + requirements-test.txt | 2 +- swh/provenance/__init__.py | 14 +- swh/provenance/api/client.py | 528 +++++++++++++++++++- swh/provenance/api/server.py | 844 +++++++++++++++++++++++++++++--- swh/provenance/cli.py | 33 +- swh/provenance/interface.py | 9 + swh/provenance/mongo/backend.py | 3 + swh/provenance/postgresql/provenance.py | 3 + swh/provenance/provenance.py | 154 +++--- swh/provenance/tests/conftest.py | 69 +-- tox.ini | 3 +- 14 files changed, 1477 insertions(+), 197 deletions(-)
Changes applied before test
commit f7e98b073f43855f60b4149f57843f050c1dfda2
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 13:27:47 2021 +0200
Split `Provenance::flush` into two methods and add StatsD support
commit 4995134725cb5e62bcdf1d0c19ea4e2fabac5465
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 1 11:27:02 2021 +0200
Remove old client/server storage based on `swh.core.api.RPCClient`
This implementation was a first attempt for conflict resolution that didn't worked as expected.
The new `ProvenanceStorageRabbitMQ` remote storage should be used instead.
commit 50a1441f2068a2abe1293be893e66806cf0977b9
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Sep 21 16:13:53 2021 +0200
Add support for remote backend on existing storage tests
commit ba22e90f707688aed716d20253256ee12c414a33
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 13:39:59 2021 +0200
Improve server/client shoutdown logic and error handling
commit 4ec6d3359e20920469cde43516bc120d4352f915
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Aug 31 13:36:34 2021 +0200
Rework `ProvenanceStorageRabbitMQWorker` to handle connection loss
Use `pika.SelectConnection` and make an explicit handle of its life-cycle.
Improve connection error handling on both client and server side.
Change the RabbitMQ scheme to use 5 exchanges (one per entity + location).
Each exchange handles all entity related insertions, dispatching to different
queues depending on the requested `ProvenanceStorageInterface` methods (16
queues per methods). For instance, the `content` exchange handles all requests
for `content_add` and `relation_add` for both relations `CNT_EARLY_IN_REV` and
`CNT_IN_DIR` (ie. relations with content as source). In each case, requests
are forwarded to 1 of 16 possible workers, depending on the sha1 id of the
content.
commit f6f174fb76b2c42435b6c75eb90c175d9cb0fca5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Aug 20 12:21:27 2021 +0200
Add new RabbitMQ-based client/server API
Get methods in the `ProvenanceStorageInterface` are called through a server that
guarantees conflict-free writings to the underlying database.
Set methods are called directly from the client to avoid RCP overhead for reads.
The server spawns multiple processes to handle independent requests concurrently.
commit 437c2b4ad60e15c0b102988a99766b2c503c91d5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Sep 24 11:08:08 2021 +0200
Add `close` method to both `ProvenanceInterface` and `ProvenanceStorageInterface`
The idea is to have a mechanism to explicitly release resources when needed.
commit 6c3071493b5d3f187113493275d402a27866da95
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 16:14:10 2021 +0200
Rename remote storage backend classes
Make names consistent with the naming convention used for other components.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/377/ for more details.
Comment Actions
Overall looks ok to me but:
- I would have preferred to see the statsd and the refactoring parts of this diff in dedicated git revisions,
- if there is indeed a new behavior/fix in there, it should not be part of the "split the method" refactoring.
| swh/provenance/provenance.py | ||
|---|---|---|
| 137–141 | It's a bit hard to tell since the fact the definition of flush_origin_revision_layer comes before flush_revision_content_layer (I'm guessing to lexicographic order of method names) while it was the other way around originally, so it's not easy to compare both parts of the diff but it looks to me this third part of the rev_orgs dict was not present before this diff. If I'm not mistaken, this part of the really should not be "lost" in the middle of a "pure refactoring" diff (split a method). | |
| swh/provenance/provenance.py | ||
|---|---|---|
| 137–141 | Added a new diff for that. Please check it | |
Comment Actions
Build is green
Patch application report for D6351 (id=23094)
Could not rebase; Attempt merge onto 4c087ea0ec...
Updating 4c087ea..65d40bd Fast-forward .gitignore | 4 +- mypy.ini | 3 + pytest.ini | 5 + requirements-test.txt | 2 +- swh/provenance/__init__.py | 14 +- swh/provenance/api/client.py | 528 +++++++++++++++++++- swh/provenance/api/server.py | 844 +++++++++++++++++++++++++++++--- swh/provenance/cli.py | 33 +- swh/provenance/interface.py | 9 + swh/provenance/mongo/backend.py | 3 + swh/provenance/postgresql/provenance.py | 3 + swh/provenance/provenance.py | 151 +++--- swh/provenance/tests/conftest.py | 69 +-- tox.ini | 3 +- 14 files changed, 1474 insertions(+), 197 deletions(-)
Changes applied before test
commit 65d40bd64f74983ccef997f25e696baeb5d78173
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:21:42 2021 +0200
Add support for StatsD to provenance backend
commit eb28c1d07615b602660c53bdab3e04b01ee8289a
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:17:34 2021 +0200
Split `Provenance::flush` method in two (one per layer)
Fix preferred origin setting on origin-revision layer.
commit 4995134725cb5e62bcdf1d0c19ea4e2fabac5465
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 1 11:27:02 2021 +0200
Remove old client/server storage based on `swh.core.api.RPCClient`
This implementation was a first attempt for conflict resolution that didn't worked as expected.
The new `ProvenanceStorageRabbitMQ` remote storage should be used instead.
commit 50a1441f2068a2abe1293be893e66806cf0977b9
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Sep 21 16:13:53 2021 +0200
Add support for remote backend on existing storage tests
commit ba22e90f707688aed716d20253256ee12c414a33
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 13:39:59 2021 +0200
Improve server/client shoutdown logic and error handling
commit 4ec6d3359e20920469cde43516bc120d4352f915
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Aug 31 13:36:34 2021 +0200
Rework `ProvenanceStorageRabbitMQWorker` to handle connection loss
Use `pika.SelectConnection` and make an explicit handle of its life-cycle.
Improve connection error handling on both client and server side.
Change the RabbitMQ scheme to use 5 exchanges (one per entity + location).
Each exchange handles all entity related insertions, dispatching to different
queues depending on the requested `ProvenanceStorageInterface` methods (16
queues per methods). For instance, the `content` exchange handles all requests
for `content_add` and `relation_add` for both relations `CNT_EARLY_IN_REV` and
`CNT_IN_DIR` (ie. relations with content as source). In each case, requests
are forwarded to 1 of 16 possible workers, depending on the sha1 id of the
content.
commit f6f174fb76b2c42435b6c75eb90c175d9cb0fca5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Aug 20 12:21:27 2021 +0200
Add new RabbitMQ-based client/server API
Get methods in the `ProvenanceStorageInterface` are called through a server that
guarantees conflict-free writings to the underlying database.
Set methods are called directly from the client to avoid RCP overhead for reads.
The server spawns multiple processes to handle independent requests concurrently.
commit 437c2b4ad60e15c0b102988a99766b2c503c91d5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Sep 24 11:08:08 2021 +0200
Add `close` method to both `ProvenanceInterface` and `ProvenanceStorageInterface`
The idea is to have a mechanism to explicitly release resources when needed.
commit 6c3071493b5d3f187113493275d402a27866da95
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 16:14:10 2021 +0200
Rename remote storage backend classes
Make names consistent with the naming convention used for other components.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/381/ for more details.
Comment Actions
Build is green
Patch application report for D6351 (id=23099)
Could not rebase; Attempt merge onto 4c087ea0ec...
Updating 4c087ea..f936896 Fast-forward .gitignore | 4 +- mypy.ini | 3 + pytest.ini | 5 + requirements-test.txt | 2 +- swh/provenance/__init__.py | 14 +- swh/provenance/api/client.py | 528 +++++++++++++++++++- swh/provenance/api/server.py | 844 +++++++++++++++++++++++++++++--- swh/provenance/cli.py | 33 +- swh/provenance/interface.py | 9 + swh/provenance/mongo/backend.py | 3 + swh/provenance/postgresql/provenance.py | 3 + swh/provenance/provenance.py | 151 +++--- swh/provenance/tests/conftest.py | 69 +-- tox.ini | 3 +- 14 files changed, 1474 insertions(+), 197 deletions(-)
Changes applied before test
commit f93689681cce625e18ef91040ba6113074142a5d
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:21:42 2021 +0200
Add StatsD support to provenance backend
commit eb28c1d07615b602660c53bdab3e04b01ee8289a
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:17:34 2021 +0200
Split `Provenance::flush` method in two (one per layer)
Fix preferred origin setting on origin-revision layer.
commit 4995134725cb5e62bcdf1d0c19ea4e2fabac5465
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 1 11:27:02 2021 +0200
Remove old client/server storage based on `swh.core.api.RPCClient`
This implementation was a first attempt for conflict resolution that didn't worked as expected.
The new `ProvenanceStorageRabbitMQ` remote storage should be used instead.
commit 50a1441f2068a2abe1293be893e66806cf0977b9
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Sep 21 16:13:53 2021 +0200
Add support for remote backend on existing storage tests
commit ba22e90f707688aed716d20253256ee12c414a33
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 13:39:59 2021 +0200
Improve server/client shoutdown logic and error handling
commit 4ec6d3359e20920469cde43516bc120d4352f915
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Aug 31 13:36:34 2021 +0200
Rework `ProvenanceStorageRabbitMQWorker` to handle connection loss
Use `pika.SelectConnection` and make an explicit handle of its life-cycle.
Improve connection error handling on both client and server side.
Change the RabbitMQ scheme to use 5 exchanges (one per entity + location).
Each exchange handles all entity related insertions, dispatching to different
queues depending on the requested `ProvenanceStorageInterface` methods (16
queues per methods). For instance, the `content` exchange handles all requests
for `content_add` and `relation_add` for both relations `CNT_EARLY_IN_REV` and
`CNT_IN_DIR` (ie. relations with content as source). In each case, requests
are forwarded to 1 of 16 possible workers, depending on the sha1 id of the
content.
commit f6f174fb76b2c42435b6c75eb90c175d9cb0fca5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Aug 20 12:21:27 2021 +0200
Add new RabbitMQ-based client/server API
Get methods in the `ProvenanceStorageInterface` are called through a server that
guarantees conflict-free writings to the underlying database.
Set methods are called directly from the client to avoid RCP overhead for reads.
The server spawns multiple processes to handle independent requests concurrently.
commit 437c2b4ad60e15c0b102988a99766b2c503c91d5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Sep 24 11:08:08 2021 +0200
Add `close` method to both `ProvenanceInterface` and `ProvenanceStorageInterface`
The idea is to have a mechanism to explicitly release resources when needed.
commit 6c3071493b5d3f187113493275d402a27866da95
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 16:14:10 2021 +0200
Rename remote storage backend classes
Make names consistent with the naming convention used for other components.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/385/ for more details.
Comment Actions
Build is green
Patch application report for D6351 (id=23111)
Could not rebase; Attempt merge onto 4c087ea0ec...
Updating 4c087ea..f1f4b97 Fast-forward .gitignore | 4 +- mypy.ini | 3 + pytest.ini | 5 + requirements-test.txt | 2 +- swh/provenance/__init__.py | 14 +- swh/provenance/api/client.py | 528 +++++++++++++++++++- swh/provenance/api/server.py | 844 +++++++++++++++++++++++++++++--- swh/provenance/cli.py | 33 +- swh/provenance/interface.py | 9 + swh/provenance/mongo/backend.py | 3 + swh/provenance/postgresql/provenance.py | 3 + swh/provenance/provenance.py | 146 +++--- swh/provenance/tests/conftest.py | 69 +-- tox.ini | 3 +- 14 files changed, 1469 insertions(+), 197 deletions(-)
Changes applied before test
commit f1f4b9769b9d24e04b45713d729940bc3782785c
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:21:42 2021 +0200
Add StatsD support to provenance backend
commit dd1b27df175f6185fd2deed405b0325d9a09d0dd
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:17:34 2021 +0200
Split `Provenance::flush` method in two (one per layer)
commit 4995134725cb5e62bcdf1d0c19ea4e2fabac5465
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 1 11:27:02 2021 +0200
Remove old client/server storage based on `swh.core.api.RPCClient`
This implementation was a first attempt for conflict resolution that didn't worked as expected.
The new `ProvenanceStorageRabbitMQ` remote storage should be used instead.
commit 50a1441f2068a2abe1293be893e66806cf0977b9
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Sep 21 16:13:53 2021 +0200
Add support for remote backend on existing storage tests
commit ba22e90f707688aed716d20253256ee12c414a33
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 13:39:59 2021 +0200
Improve server/client shoutdown logic and error handling
commit 4ec6d3359e20920469cde43516bc120d4352f915
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Aug 31 13:36:34 2021 +0200
Rework `ProvenanceStorageRabbitMQWorker` to handle connection loss
Use `pika.SelectConnection` and make an explicit handle of its life-cycle.
Improve connection error handling on both client and server side.
Change the RabbitMQ scheme to use 5 exchanges (one per entity + location).
Each exchange handles all entity related insertions, dispatching to different
queues depending on the requested `ProvenanceStorageInterface` methods (16
queues per methods). For instance, the `content` exchange handles all requests
for `content_add` and `relation_add` for both relations `CNT_EARLY_IN_REV` and
`CNT_IN_DIR` (ie. relations with content as source). In each case, requests
are forwarded to 1 of 16 possible workers, depending on the sha1 id of the
content.
commit f6f174fb76b2c42435b6c75eb90c175d9cb0fca5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Aug 20 12:21:27 2021 +0200
Add new RabbitMQ-based client/server API
Get methods in the `ProvenanceStorageInterface` are called through a server that
guarantees conflict-free writings to the underlying database.
Set methods are called directly from the client to avoid RCP overhead for reads.
The server spawns multiple processes to handle independent requests concurrently.
commit 437c2b4ad60e15c0b102988a99766b2c503c91d5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Sep 24 11:08:08 2021 +0200
Add `close` method to both `ProvenanceInterface` and `ProvenanceStorageInterface`
The idea is to have a mechanism to explicitly release resources when needed.
commit 6c3071493b5d3f187113493275d402a27866da95
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 16:14:10 2021 +0200
Rename remote storage backend classes
Make names consistent with the naming convention used for other components.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/390/ for more details.
Comment Actions
Build is green
Patch application report for D6351 (id=23175)
Could not rebase; Attempt merge onto 4c087ea0ec...
Updating 4c087ea..4c5db27 Fast-forward .gitignore | 4 +- mypy.ini | 3 + pytest.ini | 5 + requirements-test.txt | 2 +- requirements.txt | 1 + swh/provenance/__init__.py | 28 +- swh/provenance/api/client.py | 530 +++++++++++++++++++- swh/provenance/api/server.py | 844 +++++++++++++++++++++++++++++--- swh/provenance/cli.py | 39 +- swh/provenance/interface.py | 20 + swh/provenance/mongo/backend.py | 19 +- swh/provenance/postgresql/provenance.py | 18 +- swh/provenance/provenance.py | 149 +++--- swh/provenance/tests/conftest.py | 84 ++-- tox.ini | 3 +- 15 files changed, 1529 insertions(+), 220 deletions(-)
Changes applied before test
commit 4c5db2733f7695429f92b5335142ff7ea3803182
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:21:42 2021 +0200
Add StatsD support to provenance backend
commit 2740c1611a364214acf7d6c292143043a9e827b1
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:17:34 2021 +0200
Split `Provenance::flush` method in two (one per layer)
commit 8c0d6a735afcd07e102cd90de6e5c0a6531fb141
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 1 11:27:02 2021 +0200
Remove old client/server storage based on `swh.core.api.RPCClient`
This implementation was a first attempt for conflict resolution that didn't worked as expected.
The new `ProvenanceStorageRabbitMQ` remote storage should be used instead.
commit 15fbf939c19db9f29c99b7c1d045f61967f3886c
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Sep 21 16:13:53 2021 +0200
Add support for remote backend on existing storage tests
commit 2ca0c9bdf640c7cbbcf84580aa211d8cb51d55a4
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 13:39:59 2021 +0200
Improve server/client shoutdown logic and error handling
commit 0ec7250ea299d42697cb3b480171efcd2926d049
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Tue Aug 31 13:36:34 2021 +0200
Rework `ProvenanceStorageRabbitMQWorker` to handle connection loss
Use `pika.SelectConnection` and make an explicit handle of its life-cycle.
Improve connection error handling on both client and server side.
Change the RabbitMQ scheme to use 5 exchanges (one per entity + location).
Each exchange handles all entity related insertions, dispatching to different
queues depending on the requested `ProvenanceStorageInterface` methods (16
queues per methods). For instance, the `content` exchange handles all requests
for `content_add` and `relation_add` for both relations `CNT_EARLY_IN_REV` and
`CNT_IN_DIR` (ie. relations with content as source). In each case, requests
are forwarded to 1 of 16 possible workers, depending on the sha1 id of the
content.
commit f0f3a584ea6965021990ddba926ae13c29b9560a
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Aug 20 12:21:27 2021 +0200
Add new RabbitMQ-based client/server API
Get methods in the `ProvenanceStorageInterface` are called through a server that
guarantees conflict-free writings to the underlying database.
Set methods are called directly from the client to avoid RCP overhead for reads.
The server spawns multiple processes to handle independent requests concurrently.
commit 846b20e0e9995a13591a1641bf92036ff3764be5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Sep 24 11:08:08 2021 +0200
Add `open`/`close` methods to both `ProvenanceInterface` and `ProvenanceStorageInterface`
The idea is to have a mechanism to explicitly allocate/release resources when needed.
commit 6c3071493b5d3f187113493275d402a27866da95
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 16:14:10 2021 +0200
Rename remote storage backend classes
Make names consistent with the naming convention used for other components.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/399/ for more details.
Comment Actions
Build is green
Patch application report for D6351 (id=23183)
Could not rebase; Attempt merge onto 4c087ea0ec...
Updating 4c087ea..0160d4f Fast-forward requirements-test.txt | 1 - requirements.txt | 1 + swh/provenance/__init__.py | 24 ++--- swh/provenance/api/client.py | 13 --- swh/provenance/api/server.py | 92 +------------------- swh/provenance/cli.py | 13 ++- swh/provenance/interface.py | 20 +++++ swh/provenance/mongo/backend.py | 19 +++- swh/provenance/postgresql/provenance.py | 18 ++-- swh/provenance/provenance.py | 149 ++++++++++++++++++-------------- swh/provenance/tests/conftest.py | 63 ++++++-------- 11 files changed, 173 insertions(+), 240 deletions(-)
Changes applied before test
commit 0160d4f7c3cfc3f0193b729f0c04bd2ff7ad7129
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:21:42 2021 +0200
Add StatsD support to provenance backend
commit 4f6bf0a4670e69730e47f519ac8bca6673be29f6
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:17:34 2021 +0200
Split `Provenance::flush` method in two (one per layer)
commit 8d401db34539f5df2ce2bd37080ec8ae1557417b
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 1 11:27:02 2021 +0200
Remove old client/server storage based on `swh.core.api.RPCClient`
This implementation was a first attempt for conflict resolution that didn't worked as expected.
commit 846b20e0e9995a13591a1641bf92036ff3764be5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Sep 24 11:08:08 2021 +0200
Add `open`/`close` methods to both `ProvenanceInterface` and `ProvenanceStorageInterface`
The idea is to have a mechanism to explicitly allocate/release resources when needed.
commit 6c3071493b5d3f187113493275d402a27866da95
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 16:14:10 2021 +0200
Rename remote storage backend classes
Make names consistent with the naming convention used for other components.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/405/ for more details.
Comment Actions
Build is green
Patch application report for D6351 (id=23270)
Could not rebase; Attempt merge onto 4c087ea0ec...
Updating 4c087ea..e2a1843 Fast-forward requirements-test.txt | 1 - requirements.txt | 2 +- swh/provenance/__init__.py | 24 ++--- swh/provenance/api/client.py | 13 --- swh/provenance/api/server.py | 92 +----------------- swh/provenance/cli.py | 71 +++++++------- swh/provenance/interface.py | 47 ++++++++- swh/provenance/mongo/backend.py | 37 ++++++- swh/provenance/postgresql/provenance.py | 36 +++++-- swh/provenance/provenance.py | 165 +++++++++++++++++++------------- swh/provenance/tests/conftest.py | 61 +++++------- 11 files changed, 268 insertions(+), 281 deletions(-)
Changes applied before test
commit e2a1843d5ebe01a9cdfe46b6b74dde1e293b8c01
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:21:42 2021 +0200
Add StatsD support to provenance backend
commit 246e55f9b7e3475ea4509e08370827a3190db916
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Mon Sep 27 15:17:34 2021 +0200
Split `Provenance::flush` method in two (one per layer)
commit f0210c3753c3a4122ee3c54f7fac97d170a142fa
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Fri Sep 24 11:08:08 2021 +0200
Add `open`/`close` methods to both `ProvenanceInterface` and `ProvenanceStorageInterface`
This allows to have an explicit mechanism to allocate/release resources when needed.
The necessary methods for the classes implementing these interfaces to be turned in contexts
managers are added as well (ie. `__enter__`/`__exit__`).
commit 172e327c25883bee768a9c16b850ce6aab7e2eb2
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date: Wed Sep 15 16:14:10 2021 +0200
Remove remote provenance storage based on `swh.core.api.RPCClient`
This implementation was a first attempt for conflict resolution that didn't worked as expected.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/421/ for more details.