Page MenuHomeSoftware Heritage
Feed Advanced Search

Jun 3 2021

douardda closed D5813: Make ProvenanceDB.commit() optionally raise the exception.
Jun 3 2021, 3:16 PM
douardda committed rDPROV33eada55f05a: Make ProvenanceDB.commit() optionally raise the exception (authored by douardda).
Make ProvenanceDB.commit() optionally raise the exception
Jun 3 2021, 3:16 PM
douardda updated the diff for D5813: Make ProvenanceDB.commit() optionally raise the exception.

rebase on master and apply vlorentz' suggestion

Jun 3 2021, 3:11 PM
douardda updated the summary of D5813: Make ProvenanceDB.commit() optionally raise the exception.
Jun 3 2021, 3:09 PM
douardda added a comment to D5811: Add a simple out-of-order dataset.

@aeviso I have a question: in this scenario, do we expect the date in the directory table for A/C and A/B/C (sha1 c9cabe7f49012e3fdef6ac6b929efb5654f583cf) directories to be invalidated or updated due to the arrival, at rev R05, of a version of the b file dated earlier than originally seen in R01?

Jun 3 2021, 10:52 AM
douardda updated the diff for D5813: Make ProvenanceDB.commit() optionally raise the exception.

rebase

Jun 3 2021, 10:08 AM
douardda updated the diff for D5812: Add a test_provenance_heuristics_content_find_all() test.

rebase

Jun 3 2021, 10:08 AM
douardda updated the diff for D5811: Add a simple out-of-order dataset.

rebase

Jun 3 2021, 10:07 AM
douardda updated the diff for D5805: Add a new (git) dataset generation scaffolding for tests.

rebase

Jun 3 2021, 10:07 AM
douardda updated the diff for D5781: Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all).

rebase

Jun 3 2021, 10:06 AM
douardda updated the diff for D5780: Add a test for content_find_all().

rebase

Jun 3 2021, 10:05 AM
douardda updated the diff for D5773: Refactor the isochrone graph computation.

rebase

Jun 3 2021, 10:05 AM
douardda updated the diff for D5772: Add a test for the build_isochrone_graph() function.

rebase

Jun 3 2021, 10:04 AM

Jun 2 2021

douardda updated the diff for D5813: Make ProvenanceDB.commit() optionally raise the exception.

rebase

Jun 2 2021, 5:39 PM
douardda updated the diff for D5812: Add a test_provenance_heuristics_content_find_all() test.

rebase

Jun 2 2021, 5:38 PM
douardda updated the diff for D5811: Add a simple out-of-order dataset.

rebase

Jun 2 2021, 5:38 PM
douardda updated the diff for D5805: Add a new (git) dataset generation scaffolding for tests.

actually pick the correct revisions for this diff...

Jun 2 2021, 5:37 PM
douardda updated the diff for D5805: Add a new (git) dataset generation scaffolding for tests.

rebase

Jun 2 2021, 5:35 PM
douardda updated the diff for D5781: Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all).

rebase

Jun 2 2021, 5:35 PM
douardda updated the diff for D5780: Add a test for content_find_all().

rebase

Jun 2 2021, 5:34 PM
douardda updated the diff for D5773: Refactor the isochrone graph computation.

fix typos, add more comments/docstrings, and remove completly the IsochroneNode.files attribute

Jun 2 2021, 5:33 PM
douardda added a comment to D5781: Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all).

This diff is already accepted by ardumont, who knows SQL better than I do. There is no much for me to review here

Jun 2 2021, 5:25 PM
douardda added a comment to D5781: Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all).

This diff is already accepted by ardumont, who knows SQL better than I do. There is no much for me to review here

Jun 2 2021, 5:19 PM
douardda added a comment to D5813: Make ProvenanceDB.commit() optionally raise the exception.

Why would we want to do this? The actual idea is to guarantee that the commit method never fails, so we can remove the while not provenance.commit() line in the revision_add function.

Jun 2 2021, 4:58 PM
douardda added a comment to D5781: Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all).

We are about to start refactoring the provenance backend. It would be nice to have this changes pushed, since re-basing them might by complicated after refactoring.

Jun 2 2021, 4:47 PM
douardda updated subscribers of D5812: Add a test_provenance_heuristics_content_find_all() test.
Jun 2 2021, 3:28 PM
douardda updated subscribers of D5811: Add a simple out-of-order dataset.

@aeviso I have a question: in this scenario, do we expect the date in the directory table for A/C and A/B/C (sha1 c9cabe7f49012e3fdef6ac6b929efb5654f583cf) directories to be invalidated or updated due to the arrival, at rev R05, of a version of the b file dated earlier than originally seen in R01?

Jun 2 2021, 3:05 PM
douardda added inline comments to D5773: Refactor the isochrone graph computation.
Jun 2 2021, 2:46 PM
douardda added a comment to D5773: Refactor the isochrone graph computation.

I'm not really sure this new algorithm does the same as the previous one. Some subtle things were changed and I have the filling the semantics are different now. Also, I found the previous version to be clearer, I rather stay with it

Jun 2 2021, 12:34 PM
douardda requested review of D5813: Make ProvenanceDB.commit() optionally raise the exception.
Jun 2 2021, 12:33 PM
douardda requested review of D5812: Add a test_provenance_heuristics_content_find_all() test.
Jun 2 2021, 12:30 PM
douardda requested review of D5811: Add a simple out-of-order dataset.
Jun 2 2021, 12:28 PM

Jun 1 2021

douardda requested review of D5805: Add a new (git) dataset generation scaffolding for tests.
Jun 1 2021, 4:42 PM
douardda updated the diff for D5781: Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all).

rebase

Jun 1 2021, 11:33 AM
douardda updated the diff for D5780: Add a test for content_find_all().

rebase + use set() to compare expected results, as requested by aeviso

Jun 1 2021, 11:31 AM
douardda updated the diff for D5773: Refactor the isochrone graph computation.

rebase

Jun 1 2021, 11:29 AM
douardda updated the diff for D5772: Add a test for the build_isochrone_graph() function.

rebase

Jun 1 2021, 11:26 AM
douardda closed D5774: Replace ProvenanceDB.remove_cache by a dict of Set.
Jun 1 2021, 11:25 AM
douardda committed rDPROV5aa0314dd72d: Replace ProvenanceDB.remove_cache by a dict of Set (authored by douardda).
Replace ProvenanceDB.remove_cache by a dict of Set
Jun 1 2021, 11:25 AM
douardda added inline comments to D5780: Add a test for content_find_all().
Jun 1 2021, 11:24 AM
douardda added a comment to D5780: Add a test for content_find_all().

what about disabling E501 for the whole file, instead of repeating the comment? https://stackoverflow.com/a/64431741

Jun 1 2021, 11:15 AM
douardda updated the diff for D5774: Replace ProvenanceDB.remove_cache by a dict of Set.

rebase

Jun 1 2021, 11:00 AM
douardda added a comment to D5772: Add a test for the build_isochrone_graph() function.

Not sure if I follow what the test is doing since it not clear to me what's the content of storage_and_CMDBTS,

Jun 1 2021, 10:55 AM
douardda closed D5771: Replace the 'dates' argument of IsochroneNode() by a simple 'date' one.
Jun 1 2021, 10:42 AM
douardda committed rDPROV113e11031aa5: Replace the 'dates' argument of IsochroneNode() by a simple 'date' one (authored by douardda).
Replace the 'dates' argument of IsochroneNode() by a simple 'date' one
Jun 1 2021, 10:42 AM

May 28 2021

douardda added inline comments to D5773: Refactor the isochrone graph computation.
May 28 2021, 11:06 AM
douardda added inline comments to D5781: Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all).
May 28 2021, 10:51 AM
douardda updated subscribers of D5781: Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all).
May 28 2021, 10:43 AM
douardda updated subscribers of D5780: Add a test for content_find_all().
May 28 2021, 10:42 AM
douardda updated subscribers of D5773: Refactor the isochrone graph computation.
May 28 2021, 10:42 AM
douardda updated subscribers of D5772: Add a test for the build_isochrone_graph() function.
May 28 2021, 10:41 AM
douardda added a reviewer for D5780: Add a test for content_find_all(): aeviso.
May 28 2021, 10:40 AM
douardda added inline comments to D5773: Refactor the isochrone graph computation.
May 28 2021, 10:31 AM

May 26 2021

douardda added a comment to D5773: Refactor the isochrone graph computation.

I'm not really sure this new algorithm does the same as the previous one. Some subtle things were changed and I have the filling the semantics are different now. Also, I found the previous version to be clearer, I rather stay with it

May 26 2021, 5:44 PM

May 25 2021

douardda requested review of D5781: Simplify DB queries in ProvenanceWithPathDB.content_find_(first|all).
May 25 2021, 4:52 PM
douardda requested review of D5780: Add a test for content_find_all().
May 25 2021, 4:49 PM
douardda updated the diff for D5774: Replace ProvenanceDB.remove_cache by a dict of Set.

rebase

May 25 2021, 4:34 PM
douardda updated the diff for D5773: Refactor the isochrone graph computation.

rebase

May 25 2021, 4:33 PM
douardda updated the diff for D5772: Add a test for the build_isochrone_graph() function.

rebase

May 25 2021, 4:33 PM
douardda updated the diff for D5771: Replace the 'dates' argument of IsochroneNode() by a simple 'date' one.

rebase + also apply to add_child()

May 25 2021, 4:32 PM
douardda committed rDPROV77fce4e59d41: Fix a couple of typos (authored by douardda).
Fix a couple of typos
May 25 2021, 4:31 PM
douardda updated the diff for D5774: Replace ProvenanceDB.remove_cache by a dict of Set.

rebase

May 25 2021, 3:26 PM
douardda updated the diff for D5773: Refactor the isochrone graph computation.

small improvement of a comment + type annotation for the local variable 'fdates'

May 25 2021, 3:26 PM
douardda added inline comments to D5769: client: Open a paginated list user deposits endpoint.
May 25 2021, 10:32 AM
douardda requested review of D5774: Replace ProvenanceDB.remove_cache by a dict of Set.
May 25 2021, 10:21 AM
douardda added a reviewer for D5773: Refactor the isochrone graph computation: aeviso.
May 25 2021, 10:20 AM
douardda requested review of D5773: Refactor the isochrone graph computation.
May 25 2021, 10:19 AM
douardda added a reviewer for D5772: Add a test for the build_isochrone_graph() function: aeviso.
May 25 2021, 10:19 AM
douardda requested review of D5772: Add a test for the build_isochrone_graph() function.
May 25 2021, 10:18 AM
douardda requested review of D5771: Replace the 'dates' argument of IsochroneNode() by a simple 'date' one.
May 25 2021, 10:16 AM

May 21 2021

douardda accepted D5750: cassandra: Add support for ScyllaDB.

consider the minor fix in the README file, otherwise ok, thanks!

May 21 2021, 10:27 AM
douardda added inline comments to D5743: azure: Specialize get_batch() to fetch objects concurrently.
May 21 2021, 10:23 AM

May 20 2021

douardda closed D5755: Fix snapshot entries in swh_model_data test data.
May 20 2021, 3:21 PM
douardda committed rDMOD4808fc2ae5ec: Fix snapshot entries in swh_model_data test data (authored by douardda).
Fix snapshot entries in swh_model_data test data
May 20 2021, 3:21 PM

May 19 2021

douardda requested review of D5755: Fix snapshot entries in swh_model_data test data.
May 19 2021, 3:24 PM
douardda triaged T3333: Document the different storage backends as Normal priority.
May 19 2021, 10:57 AM · Documentation, Storage manager
douardda requested changes to D5750: cassandra: Add support for ScyllaDB.

please also add a piece of documentation somewhere on this alt. "cassandra" backend config (aka. "how to use scylladb" or something).

May 19 2021, 10:51 AM
douardda added inline comments to D5743: azure: Specialize get_batch() to fetch objects concurrently.
May 19 2021, 10:42 AM

May 18 2021

douardda closed D5747: Make the TenaciousProxyStorage also handle content_add_metadata.
May 18 2021, 4:27 PM
douardda committed rDSTO0ed4a975d1d0: Make the TenaciousProxyStorage also handle content_add_metadata (authored by douardda).
Make the TenaciousProxyStorage also handle content_add_metadata
May 18 2021, 4:27 PM
douardda requested review of D5747: Make the TenaciousProxyStorage also handle content_add_metadata.
May 18 2021, 1:04 PM
douardda accepted D5746: Fix tests due to some dependencies upgrade.

tests are green, so it should be ok I guess

May 18 2021, 1:02 PM

May 17 2021

douardda accepted D5657: Spool large packfiles to disk instead of consuming tons of memory.

LGTM, any reason for not landing it?

May 17 2021, 1:53 PM

May 11 2021

douardda created P1040 (An Untitled Masterwork).
May 11 2021, 12:30 PM
douardda committed rDPROV0883791f9823: Update to swh-core 0.14 (authored by douardda).
Update to swh-core 0.14
May 11 2021, 12:18 PM
douardda accepted D5736: Stop setting MetadataAuthority.metadata and MetadataFetcher.metadata.
May 11 2021, 11:55 AM
douardda added inline comments to D5658: git_bare: Deduplicate object downloads and writes.
May 11 2021, 9:48 AM

May 10 2021

douardda created P1039 (An Untitled Masterwork).
May 10 2021, 5:43 PM
douardda committed rDSTOb487a21f2708: Deprecate the "local" storage cls in favor of "postgresql" (authored by douardda).
Deprecate the "local" storage cls in favor of "postgresql"
May 10 2021, 4:32 PM
douardda closed D5721: Move all proxy storages in swh/storage/proxies/ and deprecate "local" cls.
May 10 2021, 4:32 PM
douardda committed rDSTO910525392c51: Move all proxy storages in swh/storage/proxies/ (authored by douardda).
Move all proxy storages in swh/storage/proxies/
May 10 2021, 4:32 PM
douardda requested review of D5721: Move all proxy storages in swh/storage/proxies/ and deprecate "local" cls.
May 10 2021, 3:05 PM

May 7 2021

douardda closed D5680: Make the TenaciousProxyStorage retry when a single object add fails.
May 7 2021, 1:55 PM
douardda committed rDSTO761709957a40: Make the TenaciousProxyStorage retry when a single object add fails (authored by douardda).
Make the TenaciousProxyStorage retry when a single object add fails
May 7 2021, 1:55 PM
douardda updated the diff for D5680: Make the TenaciousProxyStorage retry when a single object add fails.

rebase

May 7 2021, 1:47 PM

May 6 2021

douardda accepted D5702: Rename pytest_postgresql_get_config; names starting with 'pytest_' in pytest plugins are reserved.
May 6 2021, 3:32 PM
douardda updated the diff for D5680: Make the TenaciousProxyStorage retry when a single object add fails.

rebase

May 6 2021, 2:48 PM
douardda updated the diff for D5680: Make the TenaciousProxyStorage retry when a single object add fails.

Make the number of (single-object insertion) retries configurable

May 6 2021, 2:47 PM
douardda added inline comments to D5680: Make the TenaciousProxyStorage retry when a single object add fails.
May 6 2021, 2:31 PM

May 5 2021

douardda requested review of D5680: Make the TenaciousProxyStorage retry when a single object add fails.
May 5 2021, 12:53 PM