Page MenuHomeSoftware Heritage

Replace ProvenanceDB.remove_cache by a dict of Set
ClosedPublic

Authored by douardda on May 25 2021, 10:18 AM.

Details

Summary

instead of a dict of dict, which was actually used as a set.

Depends in D5773

Diff Detail

Repository
rDPROV Provenance database
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D5774 (id=20639)

Could not rebase; Attempt merge onto b2ddef88eb...

Updating b2ddef8..cd28ce4
Fast-forward
 swh/provenance/model.py                        |  21 ++-
 swh/provenance/postgresql/provenancedb_base.py |  22 +--
 swh/provenance/provenance.py                   | 215 +++++++++++++++----------
 swh/provenance/tests/test_provenance_db.py     |  96 ++++++++++-
 4 files changed, 251 insertions(+), 103 deletions(-)
Changes applied before test
commit cd28ce4dbbc53837bc8ec0fa47e471ff3aaab2dc
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri May 21 19:34:07 2021 +0200

    Replace ProvenanceDB.remove_cache by a dict of Set
    
    instead of a dict of dict, which was actually used as a set.

commit 53fe043bc837336685d201ae3741ad461c3217ff
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 12 09:47:03 2021 +0200

    Refactor the isochrone graph computation
    
    attempt to simplify a bit this part of the code:
    
    - IsochroneNode are now only used for directories
    - FileEntry are stored in a new IsochroneNode.files attribute, so
    - IsochroneNode.children only stores IsochroneNode (thus DirectoryEntry)
      objects,
    - rename IsochroneNode.date as 'dbdate' and clarify its semantics

commit 218b9bfdc14864d4bc82e7ee3541de6230dbc33b
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 19 16:16:53 2021 +0200

    Add 'ls_files()' and 'ls_dirs()' methods to the DirectoryEntry class
    
    to make it a bit easier to compute the isochrone graph (see following
    revisions).

commit eeb108e100d94bb67db466616c7af311385ae360
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 19 16:14:41 2021 +0200

    Add __str__ methods to RevisionEntry, DirectoryEntry and FileEntry
    
    to ease logging and debugging.

commit 2ea255d88bea66963f0815768ddaa65af0517112
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 12 12:44:07 2021 +0200

    Improve a bit the code of ProvenanceDBBase

commit 235cb4ade51da82b36c9bfeae9c7415681ec4b1b
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 12 10:30:23 2021 +0200

    Add a test for the build_isochrone_graph() function
    
    this test is far from ideal, since it's mostly the record of what happen
    during a "known good" session of revision insertions, but at least it
    should allow to refactor code related to the isochrone graph computation
    with a bit more confidence...

commit 6ae8de468d94ab3398016add37747a13e43e8cac
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue May 11 15:33:57 2021 +0200

    Replace the 'dates' argument of IsochroneNode() by a simple 'date' one
    
    there is no need for passing a dict here, we only care about the date
    for the node being instanciated.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/25/ for more details.

This revision is now accepted and ready to land.May 25 2021, 10:25 AM

Build is green

Patch application report for D5774 (id=20672)

Could not rebase; Attempt merge onto 77fce4e59d...

Updating 77fce4e..4891976
Fast-forward
 swh/provenance/model.py                        |  21 ++-
 swh/provenance/postgresql/provenancedb_base.py |  22 +--
 swh/provenance/provenance.py                   | 202 +++++++++++++++----------
 swh/provenance/tests/test_provenance_db.py     |  96 +++++++++++-
 4 files changed, 243 insertions(+), 98 deletions(-)
Changes applied before test
commit 489197686cfa447dc4ee0a827ef578cdc041b1d4
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri May 21 19:34:07 2021 +0200

    Replace ProvenanceDB.remove_cache by a dict of Set
    
    instead of a dict of dict, which was actually used as a set.

commit 2860fe6126de1804ab9190026545c935bbbbd99a
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 12 09:47:03 2021 +0200

    Refactor the isochrone graph computation
    
    attempt to simplify a bit this part of the code:
    
    - IsochroneNode are now only used for directories
    - FileEntry are stored in a new IsochroneNode.files attribute, so
    - IsochroneNode.children only stores IsochroneNode (thus DirectoryEntry)
      objects,
    - rename IsochroneNode.date as 'dbdate' and clarify its semantics

commit 8e4a2f69b53fd8ef74613509eb7a5f6707855a7a
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 19 16:16:53 2021 +0200

    Add 'ls_files()' and 'ls_dirs()' methods to the DirectoryEntry class
    
    to make it a bit easier to compute the isochrone graph (see following
    revisions).

commit 4ba05e92698d73a6a05078a969eea85d08cd5dca
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 19 16:14:41 2021 +0200

    Add __str__ methods to RevisionEntry, DirectoryEntry and FileEntry
    
    to ease logging and debugging.

commit fb0ef598657a9810deac43d495ab882718265543
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 12 12:44:07 2021 +0200

    Improve a bit the code of ProvenanceDBBase

commit edff905d0df269cb90246ef77b554b1ca58cbeef
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed May 12 10:30:23 2021 +0200

    Add a test for the build_isochrone_graph() function
    
    this test is far from ideal, since it's mostly the record of what happen
    during a "known good" session of revision insertions, but at least it
    should allow to refactor code related to the isochrone graph computation
    with a bit more confidence...

commit 113e11031aa5365a9244409a0cfe646061cb94f9
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue May 11 15:33:57 2021 +0200

    Replace the 'dates' argument of IsochroneNode() by a simple 'date' one
    
    there is no need for passing a dict here, we only care about the date
    for the node being instanciated.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/35/ for more details.

Build is green

Patch application report for D5774 (id=20730)

Rebasing onto 113e11031a...

Current branch diff-target is up to date.
Changes applied before test
commit 5aa0314dd72dab52b660a330f4f7f53ae727bb97
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri May 21 19:34:07 2021 +0200

    Replace ProvenanceDB.remove_cache by a dict of Set
    
    instead of a dict of dict, which was actually used as a set.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/39/ for more details.