Clean up history graph implementation
ClosedPublic
Actions

Authored by aeviso on Dec 17 2021, 12:32 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Commits

rDPROVb439a081e08a: Clean up history graph implementation

Summary

Unused flags are removed to avoid unnecessary queries to the
provenance storage. Methods in the ProvenanceInterface used
to query these flags are also removed as they are no longer
necessary.

Diff Detail

Repository

rDPROV Provenance database

Branch

master

Lint

Lint Skipped

Unit

Unit Tests Skipped

Build Status

Buildable 25732
Build 40220: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 40219: arc lint + arc unit

Event Timeline

aeviso created this revision.Dec 17 2021, 12:32 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptDec 17 2021, 12:32 PM

Build is green

Patch application report for D6862 (id=24865)

Could not rebase; Attempt merge onto bef0ab385d...

Updating bef0ab3..132f976
Fast-forward
 swh/provenance/graph.py                            |  87 ++-------
 swh/provenance/interface.py                        |  12 --
 swh/provenance/origin.py                           |  15 +-
 swh/provenance/provenance.py                       |  10 -
 swh/provenance/revision.py                         |   4 +-
 .../tests/data/graphs_cmdbts2_lower_1.yaml         |  31 ----
 .../tests/data/graphs_cmdbts2_lower_2.yaml         |  31 ----
 .../tests/data/graphs_cmdbts2_upper_1.yaml         |  27 ---
 .../tests/data/graphs_cmdbts2_upper_2.yaml         |  26 ---
 .../tests/data/graphs_out-of-order_lower_1.yaml    |  19 +-
 .../data/history_graphs_with-merges_visits-01.yaml | 201 +++++----------------
 swh/provenance/tests/test_history_graph.py         |   3 +-
 swh/provenance/tests/test_isochrone_graph.py       |   1 -
 13 files changed, 74 insertions(+), 393 deletions(-)

Changes applied before test

commit 132f9769e3fe4f5849aef042281664bdc10d1bbe
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Dec 17 12:29:13 2021 +0100

    Clean up history graph implementation
    
    Unused flags are removed to avoid unnecessary queries to the
    provenance storage. Methods in the `ProvenanceInterface` used
    to query these flags are also removed as they are no longer
    necessary.

commit 3da57b271d046d8fa9c0dc589ae9d5ca3c164618
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Dec 15 16:43:02 2021 +0100

    Clean up code after frontier redefinition from previous commit
    
    The `known` flag from `IsochroneNode` is no longer being used, and
    `maxdate` calculation can thus be simplified. Isochrone graph related
    tests are updated as well to reflect the changes.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/552/ for more details.

Harbormaster completed remote builds in B25732: Diff 24865.Dec 17 2021, 12:38 PM

aeviso requested review of this revision.Dec 17 2021, 12:38 PM

vlorentz accepted this revision.Dec 17 2021, 2:45 PM

This revision is now accepted and ready to land.Dec 17 2021, 2:45 PM

rebase

aeviso edited the summary of this revision. (Show Details)Dec 20 2021, 12:25 PM

aeviso edited parent revisions, added: D6864: Move date calculation for invalidated frontiers to the revision processing module; removed: D6846: Clean up code after frontier redefinition from previous commit.

Build is green

Patch application report for D6862 (id=24875)

Could not rebase; Attempt merge onto bef0ab385d...

Updating bef0ab3..b439a08
Fast-forward
 swh/provenance/graph.py                            |  89 ++-------
 swh/provenance/interface.py                        |  12 --
 swh/provenance/origin.py                           |  15 +-
 swh/provenance/provenance.py                       |  10 -
 swh/provenance/revision.py                         |   4 +-
 .../tests/data/graphs_cmdbts2_lower_1.yaml         |  31 ----
 .../tests/data/graphs_cmdbts2_lower_2.yaml         |  31 ----
 .../tests/data/graphs_cmdbts2_upper_1.yaml         |  27 ---
 .../tests/data/graphs_cmdbts2_upper_2.yaml         |  26 ---
 .../tests/data/graphs_out-of-order_lower_1.yaml    |  19 +-
 .../data/history_graphs_with-merges_visits-01.yaml | 201 +++++----------------
 swh/provenance/tests/test_history_graph.py         |   3 +-
 swh/provenance/tests/test_isochrone_graph.py       |   1 -
 13 files changed, 75 insertions(+), 394 deletions(-)

Changes applied before test

commit b439a081e08ab2fc120a2a844faccacf882a71fc
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Dec 17 12:29:13 2021 +0100

    Clean up history graph implementation
    
    Unused flags are removed to avoid unnecessary queries to the
    provenance storage. Methods in the `ProvenanceInterface` used
    to query these flags are also removed as they are no longer
    necessary.

commit a2b7570ed913261443a36dfda5ab54828981ad3c
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Dec 20 12:19:19 2021 +0100

    Move date calculation for invalidated frontiers to the revision processing module
    
    Up to now, when a frontier was invalidated, the isochrone graph building function
    was updating its associated max date to match that of the revision being processed.
    This is now done in the revision processing module, where it is clearer when and
    why the update is required.

commit 1926b9d575fb7c406f02fea52f03cb9ab0ef5b70
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Dec 15 16:43:02 2021 +0100

    Clean up code after frontier redefinition from previous commit
    
    The `known` flag from `IsochroneNode` is no longer being used, and
    `maxdate` calculation can thus be simplified. Isochrone graph related
    tests are updated as well to reflect the changes.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/555/ for more details.

Harbormaster completed remote builds in B25741: Diff 24875.Dec 20 2021, 12:31 PM

Closed by commit rDPROVb439a081e08a: Clean up history graph implementation (authored by aeviso). · Explain WhyDec 21 2021, 3:52 PM

This revision was automatically updated to reflect the committed changes.

aeviso added a commit: rDPROVb439a081e08a: Clean up history graph implementation.