Page MenuHomeSoftware Heritage

Clean up history graph implementation
ClosedPublic

Authored by aeviso on Dec 17 2021, 12:32 PM.

Details

Summary

Unused flags are removed to avoid unnecessary queries to the
provenance storage. Methods in the ProvenanceInterface used
to query these flags are also removed as they are no longer
necessary.

Diff Detail

Repository
rDPROV Provenance database
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D6862 (id=24865)

Could not rebase; Attempt merge onto bef0ab385d...

Updating bef0ab3..132f976
Fast-forward
 swh/provenance/graph.py                            |  87 ++-------
 swh/provenance/interface.py                        |  12 --
 swh/provenance/origin.py                           |  15 +-
 swh/provenance/provenance.py                       |  10 -
 swh/provenance/revision.py                         |   4 +-
 .../tests/data/graphs_cmdbts2_lower_1.yaml         |  31 ----
 .../tests/data/graphs_cmdbts2_lower_2.yaml         |  31 ----
 .../tests/data/graphs_cmdbts2_upper_1.yaml         |  27 ---
 .../tests/data/graphs_cmdbts2_upper_2.yaml         |  26 ---
 .../tests/data/graphs_out-of-order_lower_1.yaml    |  19 +-
 .../data/history_graphs_with-merges_visits-01.yaml | 201 +++++----------------
 swh/provenance/tests/test_history_graph.py         |   3 +-
 swh/provenance/tests/test_isochrone_graph.py       |   1 -
 13 files changed, 74 insertions(+), 393 deletions(-)
Changes applied before test
commit 132f9769e3fe4f5849aef042281664bdc10d1bbe
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Dec 17 12:29:13 2021 +0100

    Clean up history graph implementation
    
    Unused flags are removed to avoid unnecessary queries to the
    provenance storage. Methods in the `ProvenanceInterface` used
    to query these flags are also removed as they are no longer
    necessary.

commit 3da57b271d046d8fa9c0dc589ae9d5ca3c164618
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Dec 15 16:43:02 2021 +0100

    Clean up code after frontier redefinition from previous commit
    
    The `known` flag from `IsochroneNode` is no longer being used, and
    `maxdate` calculation can thus be simplified. Isochrone graph related
    tests are updated as well to reflect the changes.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/552/ for more details.

This revision is now accepted and ready to land.Dec 17 2021, 2:45 PM

Build is green

Patch application report for D6862 (id=24875)

Could not rebase; Attempt merge onto bef0ab385d...

Updating bef0ab3..b439a08
Fast-forward
 swh/provenance/graph.py                            |  89 ++-------
 swh/provenance/interface.py                        |  12 --
 swh/provenance/origin.py                           |  15 +-
 swh/provenance/provenance.py                       |  10 -
 swh/provenance/revision.py                         |   4 +-
 .../tests/data/graphs_cmdbts2_lower_1.yaml         |  31 ----
 .../tests/data/graphs_cmdbts2_lower_2.yaml         |  31 ----
 .../tests/data/graphs_cmdbts2_upper_1.yaml         |  27 ---
 .../tests/data/graphs_cmdbts2_upper_2.yaml         |  26 ---
 .../tests/data/graphs_out-of-order_lower_1.yaml    |  19 +-
 .../data/history_graphs_with-merges_visits-01.yaml | 201 +++++----------------
 swh/provenance/tests/test_history_graph.py         |   3 +-
 swh/provenance/tests/test_isochrone_graph.py       |   1 -
 13 files changed, 75 insertions(+), 394 deletions(-)
Changes applied before test
commit b439a081e08ab2fc120a2a844faccacf882a71fc
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Dec 17 12:29:13 2021 +0100

    Clean up history graph implementation
    
    Unused flags are removed to avoid unnecessary queries to the
    provenance storage. Methods in the `ProvenanceInterface` used
    to query these flags are also removed as they are no longer
    necessary.

commit a2b7570ed913261443a36dfda5ab54828981ad3c
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Dec 20 12:19:19 2021 +0100

    Move date calculation for invalidated frontiers to the revision processing module
    
    Up to now, when a frontier was invalidated, the isochrone graph building function
    was updating its associated max date to match that of the revision being processed.
    This is now done in the revision processing module, where it is clearer when and
    why the update is required.

commit 1926b9d575fb7c406f02fea52f03cb9ab0ef5b70
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Dec 15 16:43:02 2021 +0100

    Clean up code after frontier redefinition from previous commit
    
    The `known` flag from `IsochroneNode` is no longer being used, and
    `maxdate` calculation can thus be simplified. Isochrone graph related
    tests are updated as well to reflect the changes.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/555/ for more details.

This revision was automatically updated to reflect the committed changes.