Page MenuHomeSoftware Heritage

Adapt extid filtering so it happens server side
ClosedPublic

Authored by ardumont on Sep 16 2021, 8:44 AM.

Details

Summary

This lifts the storage endpoint adaptations which allow to filter directly server side
server (storage > v0.37).

Related to T3567

Test Plan

tox (happy if v0.37 is released)

Diff Detail

Repository
rDLDHG Mercurial loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D6275 (id=22726)

Could not rebase; Attempt merge onto bf39050051...

Updating bf39050..0d05077
Fast-forward
 requirements-swh.txt                         |   2 +-
 swh/loader/mercurial/from_disk.py            | 138 +++++++++++++++++++--------
 swh/loader/mercurial/tests/test_from_disk.py | 124 +++++++++++++++---------
 3 files changed, 180 insertions(+), 84 deletions(-)
Changes applied before test
commit 0d0507768ad5d19038e77bdcf48557e4c8158937
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Sep 16 08:40:49 2021 +0200

    Adapt extid filtering so it happens server side
    
    This lift the endpoint adaptations which allow the filtering directly in the
    server (storage > v0.37).
    
    Related to T3567

commit da1bd7c460b238edcbfdf8c7e931a4cf6fd9ab58
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Sep 15 14:53:24 2021 +0200

    Build snapshot on visits
    
    If an uneventful visit happens, then an empty snapshot should happen. When filtering
    revisions for speed, still the heads should appear in the snapshot.
    
    Related to T3571

commit 4c9eb38ec516de3f65c3996d937cd3328ded6261
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Sep 10 17:00:15 2021 +0200

    Add support to filter out already seen revisions across hg origins
    
    This uses the extid mapping from the storage to actually achieve this.
    
    Related to T3563

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/303/ for more details.

ardumont edited the summary of this revision. (Show Details)
anlambert added a subscriber: anlambert.

Looks good to me.

This revision is now accepted and ready to land.Sep 16 2021, 10:54 AM

Build is green

Patch application report for D6275 (id=22746)

Could not rebase; Attempt merge onto bf39050051...

Updating bf39050..8e3b880
Fast-forward
 requirements-swh.txt                         |   2 +-
 swh/loader/mercurial/from_disk.py            | 135 +++++++++++++++++++--------
 swh/loader/mercurial/tests/test_from_disk.py | 124 +++++++++++++++---------
 3 files changed, 178 insertions(+), 83 deletions(-)
Changes applied before test
commit 8e3b880ebcb85697a476aa002fa0808d0107d821
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Sep 16 08:40:49 2021 +0200

    Adapt extid filtering so it happens server side
    
    This lift the endpoint adaptations which allow the filtering directly in the
    server (storage > v0.37).
    
    Related to T3567

commit a0f8e9e856a4d659e9e73d5973f4c87884b79f6f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Sep 15 14:53:24 2021 +0200

    Build snapshot on visits
    
    If an uneventful visit happens, then an empty snapshot should happen. When filtering
    revisions for speed, still the heads should appear in the snapshot.
    
    Related to T3571

commit 765cb5aefa8bb370bc1343d990ad328188a4eba5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Sep 10 17:00:15 2021 +0200

    Add support to filter out already seen revisions across hg origins
    
    This uses the extid mapping from the storage to actually achieve this.
    
    Related to T3563

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/306/ for more details.