Page MenuHomeSoftware Heritage

[duplicated] Use versioned ExtIDs in main loader mercurial implementation
AbandonedPublic

Authored by ardumont on Jul 28 2021, 5:18 PM.

Details

Reviewers
None
Group Reviewers
Reviewers
Summary

To check something.

Duplicated from D6036 to debug the flakiness.

Depends on D6041
Depends on D6040

Diff Detail

Repository
rDLDHG Mercurial loader
Branch
extid-v2
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 22809
Build 35570: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 35569: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D6043 (id=21847)

Could not rebase; Attempt merge onto f3232bfd67...

Updating f3232bf..074cad0
Fast-forward
 swh/loader/mercurial/from_disk.py         | 21 ++++++++++++++++-----
 swh/loader/mercurial/hgutil.py            | 18 ++++++++++++++++--
 swh/loader/mercurial/tests/test_hgutil.py | 12 +++++++++---
 3 files changed, 41 insertions(+), 10 deletions(-)
Changes applied before test
commit 074cad0a2db45b416adaddc852b4340cea4c9382
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jul 28 10:22:15 2021 +0200

    Use versioned ExtIDs in main loader mercurial implementation
    
    For now this hardcodes the version to 1 for either reading or writing instructions.
    
    This allows:
    - store the new hashes with a version (actually no version means version 0).
    - to keep the old loader mercurial ExtID references in the archives (no need to clean
      them up as that poses other problems regarding the journal)
    - in effect unblock the current ingestion/updates of existing origins which already have
      more than one ExtIDs due to different incompatible versions.
    
    The storage implementation does not allow filtering on the extid_version so it's up to
    the loader to do the filtering. Hence the current implementation.

commit 569a2eb73fb324e1dcddc75b287b5c1fb896f559
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jul 28 16:40:08 2021 +0200

    Debug hgutils flakiness 2/2
    
    As this timeout is more than the harcoded one second in the clone function. That must be
    the reason why now this is always working.

commit 519629942604d026cb515f7fe9f2e327af4c8e04
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jul 28 16:35:59 2021 +0200

    Debug hgutils flakiness

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/271/ for more details.