Page MenuHomeSoftware Heritage

replay: Fix detection of external paths to remove
ClosedPublic

Authored by anlambert on Jan 21 2022, 5:44 PM.

Details

Summary

When trying to detect if some external paths previously set in a revision
need to be removed when the content of a svn:externals property is modified,
the external URL and the target revision must also be taken into account.

Previously, only the external path was considered but the external URL
or the revision might have changed. In that case, the external path must
also be removed before trying to export it again, otherwise the loader
will consider it already exported and skip the operation.

Related to T3870

Should fix latest staging events in SWH-LOADER-SVN-2Y.

Depends on D7017

Diff Detail

Repository
rDLDSVN Subversion (SVN) loader
Branch
external-to-remove-detection-fix
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 26279
Build 41086: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 41085: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D7019 (id=25440)

Could not rebase; Attempt merge onto e6e733005d...

Updating e6e7330..3e4d5d7
Fast-forward
 swh/loader/svn/replay.py            |  27 +++--
 swh/loader/svn/tests/test_loader.py | 211 ++++++++++++++++++++++++++++++++++++
 2 files changed, 230 insertions(+), 8 deletions(-)
Changes applied before test
commit 3e4d5d7cc9d74fb09b547c8ae17aa07a42b53121
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Jan 21 17:19:22 2022 +0100

    replay: Fix detection of external paths to remove
    
    When trying to detect if some external paths previously set in a revision
    need to be removed when the content of a svn:externals property is modified,
    the external URL and the target revision must also be taken into account.
    
    Previously, only the external path was considered but the external URL
    or the revision might have changed. In that case, the external path must
    also be removed before trying to export it again, otherwise the loader
    will consider it already exported and skip the operation.
    
    Related to T3870

commit 4568f71b9836a5d4ffdfddbd07d1e20b0cbb1c60
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Jan 21 17:01:01 2022 +0100

    replay: Robustify external path removal implementation
    
    When removing the subpaths of an external path, we first need to check
    if a subpath is not versioned instead of checking if it is empty.
    
    If a subpath is versioned, we can stop the paths removal process.
    
    Relate to T3870

commit eb34b0c059ea91d2a2d6a91194c3b70eb7bdc546
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Jan 21 16:30:14 2022 +0100

    replay: Use peg_rev parameter of client.export instead of rev
    
    It exist cases where there is an ambiguity for subversion to export a path
    from a given revision.
    
    See https://edoras.sdsu.edu/doc/svn-book-html-chunk/svn.advanced.pegrevs.html
    for more details.
    
    To remove the ambiguity, the peg_rev parameter of the export operation must
    be used instead of the rev one.
    
    Related to T3870

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/266/ for more details.

This revision is now accepted and ready to land.Jan 21 2022, 5:59 PM