Page MenuHomeSoftware Heritage

replay: Prevent erroneous removal of versioned path
ClosedPublic

Authored by anlambert on Feb 4 2022, 4:06 PM.

Details

Summary

When processing an external, the content of the target directory is
first removed before copying external dirs and files.

But that target directory might contain versioned paths so we must
ensure to restore them after removing the directory content.

Related to T611

Fixes some events in SWH-LOADER-SVN-53

Depends on D7087

Diff Detail

Repository
rDLDSVN Subversion (SVN) loader
Branch
external-on-versioned-path-fix
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 26544
Build 41517: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 41516: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D7091 (id=25715)

Could not rebase; Attempt merge onto 09e83fd94c...

Updating 09e83fd..cb1b59c
Fast-forward
 requirements-test.txt               |   2 +-
 swh/loader/svn/loader.py            |   6 -
 swh/loader/svn/replay.py            | 265 ++++++++++++++++++++----------------
 swh/loader/svn/svn.py               |   9 +-
 swh/loader/svn/tests/test_loader.py |  53 ++++++++
 5 files changed, 210 insertions(+), 125 deletions(-)
Changes applied before test
commit cb1b59ca340132dd265784611ede974ca43adaf9
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 14:53:42 2022 +0100

    replay: Prevent erroneous removal of versioned path
    
    When processing an external, the content of the target directory is
    first removed before copying external dirs and files.
    
    But that target directory might contain versioned paths so we must
    ensure to restore them after removing the directory content.
    
    Related to T611

commit 651bb4e0b5c42ac67a8c29b2da9986261e9ff682
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 14:27:36 2022 +0100

    replay: Improve signature of DirEditor.open_file method
    
    Add path parameter for consistency with other methods.

commit d5bdea55bc527a94d9e735648a2b835ea2510fba
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 14:07:57 2022 +0100

    replay: Move external processing in a dedicated method
    
    That code grew a lot so better putting it in a dedicated method, plus
    it enables to remove one indentation level for better readability.

commit 369fbea00ea54b202a08db8541a7e6db42a0cc95
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 13:48:00 2022 +0100

    svn: Add debug log before executing checkout or export operation

commit 8bde0663cb4a72a80212e7bab21a962ddf510e25
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 13:40:52 2022 +0100

    requirements-test: Pin pytest to < 7.0.0
    
    pytest v7.0.0 removed _pytest.tmpdir.TempdirFactory, which is used
    by some of the pytest plugins we use.

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/281/ for more details.

vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/loader/svn/replay.py
701

just in case names clash

This revision is now accepted and ready to land.Feb 4 2022, 5:44 PM
swh/loader/svn/replay.py
701

right, thanks !

swh/loader/svn/replay.py
701

In fact temp_path is a path generated by the tempfile module so we are good here.

swh/loader/svn/replay.py
701

I'm aware, but there is still a very low likelyhood of it happening