Page MenuHomeSoftware Heritage

replay: Prevent erroneous removal of versioned path
ClosedPublic

Authored by anlambert on Feb 4 2022, 4:06 PM.

Details

Summary

When processing an external, the content of the target directory is
first removed before copying external dirs and files.

But that target directory might contain versioned paths so we must
ensure to restore them after removing the directory content.

Related to T611

Fixes some events in SWH-LOADER-SVN-53

Depends on D7087

Diff Detail

Repository
rDLDSVN Subversion (SVN) loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D7091 (id=25715)

Could not rebase; Attempt merge onto 09e83fd94c...

Updating 09e83fd..cb1b59c
Fast-forward
 requirements-test.txt               |   2 +-
 swh/loader/svn/loader.py            |   6 -
 swh/loader/svn/replay.py            | 265 ++++++++++++++++++++----------------
 swh/loader/svn/svn.py               |   9 +-
 swh/loader/svn/tests/test_loader.py |  53 ++++++++
 5 files changed, 210 insertions(+), 125 deletions(-)
Changes applied before test
commit cb1b59ca340132dd265784611ede974ca43adaf9
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 14:53:42 2022 +0100

    replay: Prevent erroneous removal of versioned path
    
    When processing an external, the content of the target directory is
    first removed before copying external dirs and files.
    
    But that target directory might contain versioned paths so we must
    ensure to restore them after removing the directory content.
    
    Related to T611

commit 651bb4e0b5c42ac67a8c29b2da9986261e9ff682
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 14:27:36 2022 +0100

    replay: Improve signature of DirEditor.open_file method
    
    Add path parameter for consistency with other methods.

commit d5bdea55bc527a94d9e735648a2b835ea2510fba
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 14:07:57 2022 +0100

    replay: Move external processing in a dedicated method
    
    That code grew a lot so better putting it in a dedicated method, plus
    it enables to remove one indentation level for better readability.

commit 369fbea00ea54b202a08db8541a7e6db42a0cc95
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 13:48:00 2022 +0100

    svn: Add debug log before executing checkout or export operation

commit 8bde0663cb4a72a80212e7bab21a962ddf510e25
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Feb 4 13:40:52 2022 +0100

    requirements-test: Pin pytest to < 7.0.0
    
    pytest v7.0.0 removed _pytest.tmpdir.TempdirFactory, which is used
    by some of the pytest plugins we use.

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/281/ for more details.

vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/loader/svn/replay.py
700

just in case names clash

This revision is now accepted and ready to land.Feb 4 2022, 5:44 PM
swh/loader/svn/replay.py
700

right, thanks !

swh/loader/svn/replay.py
700

In fact temp_path is a path generated by the tempfile module so we are good here.

swh/loader/svn/replay.py
700

I'm aware, but there is still a very low likelyhood of it happening