Page MenuHomeSoftware Heritage

replay: Handle recursive external definition
ClosedPublic

Authored by anlambert on Jan 24 2022, 3:38 PM.

Details

Summary

It is possible that an external definition lead to a recursive subversion
export operation (https://issues.apache.org/jira/browse/SVN-1703).

As that recursive export will crash the loader and make the loading failed,
the only solution for that edge case is to stop processing externals while
a recursive external is detected in a revision.

We will then ignore externals when exporting a revision to check for divergence
with the recontructed filesystem.

Related to T3870#77491

Diff Detail

Repository
rDLDSVN Subversion (SVN) loader
Branch
handle-recursive-externals
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 26311
Build 41140: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 41139: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D7027 (id=25464)

Rebasing onto 3e4d5d7cc9...

Current branch diff-target is up to date.
Changes applied before test
commit bb606ed569a3f07d792f0550625560dcb47a6cd2
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Jan 24 15:32:02 2022 +0100

    replay: Handle recursive external definition
    
    It is possible that an external definition lead to a recursive subversion
    export operation (https://issues.apache.org/jira/browse/SVN-1703).
    
    As that recursive export will crash the loader and make the loading failed,
    the only solution for that edge case is to stop processing externals while
    a recursive external is detected in a revision.
    
    We will then ignore externals when exporting a revision to check for divergence
    with the recontructed filesystem.
    
    Related to T3870

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/267/ for more details.

Update: Fix recursive external detection on incremental loading.

Build is green

Patch application report for D7027 (id=25471)

Rebasing onto 3e4d5d7cc9...

Current branch diff-target is up to date.
Changes applied before test
commit 51caee0663dfcb8645a6c440084a69225dc0c4a0
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Jan 24 15:32:02 2022 +0100

    replay: Handle recursive external definition
    
    It is possible that an external definition lead to a recursive subversion
    export operation (https://issues.apache.org/jira/browse/SVN-1703).
    
    As that recursive export will crash the loader and make the loading failed,
    the only solution for that edge case is to stop processing externals while
    a recursive external is detected in a revision.
    
    We will then ignore externals when exporting a revision to check for divergence
    with the recontructed filesystem.
    
    Related to T3870

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/268/ for more details.

ardumont added a subscriber: ardumont.

ok, one suggestion inline

swh/loader/svn/replay.py
695–698

I'm not entirely sure i get all of that sentence. Is that better ^?

This revision is now accepted and ready to land.Jan 24 2022, 5:44 PM

Slightly rephrase comment

swh/loader/svn/replay.py
695–698

I'm not entirely sure i get all of that sentence. Is that better ^?

695–698

I have updated the comment and removed the indeed word.

Build is green

Patch application report for D7027 (id=25476)

Rebasing onto 3e4d5d7cc9...

Current branch diff-target is up to date.
Changes applied before test
commit afdee6dbee9be55f2eaf192dfb23733952977a2e
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Jan 24 15:32:02 2022 +0100

    replay: Handle recursive external definition
    
    It is possible that an external definition lead to a recursive subversion
    export operation (https://issues.apache.org/jira/browse/SVN-1703).
    
    As that recursive export will crash the loader and make the loading failed,
    the only solution for that edge case is to stop processing externals while
    a recursive external is detected in a revision.
    
    We will then ignore externals when exporting a revision to check for divergence
    with the recontructed filesystem.
    
    Related to T3870

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/269/ for more details.