Page MenuHomeSoftware Heritage

ra: Fix edge case in svn:eol-style property handling
ClosedPublic

Authored by anlambert on Oct 29 2021, 8:27 PM.

Details

Summary

In a subversion repository, files contained in a directory can have the
svn:eol-style property set in order to define which line endings should
be used when exporting a revision.

Nevertheless, the directory and its content can be removed in another
revision and added again in another one without setting the svn:eol-style
property this time.

The subversion loader already handles that property to ensure reconstructed
filesystem for a revision through the svn_ra API will be identical to the one
obtained by exporting that revision. But it did not handle properly the case
when a directory and its content are removed recursively, resulting in eol-style
mapping for the concerned files not cleared. This was leading to invalid
reconstructed filesystem when files get added again without the property
set as end of lines are replaced while they should not.

So ensure to properly handle the recursive removal of a directory and its
content by clearing the eol-style mapping for removed files.

Related to T3695

Issue found by analyzing load failures reported in that sentry issue.

Diff Detail

Repository
rDLDSVN Subversion (SVN) loader
Branch
fix-eol-style-handling-edge-case
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24849
Build 38809: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 38808: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D6589 (id=23951)

Rebasing onto ed8c189b82...

Current branch diff-target is up to date.
Changes applied before test
commit d424bd121af6ba2da53f4665ea9291761bcb4798
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/143/ for more details.

Update: Add function add_commit to ease tests implementation and readability

Build has FAILED

Patch application report for D6589 (id=23969)

Rebasing onto ed8c189b82...

Current branch diff-target is up to date.
Changes applied before test
commit 646a63a741259e672d3a5e4001a13a5303a4747e
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

Link to build: https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/146/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/146/console

Build is green

Patch application report for D6589 (id=23973)

Rebasing onto ed8c189b82...

Current branch diff-target is up to date.
Changes applied before test
commit 3bde065e0b64b573d650f4c328750e50ba0daae2
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/150/ for more details.

This revision was not accepted when it landed; it landed in state Needs Review.Nov 3 2021, 3:38 PM
This revision was automatically updated to reflect the committed changes.