Page MenuHomeSoftware Heritage

ra: Do not process end of lines for svn links
ClosedPublic

Authored by anlambert on Nov 2 2021, 6:14 PM.

Details

Summary

In a subversion repository, files with the svn:special property set
correspond to svn links and will be exported as symbolic links when
doing a checkout or export operation.

A subversion link is stored as a special file in the repository with
the following content: link <path_to_target>.

So when the svn:eol-style property is also set on a svn link, we must
ensure that its content will not be overriden by the content of the
link target with processed end of lines. Otherwise the reconstructed
filesystem for a revision will differ from the one resulting from
an export operation on that revision.

Related to T3695

Depends on D6589

Diff Detail

Repository
rDLDSVN Subversion (SVN) loader
Branch
fix-eol-style-handling-on-svn-link
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24850
Build 38811: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 38810: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D6595 (id=23962)

Could not rebase; Attempt merge onto ed8c189b82...

Updating ed8c189..48e7e17
Fast-forward
 swh/loader/svn/ra.py                |  14 +++-
 swh/loader/svn/tests/test_loader.py | 148 +++++++++++++++++++++++++++++++++++-
 2 files changed, 157 insertions(+), 5 deletions(-)
Changes applied before test
commit 48e7e17f74b8086606625f2842ff9fc9755a9106
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 17:33:04 2021 +0100

    ra: Do not process end of lines for svn links
    
    In a subversion repository, files with the svn:special property set
    correspond to svn links and will be exported as symbolic links when
    doing a checkout or export operation.
    
    A subversion link is stored as a special file in the repository with
    the following content: "link <path_to_target>".
    
    So when the svn:eol-style property is also set on a svn link, we must
    ensure that its content will not be overriden by the content of the
    link target with processed end of lines. Otherwise the reconstructed
    filesystem for a revision will differ from the one resulting from
    an export operation on that revision.
    
    Related to T3695

commit d424bd121af6ba2da53f4665ea9291761bcb4798
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/144/ for more details.

Update: Use function add_commit to improve test readability

Build has FAILED

Patch application report for D6595 (id=23970)

Could not rebase; Attempt merge onto ed8c189b82...

Updating ed8c189..102e8df
Fast-forward
 swh/loader/svn/ra.py                |  14 ++-
 swh/loader/svn/tests/test_loader.py | 196 +++++++++++++++++++++++++++++++++++-
 2 files changed, 205 insertions(+), 5 deletions(-)
Changes applied before test
commit 102e8df3fe2de8087eefa7470d6ec8105b203a2b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 17:33:04 2021 +0100

    ra: Do not process end of lines for svn links
    
    In a subversion repository, files with the svn:special property set
    correspond to svn links and will be exported as symbolic links when
    doing a checkout or export operation.
    
    A subversion link is stored as a special file in the repository with
    the following content: "link <path_to_target>".
    
    So when the svn:eol-style property is also set on a svn link, we must
    ensure that its content will not be overriden by the content of the
    link target with processed end of lines. Otherwise the reconstructed
    filesystem for a revision will differ from the one resulting from
    an export operation on that revision.
    
    Related to T3695

commit 646a63a741259e672d3a5e4001a13a5303a4747e
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

Link to build: https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/147/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/147/console

Build is green

Patch application report for D6595 (id=23974)

Could not rebase; Attempt merge onto ed8c189b82...

Updating ed8c189..4e50fd3
Fast-forward
 requirements.txt                    |   3 +-
 swh/loader/svn/ra.py                |  14 ++-
 swh/loader/svn/tests/test_loader.py | 197 +++++++++++++++++++++++++++++++++++-
 3 files changed, 208 insertions(+), 6 deletions(-)
Changes applied before test
commit 4e50fd3cd2b302cba46bd7319a780e06475644ce
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 17:33:04 2021 +0100

    ra: Do not process end of lines for svn links
    
    In a subversion repository, files with the svn:special property set
    correspond to svn links and will be exported as symbolic links when
    doing a checkout or export operation.
    
    A subversion link is stored as a special file in the repository with
    the following content: "link <path_to_target>".
    
    So when the svn:eol-style property is also set on a svn link, we must
    ensure that its content will not be overriden by the content of the
    link target with processed end of lines. Otherwise the reconstructed
    filesystem for a revision will differ from the one resulting from
    an export operation on that revision.
    
    Related to T3695

commit 3bde065e0b64b573d650f4c328750e50ba0daae2
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/151/ for more details.

This revision is now accepted and ready to land.Nov 3 2021, 3:23 PM