Page MenuHomeSoftware Heritage

ra: Handle unsetting svn:special property on svn links
ClosedPublic

Authored by anlambert on Nov 2 2021, 6:15 PM.

Details

Summary

Subversion links have the svn:special property set when added in
a repository.

Nevertheless, it exist cases when such a property gets unset on
svn links in another revision. In that case, the svn links will
no longer be extracted as symbolic links during a checkout or
export operation but rather to regular files with the following
content: link <path_to_target>.

So when the svn:special property gets unset on a svn link, ensure
to remove the symbolic link previously created by the ra module
and replace it by a regular file containing the svn link info.
Otherwise the reconstructed filesystem for a revision will differ
from the one resulting from an export operation on that revision.

Related to T3695

Depends on D6595

Diff Detail

Repository
rDLDSVN Subversion (SVN) loader
Branch
handle-unset-of-svn-special-property
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24851
Build 38813: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 38812: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D6596 (id=23963)

Could not rebase; Attempt merge onto ed8c189b82...

Updating ed8c189..fffd187
Fast-forward
 swh/loader/svn/ra.py                |  18 ++-
 swh/loader/svn/tests/test_loader.py | 231 +++++++++++++++++++++++++++++++++++-
 2 files changed, 243 insertions(+), 6 deletions(-)
Changes applied before test
commit fffd1871a20c88405b23b023bd1423d35c138c0d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 18:00:10 2021 +0100

    ra: Handle unsetting svn:special property on svn links
    
    Subversion links have the svn:special property set when added in
    a repository.
    
    Nevertheless, it exist cases when such a property gets unset on
    svn links in another revision. In that case, the svn links will
    no longer be extracted as symbolic links during a checkout or
    export operation but rather to regular files with the following
    content: "link <path_to_target>".
    
    So when the svn:special property gets unset on a svn link, ensure
    to remove the symbolic link previously created by the ra module
    and replace it by a regular file containing the svn link info.
    Otherwise the reconstructed filesystem for a revision will differ
    from the one resulting from an export operation on that revision.
    
    Related to T3695

commit 48e7e17f74b8086606625f2842ff9fc9755a9106
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 17:33:04 2021 +0100

    ra: Do not process end of lines for svn links
    
    In a subversion repository, files with the svn:special property set
    correspond to svn links and will be exported as symbolic links when
    doing a checkout or export operation.
    
    A subversion link is stored as a special file in the repository with
    the following content: "link <path_to_target>".
    
    So when the svn:eol-style property is also set on a svn link, we must
    ensure that its content will not be overriden by the content of the
    link target with processed end of lines. Otherwise the reconstructed
    filesystem for a revision will differ from the one resulting from
    an export operation on that revision.
    
    Related to T3695

commit d424bd121af6ba2da53f4665ea9291761bcb4798
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/145/ for more details.

Update: Use function add_commit to improve test readability

Build has FAILED

Patch application report for D6596 (id=23971)

Could not rebase; Attempt merge onto ed8c189b82...

Updating ed8c189..f9e674b
Fast-forward
 swh/loader/svn/ra.py                |  18 ++-
 swh/loader/svn/tests/test_loader.py | 291 +++++++++++++++++++++++++++++++++++-
 2 files changed, 303 insertions(+), 6 deletions(-)
Changes applied before test
commit f9e674b09216d6cffb572af9516255b2c12e795b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 18:00:10 2021 +0100

    ra: Handle unsetting svn:special property on svn links
    
    Subversion links have the svn:special property set when added in
    a repository.
    
    Nevertheless, it exist cases when such a property gets unset on
    svn links in another revision. In that case, the svn links will
    no longer be extracted as symbolic links during a checkout or
    export operation but rather to regular files with the following
    content: "link <path_to_target>".
    
    So when the svn:special property gets unset on a svn link, ensure
    to remove the symbolic link previously created by the ra module
    and replace it by a regular file containing the svn link info.
    Otherwise the reconstructed filesystem for a revision will differ
    from the one resulting from an export operation on that revision.
    
    Related to T3695

commit 102e8df3fe2de8087eefa7470d6ec8105b203a2b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 17:33:04 2021 +0100

    ra: Do not process end of lines for svn links
    
    In a subversion repository, files with the svn:special property set
    correspond to svn links and will be exported as symbolic links when
    doing a checkout or export operation.
    
    A subversion link is stored as a special file in the repository with
    the following content: "link <path_to_target>".
    
    So when the svn:eol-style property is also set on a svn link, we must
    ensure that its content will not be overriden by the content of the
    link target with processed end of lines. Otherwise the reconstructed
    filesystem for a revision will differ from the one resulting from
    an export operation on that revision.
    
    Related to T3695

commit 646a63a741259e672d3a5e4001a13a5303a4747e
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

Link to build: https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/148/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/148/console

Build has FAILED

Patch application report for D6596 (id=23972)

Could not rebase; Attempt merge onto ed8c189b82...

Updating ed8c189..e7070aa
Fast-forward
 swh/loader/svn/ra.py                |  20 ++-
 swh/loader/svn/tests/test_loader.py | 291 +++++++++++++++++++++++++++++++++++-
 2 files changed, 305 insertions(+), 6 deletions(-)
Changes applied before test
commit e7070aa1b71f01acb1e9a7e0a380837e86969124
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 18:00:10 2021 +0100

    ra: Handle unsetting svn:special property on svn links
    
    Subversion links have the svn:special property set when added in
    a repository.
    
    Nevertheless, it exist cases when such a property gets unset on
    svn links in another revision. In that case, the svn links will
    no longer be extracted as symbolic links during a checkout or
    export operation but rather to regular files with the following
    content: "link <path_to_target>".
    
    So when the svn:special property gets unset on a svn link, ensure
    to remove the symbolic link previously created by the ra module
    and replace it by a regular file containing the svn link info.
    Otherwise the reconstructed filesystem for a revision will differ
    from the one resulting from an export operation on that revision.
    
    Related to T3695

commit 102e8df3fe2de8087eefa7470d6ec8105b203a2b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 17:33:04 2021 +0100

    ra: Do not process end of lines for svn links
    
    In a subversion repository, files with the svn:special property set
    correspond to svn links and will be exported as symbolic links when
    doing a checkout or export operation.
    
    A subversion link is stored as a special file in the repository with
    the following content: "link <path_to_target>".
    
    So when the svn:eol-style property is also set on a svn link, we must
    ensure that its content will not be overriden by the content of the
    link target with processed end of lines. Otherwise the reconstructed
    filesystem for a revision will differ from the one resulting from
    an export operation on that revision.
    
    Related to T3695

commit 646a63a741259e672d3a5e4001a13a5303a4747e
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

Link to build: https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/149/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/149/console

Build is green

Patch application report for D6596 (id=23975)

Could not rebase; Attempt merge onto ed8c189b82...

Updating ed8c189..b326099
Fast-forward
 requirements.txt                    |   3 +-
 swh/loader/svn/ra.py                |  20 ++-
 swh/loader/svn/tests/test_loader.py | 292 +++++++++++++++++++++++++++++++++++-
 3 files changed, 308 insertions(+), 7 deletions(-)
Changes applied before test
commit b326099c734844126cf834cc8ffa44d5bec98108
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 18:00:10 2021 +0100

    ra: Handle unsetting svn:special property on svn links
    
    Subversion links have the svn:special property set when added in
    a repository.
    
    Nevertheless, it exist cases when such a property gets unset on
    svn links in another revision. In that case, the svn links will
    no longer be extracted as symbolic links during a checkout or
    export operation but rather to regular files with the following
    content: "link <path_to_target>".
    
    So when the svn:special property gets unset on a svn link, ensure
    to remove the symbolic link previously created by the ra module
    and replace it by a regular file containing the svn link info.
    Otherwise the reconstructed filesystem for a revision will differ
    from the one resulting from an export operation on that revision.
    
    Related to T3695

commit 4e50fd3cd2b302cba46bd7319a780e06475644ce
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Nov 2 17:33:04 2021 +0100

    ra: Do not process end of lines for svn links
    
    In a subversion repository, files with the svn:special property set
    correspond to svn links and will be exported as symbolic links when
    doing a checkout or export operation.
    
    A subversion link is stored as a special file in the repository with
    the following content: "link <path_to_target>".
    
    So when the svn:eol-style property is also set on a svn link, we must
    ensure that its content will not be overriden by the content of the
    link target with processed end of lines. Otherwise the reconstructed
    filesystem for a revision will differ from the one resulting from
    an export operation on that revision.
    
    Related to T3695

commit 3bde065e0b64b573d650f4c328750e50ba0daae2
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 29 20:09:56 2021 +0200

    ra: Fix edge case in svn:eol-style property handling
    
    In a subversion repository, files contained in a directory can have the
    svn:eol-style property set in order to define which line endings should
    be used when exporting a revision.
    
    Nevertheless, the directory and its content can be removed in another
    revision and added again in another one without setting the svn:eol-style
    property this time.
    
    The subversion loader already handles that property to ensure reconstructed
    filesystem for a revision through the svn_ra API will be identical to the one
    obtained by exporting that revision. But it did not handle properly the case
    when a directory and its content are removed recursively, resulting in eol-style
    mapping for the concerned files not cleared. This was leading to invalid
    reconstructed filesystem when files get added again without the property
    set as end of lines are replaced while they should not.
    
    So ensure to properly handle the recursive removal of a directory and its
    content by clearing the eol-style mapping for removed files.
    
    Related to T3695

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/152/ for more details.

This revision is now accepted and ready to land.Nov 3 2021, 3:24 PM