Details
Details
- Reviewers
ardumont - Group Reviewers
Reviewers - Maniphest Tasks
- T611: support for external definitions in the svn/subversion loader
- Commits
- rDLDSVNd7a97a75a657: utils: Fix hidden path processing in parse_external_definition
Diff Detail
Diff Detail
- Repository
- rDLDSVN Subversion (SVN) loader
- Branch
- parse-external-hidden-file-fix
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 26699 Build 41752: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 41751: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D7129 (id=25848)
Could not rebase; Attempt merge onto d929eebd39...
Updating d929eeb..7b6c15c Fast-forward swh/loader/svn/replay.py | 79 ++++++++++++------ swh/loader/svn/tests/test_loader.py | 158 +++++++++++++++++++++++++++++++++++- swh/loader/svn/tests/test_utils.py | 6 ++ swh/loader/svn/utils.py | 3 +- 4 files changed, 218 insertions(+), 28 deletions(-)
Changes applied before test
commit 7b6c15c848aaf676a9b4e319ede6c092d4dfca6d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Wed Feb 9 12:10:10 2022 +0100
utils: Fix hidden path processing in parse_external_definition
str.lstrip takes a list of chars to left strip and do not left strip
a string so if a path starts with a '.' it was erroneously removed.
Related to T611
commit 23c92f8e2dfb7c41df9a9187ee12a23438e5b565
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Tue Feb 8 17:31:11 2022 +0100
replay: Fix external tuple use in process_external method
Some required changes were missed in 20c1445fb11e.
The external tuple created at the beginning of the process_external method
was missing a third boolean member.
The externals defintions in DirState was also missing that boolean member.
Test checking if an external was already exported was invalid.
Also alias the type of that tuple for better code readability.
Related to T611
commit c46a3b68579a089703ea324b55370abdca73c007
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Tue Feb 8 15:31:49 2022 +0100
replay: Prevent external path removal when defined on multiple dirs
The same absolute external path can be defined on multiple directories
in the repository.
To prevent removal of an external path while it is still defined in
another directory, we need to track the number of references of the
absolute external path by incrementing / decrementing a counter
when adding / removing an external.
Related to T611
commit 0ea0d1282177a5ca7878ea61c07ffeb0d4676ec1
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Mon Feb 7 15:20:17 2022 +0100
replay: Use remote repository base URL to export external located in it
Some externals might be located in the same repository we are
currently loading. In that case, replace the external base URL
which corresponds to the origin URL by the remote repository URL.
When we use SvnLoaderFromRemoteDump, that remote repository URL
corresponds to a local repository mounted from a dump file so we
can export the external path in a much faster way without any costly
network requests.
Related to T611See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/293/ for more details.
Comment Actions
Build is green
Patch application report for D7129 (id=25857)
Could not rebase; Attempt merge onto d929eebd39...
Updating d929eeb..d7a97a7 Fast-forward swh/loader/svn/replay.py | 83 +++++++++++++------ swh/loader/svn/tests/test_loader.py | 158 +++++++++++++++++++++++++++++++++++- swh/loader/svn/tests/test_utils.py | 6 ++ swh/loader/svn/utils.py | 3 +- 4 files changed, 222 insertions(+), 28 deletions(-)
Changes applied before test
commit d7a97a75a6575453f1469a13987ece8c685f08d1
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Wed Feb 9 12:10:10 2022 +0100
utils: Fix hidden path processing in parse_external_definition
str.lstrip takes a list of chars to left strip and do not left strip
a string so if a path starts with a '.' it was erroneously removed.
Related to T611
commit 46f25850ae8e23e004db988abd67ff4841ee0197
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Tue Feb 8 17:31:11 2022 +0100
replay: Fix external tuple use in process_external method
Some required changes were missed in 20c1445fb11e.
The external tuple created at the beginning of the process_external method
was missing a third boolean member.
The externals defintions in DirState was also missing that boolean member.
Test checking if an external was already exported was invalid.
Also alias the type of that tuple for better code readability.
Related to T611
commit 7220edbba0f43ab06dee016aa45fc9d3f0673953
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Tue Feb 8 15:31:49 2022 +0100
replay: Prevent external path removal when defined on multiple dirs
The same absolute external path can be defined on multiple directories
in the repository.
To prevent removal of an external path while it is still defined in
another directory, we need to track the number of references of the
absolute external path by incrementing / decrementing a counter
when adding / removing an external.
Related to T611
commit 0ea0d1282177a5ca7878ea61c07ffeb0d4676ec1
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Mon Feb 7 15:20:17 2022 +0100
replay: Use remote repository base URL to export external located in it
Some externals might be located in the same repository we are
currently loading. In that case, replace the external base URL
which corresponds to the origin URL by the remote repository URL.
When we use SvnLoaderFromRemoteDump, that remote repository URL
corresponds to a local repository mounted from a dump file so we
can export the external path in a much faster way without any costly
network requests.
Related to T611See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/296/ for more details.