Differential D8879

svn: Also escape hash signs in URLs to avoid libsvn assertion
ClosedPublic
Actions

Authored by anlambert on Nov 23 2022, 5:06 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Commits

rDLDSVNbcd1e753c60b: svn: Also escape hash signs in URLs to avoid libsvn assertion

Summary

See test before fix:

17:04 $ pytest -sv swh/loader/svn/tests/test_loader.py::test_loader_with_special_chars_in_svn_url 
================================================================================================================================== test session starts ==================================================================================================================================
platform linux -- Python 3.9.2, pytest-7.1.3, pluggy-1.0.0 -- /home/anlambert/.virtualenvs/swh/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/anlambert/swh/swh-environment/swh-loader-svn/.hypothesis/examples')
rootdir: /home/anlambert/swh/swh-environment/swh-loader-svn, configfile: pytest.ini
plugins: redis-2.4.0, asyncio-0.19.0, subprocess-1.4.2, postgresql-3.1.3, forked-1.4.0, flask-1.2.0, anyio-3.6.1, xdist-2.5.0, cov-3.0.0, dash-2.6.1, requests-mock-1.9.3, django-test-migrations-1.2.0, django-4.5.2, mock-3.8.2, swh.core-2.16.1, swh.journal-1.1.0, hypothesis-6.56.2
asyncio: mode=strict
collected 2 items                                                                                                                                                                                                                                                                       

swh/loader/svn/tests/test_loader.py::test_loader_with_special_chars_in_svn_url[file with spaces.txt] PASSED
swh/loader/svn/tests/test_loader.py::test_loader_with_special_chars_in_svn_url[file#with#hash#signs.txt] python3: ../subversion/libsvn_subr/dirent_uri.c:1562: uri_skip_ancestor: Assertion `svn_uri_is_canonical(child_uri, NULL)' failed.
Fatal Python error: Aborted

Current thread 0x00007fb4a749f740 (most recent call first):
  File "/home/anlambert/swh/swh-environment/swh-loader-svn/swh/loader/svn/svn.py", line 324 in export
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/tenacity/__init__.py", line 407 in __call__
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/tenacity/__init__.py", line 324 in wrapped_f
  File "/home/anlambert/swh/swh-environment/swh-loader-svn/swh/loader/svn/tests/test_loader.py", line 2199 in test_loader_with_special_chars_in_svn_url
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/python.py", line 192 in pytest_pyfunc_call
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/python.py", line 1761 in runtest
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/runner.py", line 166 in pytest_runtest_call
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/runner.py", line 259 in <lambda>
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/runner.py", line 338 in from_call
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/runner.py", line 258 in call_runtest_hook
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/runner.py", line 219 in call_and_report
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/runner.py", line 130 in runtestprotocol
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/runner.py", line 111 in pytest_runtest_protocol
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/main.py", line 347 in pytest_runtestloop
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/main.py", line 322 in _main
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/main.py", line 268 in wrap_session
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/main.py", line 315 in pytest_cmdline_main
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/config/__init__.py", line 164 in main
  File "/home/anlambert/.virtualenvs/swh/lib/python3.9/site-packages/_pytest/config/__init__.py", line 187 in console_main
  File "/home/anlambert/.virtualenvs/swh/bin/pytest", line 8 in <module>
Aborted

Diff Detail

Repository

rDLDSVN Subversion (SVN) loader

Branch

svn-urls-escape-hash-signs

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 32943
Build 51633: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 51632: arc lint + arc unit

Event Timeline

anlambert created this revision.Nov 23 2022, 5:06 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptNov 23 2022, 5:06 PM

Build is green

Patch application report for D8879 (id=32010)

Rebasing onto a843858b0c...

Current branch diff-target is up to date.

Changes applied before test

commit bcd1e753c60b35d0af628b5e863ac453a26d9e12
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Nov 23 16:59:23 2022 +0100

    svn: Also escape hash signs in URLs to avoid libsvn assertion

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/356/ for more details.

Harbormaster completed remote builds in B32943: Diff 32010.Nov 23 2022, 5:09 PM

anlambert requested review of this revision.Nov 23 2022, 5:09 PM

Why not urlencode? Couldn't this be an issue with other characters?