Differential D8813

nixguix: Reject Git SSH URLs and pseudo-URLs
ClosedPublic
Actions

Authored by vlorentz on Nov 4 2022, 1:51 PM.

Details

Reviewers

anlambert

Group Reviewers

Reviewers

Maniphest Tasks

Unknown Object (Maniphest Task)

Commits

rDLSe8699422d757: nixguix: Reject Git SSH URLs and pseudo-URLs

Summary

For consistency with Maven and Packagist listers

Depends on D8812.

Tangentially related to T4670.

Diff Detail

Repository

rDLS Listers

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 32710
Build 51248: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 51247: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	419 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.opam.tests.test_lister::test_opam_binary datadir = '/var/lib/jenkins/workspace/DLS/tests-on-diff@2/.tox/py3/lib/python3.7/site-packages/swh/lister/opam/tests/data' swh_scheduler = <swh.scheduler.backend.SchedulerBackend object at 0x7f657e7649b0> tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_opam_binary0')
	140 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.opam.tests.test_lister::test_opam_multi_instance datadir = '/var/lib/jenkins/workspace/DLS/tests-on-diff@2/.tox/py3/lib/python3.7/site-packages/swh/lister/opam/tests/data' swh_scheduler = <swh.scheduler.backend.SchedulerBackend object at 0x7f657e1d9e10> tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_opam_multi_instance0')
	2 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.arch.lister::swh.lister.arch.lister.size_to_bytes
	2,176 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.arch.tests.test_lister::test_arch_lister
	504 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.arch.tests.test_tasks::test_arch_lister
		View Full Test Results (2 Failed · 419 Passed)

Event Timeline

vlorentz created this revision.Nov 4 2022, 1:51 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptNov 4 2022, 1:51 PM

Build is green

Patch application report for D8813 (id=31753)

Could not rebase; Attempt merge onto 60707a45dd...

Updating 60707a4..34bad1c
Fast-forward
 swh/lister/maven/lister.py                         |   3 +-
 .../maven/tests/data/sprova4j-0.1.0.invalidurl.pom |  30 ++++
 swh/lister/maven/tests/test_lister.py              |  47 +++++++
 swh/lister/nixguix/lister.py                       |   5 +-
 swh/lister/nixguix/tests/data/sources-failure.json |  10 ++
 swh/lister/nixguix/tests/test_lister.py            |  13 +-
 swh/lister/packagist/lister.py                     |   4 +
 .../packagist/tests/data/payrix_payrix-php.json    | 151 +++++++++++++++++++++
 .../packagist/tests/data/with_invalid_url.json     |  24 ++++
 swh/lister/packagist/tests/test_lister.py          |  10 +-
 swh/lister/utils.py                                |  50 ++++++-
 11 files changed, 336 insertions(+), 11 deletions(-)
 create mode 100644 swh/lister/maven/tests/data/sprova4j-0.1.0.invalidurl.pom
 create mode 100644 swh/lister/packagist/tests/data/payrix_payrix-php.json
 create mode 100644 swh/lister/packagist/tests/data/with_invalid_url.json

Changes applied before test

commit 34bad1c39d1cec3769a5b43e7b08f5fc40e50c8a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 4 13:50:25 2022 +0100

    nixguix: Reject Git SSH URLs and pseudo-URLs
    
    For consistency with Maven and Packagist listers

commit 15101487ad9676f3b0df2a0685bc0782342cb212
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 4 13:48:14 2022 +0100

    maven, packagist: Validate origin URLs

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/843/ for more details.

Harbormaster completed remote builds in B32707: Diff 31753.Nov 4 2022, 1:58 PM

vlorentz requested review of this revision.Nov 4 2022, 1:58 PM

rebase

Build has FAILED

Patch application report for D8813 (id=31757)

Could not rebase; Attempt merge onto 60707a45dd...

Updating 60707a4..545da61
Fast-forward
 .../maven/tests/data/sprova4j-0.1.0.invalidurl.pom |  30 ++++
 swh/lister/maven/tests/test_lister.py              |  47 +++++++
 swh/lister/nixguix/lister.py                       |   2 +-
 swh/lister/nixguix/tests/data/sources-failure.json |  10 ++
 swh/lister/nixguix/tests/test_lister.py            |  13 +-
 .../packagist/tests/data/payrix_payrix-php.json    | 151 +++++++++++++++++++++
 .../packagist/tests/data/with_invalid_url.json     |  24 ++++
 swh/lister/packagist/tests/test_lister.py          |  10 +-
 swh/lister/pattern.py                              |  11 +-
 swh/lister/utils.py                                |  50 ++++++-
 10 files changed, 337 insertions(+), 11 deletions(-)
 create mode 100644 swh/lister/maven/tests/data/sprova4j-0.1.0.invalidurl.pom
 create mode 100644 swh/lister/packagist/tests/data/payrix_payrix-php.json
 create mode 100644 swh/lister/packagist/tests/data/with_invalid_url.json

Changes applied before test

commit 545da61c6b7ea71c33b1aecf0a9071271d559a24
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 4 13:50:25 2022 +0100

    nixguix: Reject Git SSH URLs and pseudo-URLs
    
    For consistency with Maven and Packagist listers

commit fc353ca6ea01a33cc0e96bc55ea9306addec056a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 4 13:48:14 2022 +0100

    Validate origin URLs before sending to the scheduler

Link to build: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/845/
See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/845/console

Harbormaster failed remote builds in B32710: Diff 31757!Nov 4 2022, 3:20 PM

rebase

Build is green

Patch application report for D8813 (id=31759)

Could not rebase; Attempt merge onto 60707a45dd...

Updating 60707a4..e869942
Fast-forward
 .../maven/tests/data/sprova4j-0.1.0.invalidurl.pom |  30 ++++
 swh/lister/maven/tests/test_lister.py              |  47 +++++++
 swh/lister/nixguix/lister.py                       |   2 +-
 swh/lister/nixguix/tests/data/sources-failure.json |  10 ++
 swh/lister/nixguix/tests/test_lister.py            |  13 +-
 swh/lister/opam/tests/test_lister.py               |  28 +++-
 .../packagist/tests/data/payrix_payrix-php.json    | 151 +++++++++++++++++++++
 .../packagist/tests/data/with_invalid_url.json     |  24 ++++
 swh/lister/packagist/tests/test_lister.py          |  10 +-
 swh/lister/pattern.py                              |  11 +-
 swh/lister/utils.py                                |  50 ++++++-
 11 files changed, 360 insertions(+), 16 deletions(-)
 create mode 100644 swh/lister/maven/tests/data/sprova4j-0.1.0.invalidurl.pom
 create mode 100644 swh/lister/packagist/tests/data/payrix_payrix-php.json
 create mode 100644 swh/lister/packagist/tests/data/with_invalid_url.json

Changes applied before test

commit e8699422d757eb968779a9e44fc5017fb5a6dd97
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 4 13:50:25 2022 +0100

    nixguix: Reject Git SSH URLs and pseudo-URLs
    
    For consistency with Maven and Packagist listers

commit 8ea4200909ad096ca92de20bacb8c9df655ba90f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 4 13:48:14 2022 +0100

    Validate origin URLs before sending to the scheduler

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/847/ for more details.

Harbormaster completed remote builds in B32712: Diff 31759.Nov 4 2022, 4:05 PM

anlambert accepted this revision.Nov 4 2022, 4:17 PM

This revision is now accepted and ready to land.Nov 4 2022, 4:17 PM

Closed by commit rDLSe8699422d757: nixguix: Reject Git SSH URLs and pseudo-URLs (authored by vlorentz). · Explain WhyNov 7 2022, 10:42 AM

This revision was automatically updated to reflect the committed changes.

vlorentz added a commit: rDLSe8699422d757: nixguix: Reject Git SSH URLs and pseudo-URLs.