Page MenuHomeSoftware Heritage

maven, packagist: Validate origin URLs
ClosedPublic

Authored by vlorentz on Nov 4 2022, 1:48 PM.

Details

Reviewers
anlambert
Group Reviewers
Reviewers
Maniphest Tasks
Unknown Object (Maniphest Task)
Commits
rDLS8ea4200909ad: Validate origin URLs before sending to the scheduler
Summary

Closes T4670

Diff Detail

Repository
rDLS Listers
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz added a task: Unknown Object (Maniphest Task).

Build is green

Patch application report for D8812 (id=31752)

Rebasing onto 60707a45dd...

Current branch diff-target is up to date.
Changes applied before test
commit 15101487ad9676f3b0df2a0685bc0782342cb212
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 4 13:48:14 2022 +0100

    maven, packagist: Validate origin URLs

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/842/ for more details.

Would not it be more generic to validate the listed origin URLs at the scheduler level, more precisely in the record_listed_origins method ?
This way, all listers will benefit from URLs validation without touching their code.
Proceeding like this, we could even compute the list of rejected URLs in the send_origins method of base lister class (to log a warning for instance).

I do not think we have listers producing listed origins with url field set with something that does not lookalike an URL so that thought.

validate in the base class

Would not it be more generic to validate the listed origin URLs at the scheduler level, more precisely in the record_listed_origins method ?

In the linked task, olasd suggested we do it in both.

This way, all listers will benefit from URLs validation without touching their code.
Proceeding like this, we could even compute the list of rejected URLs in the send_origins method of base lister class (to log a warning for instance).

Oh indeed, very good point.

Build has FAILED

Patch application report for D8812 (id=31756)

Rebasing onto 60707a45dd...

Current branch diff-target is up to date.
Changes applied before test
commit fc353ca6ea01a33cc0e96bc55ea9306addec056a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 4 13:48:14 2022 +0100

    Validate origin URLs before sending to the scheduler

Link to build: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/844/
See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/844/console

Build is green

Patch application report for D8812 (id=31758)

Rebasing onto 60707a45dd...

Current branch diff-target is up to date.
Changes applied before test
commit 8ea4200909ad096ca92de20bacb8c9df655ba90f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 4 13:48:14 2022 +0100

    Validate origin URLs before sending to the scheduler

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/846/ for more details.

This revision is now accepted and ready to land.Nov 4 2022, 4:16 PM