This will allow loaders to use the right API credentials to fetch
extrinsic metadata for the origin from the forge.
Details
Details
- Reviewers
ardumont - Group Reviewers
Reviewers - Maniphest Tasks
- T4187: Pass forge type to loaders
- Commits
- rDSCH0365b853fe05: Add a 'lister_instance_name' argument to all tasks created from ListedOrigin
Diff Detail
Diff Detail
- Repository
- rDSCH Scheduling utilities
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 28772 Build 44968: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 44967: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D7628 (id=27625)
Could not rebase; Attempt merge onto 9483493ff5...
Updating 9483493..5054c62 Fast-forward swh/scheduler/backend.py | 16 +++++ swh/scheduler/celery_backend/recurrent_visits.py | 5 +- swh/scheduler/cli/origin.py | 9 ++- swh/scheduler/interface.py | 4 ++ swh/scheduler/simulator/__init__.py | 8 ++- swh/scheduler/tests/test_api_client.py | 1 + swh/scheduler/tests/test_scheduler.py | 21 ++++++ swh/scheduler/tests/test_utils.py | 88 ++++++++++++++++++++++-- swh/scheduler/utils.py | 35 ++++++++-- 9 files changed, 166 insertions(+), 21 deletions(-)
Changes applied before test
commit 5054c62a3c88d55850076be0e1992d8bc6d66dfe
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Apr 21 18:40:55 2022 +0200
Add a 'lister_name' argument to all tasks created from ListedOrigin
This will allow loaders to use the right API credentials to fetch
extrinsic metadata for the origin from the forge.
commit 8eae66baca3d79fa4397764fdec07f0025f31a38
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Apr 21 12:22:03 2022 +0200
Add a 'lister_name' argument to all tasks created from ListedOrigin
This will allow loaders to guess the forge type, and use the right
API to fetch extrinsic metadata for the origin from the forge.See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/537/ for more details.
Comment Actions
Build is green
Patch application report for D7628 (id=27626)
Could not rebase; Attempt merge onto 9483493ff5...
Updating 9483493..c3dc43e Fast-forward swh/scheduler/backend.py | 16 +++++ swh/scheduler/celery_backend/recurrent_visits.py | 5 +- swh/scheduler/cli/origin.py | 9 ++- swh/scheduler/interface.py | 4 ++ swh/scheduler/simulator/__init__.py | 8 ++- swh/scheduler/tests/test_api_client.py | 1 + swh/scheduler/tests/test_scheduler.py | 21 ++++++ swh/scheduler/tests/test_utils.py | 88 ++++++++++++++++++++++-- swh/scheduler/utils.py | 35 ++++++++-- 9 files changed, 166 insertions(+), 21 deletions(-)
Changes applied before test
commit c3dc43ec2fed001694e721e3a540dc39e305709a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Apr 21 18:40:55 2022 +0200
Add a 'lister_instance_name' argument to all tasks created from ListedOrigin
This will allow loaders to use the right API credentials to fetch
extrinsic metadata for the origin from the forge.
commit 8eae66baca3d79fa4397764fdec07f0025f31a38
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Apr 21 12:22:03 2022 +0200
Add a 'lister_name' argument to all tasks created from ListedOrigin
This will allow loaders to guess the forge type, and use the right
API to fetch extrinsic metadata for the origin from the forge.See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/538/ for more details.
Comment Actions
lgtm
Don't we need to have some migration scripts for existing data in scheduler dbs (staging, prod)?
(to match this)
Comment Actions
this does not change anything to the DB, it just adds new args to celery tasks.
new loaders will simply ignore fetchers if these args are not passed
Comment Actions
this does not change anything to the DB, it just adds new args to celery tasks.
new loaders will simply ignore fetchers if these args are not passed
Right, i was on an old scheme of ours regarding those tasks (the ones serialized in db...).
Comment Actions
Build is green
Patch application report for D7628 (id=27715)
Rebasing onto 0365b853fe...
First, rewinding head to replay your work on top of it... Fast-forwarded diff-target to base-revision-542-D7628.
Changes applied before test
See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/542/ for more details.