Implies:
- [x] new docker image release including swh.lister v3.0
- [x] new docker image release including swh.loader.core v4.0
- [x] Register new task types to the staging scheduler [1]
- [x] Missing load-pubdev task type in scheduler -> reason: python3-swh.loader.core v4.0 debian build broken
- [x] Fix debian unstable build ^ as it's needed for the scheduler node to register all new task types [1]
- [x] Fix yet another debian build (buster) [2] -> False negative (dns issue during build)
- [x] Register load-pub task type (among other loader tasks) [1] (amended)
- [ ] Schedule one task with type 'list-pubdev' [3]
- [ ] archive-staging cluster: declare new loader and lister
- [ ] Check listing/loading happening
Plan:
- Use new elastic worker infra
[1]
```
swhscheduler@scheduler0:~$ swh scheduler --config-file /etc/softwareheritage/scheduler/backend.yml task-type register
...
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin loader.svn
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin loader.crates
INFO:swh.scheduler.cli.task_type:Create task type load-crates in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin lister.arch
INFO:swh.scheduler.cli.task_type:Create task type list-arch in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin lister.aur
INFO:swh.scheduler.cli.task_type:Create task type list-aur in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin lister.bower
INFO:swh.scheduler.cli.task_type:Create task type list-bower in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin lister.crates
INFO:swh.scheduler.cli.task_type:Create task type list-crates in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin lister.golang
INFO:swh.scheduler.cli.task_type:Create task type list-golang in scheduler
INFO:swh.scheduler.cli.task_type:Create task type list-golang-incremental in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin lister.pubdev
INFO:swh.scheduler.cli.task_type:Create task type list-pubdev in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin lister.gogs
INFO:swh.scheduler.cli.task_type:Create task type list-gogs-full in scheduler
...
# ^^^^ missing couple of loading tasks -> probably an issue with the debian package build
swhscheduler@scheduler0:~$ dpkg -l python3-swh.loader.core | grep ii
ii python3-swh.loader.core 4.0.0-1~swh2~bpo10+1 all Software Heritage Loader Core
swhscheduler@scheduler0:~$ swh scheduler --config-file /etc/softwareheritage/scheduler/backend.yml task-type register | grep pubdev
...
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin loader.arch
INFO:swh.scheduler.cli.task_type:Create task type load-arch in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin loader.aur
INFO:swh.scheduler.cli.task_type:Create task type load-aur in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin loader.golang
INFO:swh.scheduler.cli.task_type:Create task type load-golang in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin loader.pubdev
INFO:swh.scheduler.cli.task_type:Create task type load-pubdev in scheduler
...
```
[1] https://jenkins.softwareheritage.org/view/swh-debian%20(draft)/job/debian/job/packages/job/DLDBASE/job/gbp-buildpackage/286/console
[2] https://jenkins.softwareheritage.org/view/swh-debian%20(draft)/job/debian/job/packages/job/DLDBASE/job/gbp-buildpackage/288/console
[3]
```
swhscheduler@scheduler0:~$ swh scheduler --url http://scheduler0.internal.staging.swh.network:5008/ task add list-pubdev
Created 1 tasks
Task 33419442
Next run: today (2022-09-09T10:30:00.252081+00:00)
Interval: 1 day, 0:00:00
Type: list-pubdev
Policy: recurring
Args:
Keyword args:
```