Plan:
- [x] lister v0.15 (includes opam lister)
- [x] loader.core: Tag v0.23 which includes the opam loader
- [x] Register opam listing and loading task in the scheduler [1]
- [x] D6061: Add opam lister task to the worker configuration
- [x] Upgrade loader.core to staging workers and restart swh-worker@lister service
- [x] Trigger first listing [2]
- [x] Monitoring
- [x] D6062: Puppetize new swh-worker@loader_opam
- [x] Deploy new loader
- [ ] Start loader service
[1]
```
swhscheduler@scheduler0:~$ dpkg -l python3-swh.loader.core
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=======================-=====================-============-=================================
ii python3-swh.loader.core 0.23.1-1~swh1~bpo10+1 all Software Heritage Loader Core
swhscheduler@scheduler0:~$ swh scheduler --config-file /etc/softwareheritage/scheduler/backend.yml task-type register
...
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin lister.opam
INFO:swh.scheduler.cli.task_type:Create task type list-opam in scheduler
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin loader.opam
INFO:swh.scheduler.cli.task_type:Create task type load-opam in scheduler
...
$ psql service=staging-swh-scheduler
16:49:33 swh-scheduler@db1:5432=> select * from task_type where type like '%opam%';
+-----------+-----------------------------------+----------------------------------------+------------------+--------------+--------------+----------------+------------------+-------------+-------------+
| type | description | backend_name | default_interval | min_interval | max_interval | backoff_factor | max_queue_length | num_retries | retry_delay |
+-----------+-----------------------------------+----------------------------------------+------------------+--------------+--------------+----------------+------------------+-------------+-------------+
| list-opam | Lister task for the Opam registry | swh.lister.opam.tasks.OpamListerTask | 1 day | 1 day | 1 day | 1 | (null) | (null) | (null) |
| load-opam | Load Opam's artifacts | swh.loader.package.opam.tasks.LoadOpam | 1 day | 1 day | 1 day | 1 | (null) | (null) | (null) |
+-----------+-----------------------------------+----------------------------------------+------------------+--------------+--------------+----------------+------------------+-------------+-------------+
(2 rows)
Time: 144.033 ms
```
[2]
```
swhscheduler@scheduler0:~$ /usr/bin/swh scheduler --config-file scheduler.yml task add list-opam url=https://opam.ocaml.org instance=opam.ocaml.org
Created 1 tasks
Task 24858921
Next run: today (2021-08-05T15:20:39.703665+00:00)
Interval: 1 day, 0:00:00
Type: list-opam
Policy: recurring
Args:
Keyword args:
instance: 'opam.ocaml.org'
url: 'https://opam.ocaml.org'
```