plan:
- [x] Declare new lister task in scheduler
- [x] R260:e3aad: Declare new lister in swh-charts
- [x] Add new lister task in scheduler [1]
- [ ] Checks
- [x] Listing ok [2] [2']
- [x] Trigger specific origins for scheduling [3]
- [ ] ingestion ok on the few origins sent?
[1]
```
swhscheduler@scheduler0:~$ swh scheduler --url http://scheduler0.internal.staging.swh.network:5008/ task add list-bower
Created 1 tasks
Task 33419582
Next run: today (2022-09-26T11:44:22.561371+00:00)
Interval: 1 day, 0:00:00
Type: list-bower
Policy: recurring
Args:
Keyword args:
```
[2]
```
│ listers [2022-09-26 11:45:16,730: INFO/MainProcess] Connected to amqp://swhconsumer:**@scheduler0.internal.staging.swh.network:5672// │
│ listers [2022-09-26 11:45:16,973: INFO/MainProcess] lister@lister-bower-84c7fcbbf4-7r74f ready. │
│ listers [2022-09-26 11:45:16,980: INFO/MainProcess] Task swh.lister.bower.tasks.BowerListerTask[a9250c2c-175a-487e-b2a4-dfbf079909d2] received │
│ listers [2022-09-26 11:45:17,363: INFO/ForkPoolWorker-1] Fetching URL https://registry.bower.io/packages with params {} │
│ listers [2022-09-26 11:50:03,596: INFO/ForkPoolWorker-1] Task swh.lister.bower.tasks.BowerListerTask[a9250c2c-175a-487e-b2a4-dfbf079909d2] succeeded in 286.4608801769791s: {'pages': 1, 'origins': 68864}
```
[2']
```
13:46:50 swh-scheduler@db1:5432=> select now(), visit_type, count(*) from listed_origins where lister_id = ( select id from listers where name='bower') group by visit_type;
+-------------------------------+------------+-------+
| now | visit_type | count |
+-------------------------------+------------+-------+
| 2022-09-26 11:47:13.155334+00 | git | 2915 |
+-------------------------------+------------+-------+
(1 row)
Time: 76.876 ms
13:50:00 swh-scheduler@db1:5432=> select now(), visit_type, count(*) from listed_origins where lister_id = ( select id from listers where name='bower') group by visit_type;
+-------------------------------+------------+-------+
| now | visit_type | count |
+-------------------------------+------------+-------+
| 2022-09-26 11:54:15.203022+00 | git | 67904 |
+-------------------------------+------------+-------+
(1 row)
Time: 221.874 ms
```
[3]
```
13:47:13 swh-scheduler@db1:5432=> select * from listers where name='bower';
+--------------------------------------+-------+---------------+-------------------------------+---------------+-------------------------------+
| id | name | instance_name | created | current_state | updated |
+--------------------------------------+-------+---------------+-------------------------------+---------------+-------------------------------+
| f43c9008-023a-4ed1-bb0e-0c5e6f5af47b | bower | bower | 2022-09-26 11:45:17.215894+00 | {} | 2022-09-26 11:45:17.215894+00 |
+--------------------------------------+-------+---------------+-------------------------------+---------------+-------------------------------+
(1 row)
Time: 5.684 ms
swhscheduler@scheduler0:~$ swh scheduler -C /etc/softwareheritage/scheduler/listener-runner.yml origin send-to-celery --lister-uuid 'f43c9008-023a-4ed1-bb0e-0c5e6f5af47b' --tablesample 1 git
100 slots available in celery queue # <--------- cheated: increased temporarily the max queue length to 200 instead of 100 so we can schedule some more origins
100 visits to send to celery
```