Page MenuHomeSoftware Heritage

Provide a unified API for listers to interact with the scheduler
Closed, MigratedEdits Locked

Description

Currently, listers interact with the scheduler to create recurrent loading tasks, and pull back references to tasks into their own database schema.

In order to improve our scheduling of recurrent visits, we need listers to provide more accurate and uniform data to the scheduler, rather than just plain loading tasks.

We should also keep in mind how such a unified API would help us to provide the data backing a full implementation of T1870.

Related Objects

Event Timeline

olasd triaged this task as High priority.Jun 9 2020, 4:15 PM
olasd created this task.
gitlab-migration changed the status of subtask T2956: Port PyPI lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2972: Port npm lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2979: Port debian lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2984: Port cgit lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2987: Port gitlab lister to the new `swh.lister.pattern.Lister` API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2989: Port CRAN lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2990: Port GNU lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2991: Port packagist lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2992: Port launchpad lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T3073: Properly document the new unified API lister from Resolved to Migrated.