Page MenuHomeSoftware Heritage
Create Task
Maniphest T2442

Provide a unified API for listers to interact with the scheduler
Closed, MigratedEdits Locked
Actions

Description

Currently, listers interact with the scheduler to create recurrent loading tasks, and pull back references to tasks into their own database schema.

In order to improve our scheduling of recurrent visits, we need listers to provide more accurate and uniform data to the scheduler, rather than just plain loading tasks.

We should also keep in mind how such a unified API would help us to provide the data backing a full implementation of T1870.

Revisions and Commits

rDSCH Scheduling utilities
D3289rDSCHd107a5553414 Implement storage of listed origins
D3271rDSCH1c93e553a16a Implement basic storage and retrieval of lister information
rDLS Listers
D4992rDLS89335445210f Remove no longer used legacy Lister API and update CLI options
D4843rDLS565e7423e39f Reimplement Bitbucket lister using new Lister API

Related Objects
Search...

Event Timeline

olasd triaged this task as High priority.Jun 9 2020, 4:15 PM
olasd created this task.
olasd added a parent task: T2444: Implement the scheduling policy for the recurrent visit scheduler.Jun 9 2020, 5:10 PM
olasd mentioned this in D3271: Implement basic storage of lister information.Jun 12 2020, 11:22 AM
olasd added a revision: D3271: Implement basic storage of lister information.Jun 15 2020, 3:02 PM
olasd added a commit: rDSCH1c93e553a16a: Implement basic storage and retrieval of lister information.Jun 15 2020, 4:52 PM
olasd added a revision: D3289: Implement storage of listed origins.Jun 16 2020, 11:01 AM
olasd mentioned this in T2453: Use the scheduler "lister api" in listers.Jun 16 2020, 11:23 AM
olasd added a parent task: T2453: Use the scheduler "lister api" in listers.
olasd added a commit: rDSCHd107a5553414: Implement storage of listed origins.Jun 16 2020, 10:31 PM
ardumont added a project: Sprint 2021 01.Jan 11 2021, 12:36 PM
tenma mentioned this in T2955: Port Bitbucket lister to the new Lister API.Jan 11 2021, 7:15 PM
tenma mentioned this in T2956: Port PyPI lister to the new Lister API.Jan 11 2021, 8:07 PM
anlambert added a subtask: T2955: Port Bitbucket lister to the new Lister API.Jan 13 2021, 7:14 PM
anlambert added a subtask: T2956: Port PyPI lister to the new Lister API.
anlambert changed the status of subtask T2955: Port Bitbucket lister to the new Lister API from Open to Work in Progress.
tenma added a revision: D4843: Reimplement Bitbucket lister using new Lister API.Jan 14 2021, 10:33 AM
tenma mentioned this in T2971: Port Gitea lister to the new Lister API.Jan 14 2021, 12:59 PM
anlambert changed the status of subtask T2972: Port npm lister to the new Lister API from Open to Work in Progress.Jan 15 2021, 4:23 PM
anlambert changed the status of subtask T2979: Port debian lister to the new Lister API from Open to Work in Progress.Jan 19 2021, 2:29 PM
tenma added a commit: rDLS565e7423e39f: Reimplement Bitbucket lister using new Lister API.Jan 20 2021, 3:29 PM
tenma closed subtask T2955: Port Bitbucket lister to the new Lister API as Resolved.Jan 20 2021, 3:47 PM
anlambert closed subtask T2972: Port npm lister to the new Lister API as Resolved.Jan 22 2021, 11:11 AM
tenma closed subtask T2956: Port PyPI lister to the new Lister API as Resolved.Jan 22 2021, 3:59 PM
anlambert added a subtask: T2987: Port gitlab lister to the new `swh.lister.pattern.Lister` API.Jan 25 2021, 2:18 PM
ardumont added a subtask: T2984: Port cgit lister to the new Lister API.Jan 25 2021, 2:19 PM
ardumont changed the status of subtask T2987: Port gitlab lister to the new `swh.lister.pattern.Lister` API from Open to Work in Progress.
anlambert changed the status of subtask T2989: Port CRAN lister to the new Lister API from Open to Work in Progress.Jan 25 2021, 2:57 PM
ardumont closed subtask T2984: Port cgit lister to the new Lister API as Resolved.Jan 26 2021, 9:57 AM
anlambert closed subtask T2989: Port CRAN lister to the new Lister API as Resolved.Jan 26 2021, 3:24 PM
anlambert closed subtask T2979: Port debian lister to the new Lister API as Resolved.Jan 26 2021, 5:29 PM
ardumont closed subtask T2987: Port gitlab lister to the new `swh.lister.pattern.Lister` API as Resolved.Jan 26 2021, 7:06 PM
anlambert changed the status of subtask T2992: Port launchpad lister to the new Lister API from Open to Work in Progress.Jan 27 2021, 4:05 PM
anlambert closed subtask T2992: Port launchpad lister to the new Lister API as Resolved.Jan 28 2021, 3:28 PM
anlambert closed subtask T2990: Port GNU lister to the new Lister API as Resolved.Jan 29 2021, 3:10 PM
anlambert changed the status of subtask T2991: Port packagist lister to the new Lister API from Open to Work in Progress.Feb 1 2021, 1:37 PM
anlambert added a revision: D4992: Remove no longer used legacy Lister API and update CLI options.Feb 2 2021, 12:45 PM
anlambert closed this task as Resolved by committing rDLS89335445210f: Remove no longer used legacy Lister API and update CLI options.Feb 2 2021, 4:08 PM
anlambert closed subtask T2991: Port packagist lister to the new Lister API as Resolved.
anlambert added a commit: rDLS89335445210f: Remove no longer used legacy Lister API and update CLI options.
ardumont closed subtask T3073: Properly document the new unified API lister as Resolved.Feb 26 2021, 5:33 PM
gitlab-migration changed the task status from Resolved to Migrated.Jan 8 2023, 4:30 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

gitlab-migration changed the status of subtask T2955: Port Bitbucket lister to the new Lister API from Resolved to Migrated.Jan 8 2023, 4:33 PM
gitlab-migration changed the status of subtask T2956: Port PyPI lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2972: Port npm lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2979: Port debian lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2984: Port cgit lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2987: Port gitlab lister to the new `swh.lister.pattern.Lister` API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2989: Port CRAN lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2990: Port GNU lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2991: Port packagist lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T2992: Port launchpad lister to the new Lister API from Resolved to Migrated.
gitlab-migration changed the status of subtask T3073: Properly document the new unified API lister from Resolved to Migrated.
About · Developer information · Legal