We need a cron-like service to periodically reschedule (= submit relevant celery tasks) recurrent tasks.
Use cases:
- ghlister daily updates
- ghlister complete re-listing (monthly?)
- git cloning/loading of new github repos
- git fetching/loading of already known github repos
- ...
As an architecture, we want the service to read a list of "manifests", that map cron-like periods to swh.core.scheduling.Task objects + their configuration parameters.
As a technological building block we might want to use [[ https://pypi.python.org/pypi/croniter/ | croniter ]].
Open design questions:
- cron-like (i.e., will //not// catch up with past tasks, e.g., if the daemon was down), or anacron-like (with catchup)