HomeSoftware Heritage

data/defaults: Reduce triggering cache flush period to scheduler

Description

data/defaults: Reduce triggering cache flush period to scheduler

As can be seen in the graph [1], every hour a timer triggers the
writer service. This writes the events as oneshot tasks in the
scheduler db (type 'origin-update-git' at the moment).

For now, this is growing every hour, then decreases slightly as tasks
are scheduled and done. But not as fast as the production of
events. In reducing that period, that gives the consumer more time to
aggregate events some more (on the same origin). In effect, reducing
the number of oneshot tasks to (re)schedule (since duplicates not yet
scheduled will be dropped).

[1] http://munin.internal.softwareheritage.org/softwareheritage.org/prado.softwareheritage.org/softwareheritage_scheduler_oneshot.html

Details

Provenance
ardumontAuthored on Jun 1 2018, 4:30 PM
ardumontPushed on Jun 1 2018, 7:44 PM
Parents
rSPSITE91e6b8113b17: Increase license indexer concurrency to keep up with mimetype
Branches
Unknown
Tags
Unknown