HomeSoftware Heritage

Make scheduling policy used in schedule_recurrent configurable

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

Make scheduling policy used in schedule_recurrent configurable

Add support for a configuration option "scheduling_policy" in the config
file loaded by the 'swh scheduler schedule-recurrent' command. This
config entry allows to specify the scheduling policies used by the
schedule-recurrent tool, instead of having them hardcoded in the source
code.

A visit type policy config entry should have at least a 'weight' value
for each policy.

Default values are unchanged.

Eg.:

scheduling_policy:
  git:
    - policy: already_visited_order_by_lag
      weight: 55
      tablesample: 0.5
    - policy: never_visited_oldest_update_first
      weight: 45
      tablesample: 0.5

Note: there may not be configuration entries for all visit types, but if

a visit type policy is configured, the config entry should be complete
(in other words, the merging of the configuration with the default
values is only done at first config level).

Details

Provenance
douarddaAuthored on Apr 15 2022, 6:08 PM
douarddaPushed on Apr 20 2022, 6:27 PM
Differential Revision
D7591: Make scheduling policy used in schedule_recurrent configurable
Build Status
Buildable 28586
Build 44683: test-and-buildJenkins console · Jenkins

Commit No Longer Exists

This commit no longer exists in the repository.