HomeSoftware Heritage

swh.scheduler.cli: Add a bulk index flag to separate read from index

Description

swh.scheduler.cli: Add a bulk index flag to separate read from index

Try to reduce the number of connection timeout error.

There are 3 solutions:

  • either increase the default timeout of 10 per request
  • same on server configuration basis (no)
  • either reduce the number of data to bulk index, this is the chosen solution

Related T1023

Details

Provenance
ardumontAuthored on Apr 13 2018, 11:07 AM
ardumontPushed on Apr 13 2018, 2:55 PM
Parents
rDSCHe972b6aebe77: swh.scheduler.cli.archive: Simplify task and task_run ids extraction
Branches
Unknown
Tags
Unknown