HomeSoftware Heritage

orchestrator: Send contents batch on a per indexer configuration basis

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

orchestrator: Send contents batch on a per indexer configuration basis

Prior to this, each indexer received the same batch of contents as the
orchestrator.

As some of our indexer need quite some time to finish when the batch
size is huge enough, the only reasonable step was to reduce such batch
size. But, in effect, this impacted as well other indexers which did
not need such restriction (this for example creates more little db
transations).

The introduction of such option avoids that impact.

This also introduces the option to check the presence on a per indexer
basis.

Related T713

Details

Provenance
ardumontAuthored on May 17 2017, 1:42 PM
ardumontPushed on May 17 2017, 2:14 PM

Commit No Longer Exists

This commit no longer exists in the repository.