HomeSoftware Heritage

cassandra: Use concurrent queries in *_missing() instead of naive grouping

Description

cassandra: Use concurrent queries in *_missing() instead of naive grouping

Instead of grouping ids in queries in arbitrary batches (which forces
the server node to coordinate with other nodes to complete the query),
this sends queries with one id each, directly to the right node.

This is the 'concurrent' algorithm from https://forge.softwareheritage.org/T3577#72791
which gives a >=2x speed-up on directories, and a >=8x speed-up on revisions.

Details

Provenance
vlorentzAuthored on Jan 6 2022, 12:41 PM
vlorentzPushed on Jan 6 2022, 5:29 PM
Differential Revision
D6885: cassandra: Use concurrent queries in *_missing() instead of naive grouping
Parents
rDSTO259bf6fe1e3b: Improve documentation of the replay command
Branches
Unknown
Tags
Unknown
Build Status
Buildable 25854
Build 40404: test-and-buildJenkins console · Jenkins