HomeSoftware Heritage

azure: Specialize get_batch() to fetch objects concurrently

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

azure: Specialize get_batch() to fetch objects concurrently

The default implementation is a naive 'for' loop calling 'get'
sequentially.

This replaces this default implementation with a local asyncio loop,
which manages a task for each of the blobs to fetch.

To avoid code duplication, this also rewrites 'get' to use the same
code as 'get_batch'. Quick benchmarks did not show a noticeable
slowdown when doing this.

Details

Provenance
vlorentzAuthored on May 17 2021, 1:40 PM
vlorentzPushed on May 25 2021, 12:50 PM
Differential Revision
D5743: azure: Specialize get_batch() to fetch objects concurrently
Build Status
Buildable 21611
Build 33582: test-and-buildJenkins console · Jenkins

Commit No Longer Exists

This commit no longer exists in the repository.