HomeSoftware Heritage

Use billiard instead of stdlib multiprocessing

Description

Use billiard instead of stdlib multiprocessing

This circumvents a few celery-related issues, and is consistent with
what the rest of the codebase does.

stdlib multiprocessing is not able to spawn children from daemonic
processes, and even says so plainly if you try:

AssertionError: daemonic processes are not allowed to have children

This is incompatible with the SWH infrastructure which needs to do this
exactly. Fortunately, we're already using billiard and celery. I'm
assuming that there could be other blocking or annoying differences
between stdlib and billiard, but we will save ourselves the trouble of
finding out.

Details

Provenance
AlphareAuthored on Apr 27 2021, 10:53 AM
AlpharePushed on Apr 30 2021, 11:04 AM
AlpharePushed on Apr 30 2021, 11:01 AM
AlpharePushed on Apr 30 2021, 10:57 AM
Differential Revision
D5623: Use billiard instead of stdlib multiprocessing
Parents
rDLDHG504ee123ce80: tox: Add sphinx environments to check sane doc build
Branches
Unknown
Tags
Unknown
Build Status
Buildable 21164
Build 32849: test-and-buildJenkins console · Jenkins