HomeSoftware Heritage

Separate loader-deposit from other loaders

Description

Separate loader-deposit from other loaders

Fix the build [1]

Loaders have been reworked to only deal with configuration as constructor parameters. As
the current docker configuration is shared amongst all loaders, this can no longer work.

The current "next-gen" loaders share a subset of those configuration though so most can
run together. Except for the loader deposit which needs dedicated extra keys (deposit,
url).

Note that some configuration keys (scheduler for example) referenced in the current
configuration are not for loaders. This make instantiation fails. So they need to be
removed.

All in all, trying to separate the dedicated deposit loader with its configuration in
its own container and let the other loaders running as before fixes the build.

That and stop referecing the scheduler configuration in the loader configuration.
Instead use a dedicated environment variable to specify the scheduler url to use.

Related to T1410

[1] https://jenkins.softwareheritage.org/view/all/job/swh-docker-dev/730/console

Details

Provenance
ardumontAuthored on Feb 17 2021, 5:07 PM
ardumontPushed on Feb 18 2021, 11:13 AM
Differential Revision
D5102: Separate loader-deposit from other loaders
Parents
rDENV41cfed6732f5: docker: Bump compose file format version to 2.1
Branches
Unknown
Tags
Unknown
Tasks
T1410: Kill implicit configuration: new configuration scheme