HomeSoftware Heritage

Unify loader instantiation

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

Unify loader instantiation

This unifies and centralizes the instantiation the same way the lister does.

This introduces a new base class swh.loader.core.loader.Loader for all loaders whose
only concern for now is to instantiate loaders from either a configuration dict or a
configuration file.

This simplifies instantiation in celery task code and avoids duplicating the
configuration load in each loader constructor.

The end goal is to simplify the future refactoring on configuration. With the following,
we will only have to adapt the Loader class when we start simplifying uniformly the
configuration.

Also note that I mostly reused the equivalent swh.lister.pattern.Lister.from_config*.
I did not refactor the common behavior (to avoid throwing another dependency in the
mix). That could always be refactored later.

(inspired by both the work on listers and the configuration system work)

Related to T1410

Details

Provenance
ardumontAuthored on Feb 8 2021, 9:04 AM
ardumontPushed on Feb 17 2021, 12:03 PM
Differential Revision
D5071: Unify loader instantiation
Tasks
T1410: Kill implicit configuration: new configuration scheme
Build Status
Buildable 19279
Build 29897: test-and-buildJenkins console · Jenkins

Commit No Longer Exists

This commit no longer exists in the repository.