Page MenuHomeSoftware Heritage

Add a helper to instantiate a new-style lister from a config file
AbandonedPublicDraft

Authored by olasd on Dec 9 2020, 5:52 PM.

Details

Summary

This helper will be used in the task entry points.

Depends on D3526

Diff Detail

Event Timeline

Build is green

Patch application report for D4700 (id=16665)

Could not rebase; Attempt merge onto d2f4781669...

Updating d2f4781..4dab647
Fast-forward
 swh/lister/__init__.py           |   9 +-
 swh/lister/pattern.py            | 260 +++++++++++++++++++++++++++++++++++++++
 swh/lister/tests/test_cli.py     |  18 ++-
 swh/lister/tests/test_pattern.py | 113 +++++++++++++++++
 4 files changed, 387 insertions(+), 13 deletions(-)
 create mode 100644 swh/lister/pattern.py
 create mode 100644 swh/lister/tests/test_pattern.py
Changes applied before test
commit 4dab6476a0916337a38d3311b2e3b908d19f438f
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Wed Nov 25 20:40:40 2020 +0100

    Add a helper to instantiate a new-style lister from a config file
    
    This helper will be used in the task entry points.

commit 45761dcb5f782ee3045184f4dbc208ce08a4fe25
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Thu Jul 16 11:59:08 2020 +0200

    Hook up listers implemented with the new pattern to the CLI
    
    We stop depending on the ListerBase implementation. The main hoop we're jumping
    through is the config override mechanism in swh.lister.get_lister, as it's
    really specifc to the ListerBase `override_config` argument, which is dropped in
    pattern.Lister (in favor of explicit arguments at lister instantiation).
    
    We implement a small shim in swh.lister.pattern.Lister to give
    backwards-compatibility for the new pattern to get_lister.
    
    This generic configuration override mechanism will probably be completely
    removed when the configuration mechanism is reworked. We'll see.

commit 011928d262546c3e2af139efea854f6ea80a09e0
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Mon Jul 6 10:27:57 2020 +0200

    Introduce a simpler base pattern for lister implementations.
    
    This new pattern uses the lister support features introduced in swh.scheduler to
    replace the database management done in previous iterations of the listers.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/72/ for more details.