HomeSoftware Heritage

Add 'pipeline' storage "class" for more readable configurations.

Description

Add 'pipeline' storage "class" for more readable configurations.

This would allow writing configurations like:

storage:
  cls: pipeline
  steps:
    - cls: filter
    - cls: buffer
    - cls: remote
      url: http://swh-storage:5002/

or

storage:
  cls: filter
  storage:
    cls: buffer
    storage:
      cls: remote
      url: http://swh-storage:5002/

instead of:

storage:
  cls: filter
  args:
    storage:
      cls: buffer
      args:
        storage:
          cls: remote
          args:
            url: http://swh-storage:5002/

Details

Provenance
vlorentzAuthored on Nov 14 2019, 2:16 PM
vlorentzPushed on Nov 19 2019, 4:17 AM
Differential Revision
D2276: Add get_storage_pipeline utility function for more readable configurations.
Parents
rDSTOea9aa4736fa7: and not only for an existing origin visit.
Branches
Unknown
Tags
Unknown
Build Status
Buildable 9111
Build 13322: test-and-buildJenkins