Page MenuHomeSoftware Heritage

Extract the path slicing logic in a dedicated PathSlicer class
ClosedPublic

Authored by douardda on Oct 8 2021, 12:33 PM.

Details

Summary

so it can be reused in other backends, e.g. for cloud based storages,
one may want to organise the objects in a pathslicing hierarchy.

Diff Detail

Repository
rDOBJS Object storage
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build has FAILED

Patch application report for D6442 (id=23394)

Rebasing onto c0bac6bc09...

Current branch diff-target is up to date.
Changes applied before test
commit 97deea02ac14f14973769b64dcf993bf40e40a97
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 12:30:22 2021 +0200

    Extract the path slicing logic in a dedicated PathSlicer class
    
    so it can be reused in other backends, e.g. for cloud based storages,
    one may want to organise the objects in a pathslicing hierarchy.

Link to build: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/31/
See console output for more information: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/31/console

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 8 2021, 12:35 PM
Harbormaster failed remote builds in B24311: Diff 23394!

Build is green

Patch application report for D6442 (id=23394)

Rebasing onto c0bac6bc09...

Current branch diff-target is up to date.
Changes applied before test
commit 97deea02ac14f14973769b64dcf993bf40e40a97
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 12:30:22 2021 +0200

    Extract the path slicing logic in a dedicated PathSlicer class
    
    so it can be reused in other backends, e.g. for cloud based storages,
    one may want to organise the objects in a pathslicing hierarchy.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/32/ for more details.

allow the pathslicer to be a noop (with an empty slicing)

Build is green

Patch application report for D6442 (id=23398)

Rebasing onto c0bac6bc09...

Current branch diff-target is up to date.
Changes applied before test
commit 0492c28eb405f0559d38b55014b6930b06e1045b
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 12:30:22 2021 +0200

    Extract the path slicing logic in a dedicated PathSlicer class
    
    so it can be reused in other backends, e.g. for cloud based storages,
    one may want to organise the objects in a pathslicing hierarchy.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/33/ for more details.

olasd added inline comments.
swh/objstorage/backends/pathslicing.py
159–160

These args should probably be moved to the main class docstring, the rendering is weird otherwise.

swh/objstorage/tests/test_pathslicer.py
42–53

LOL. Let's hope noone actually uses this ever.

vlorentz added inline comments.
swh/objstorage/backends/pathslicing.py
42

easier to read

44–52

more readable IMO

swh/objstorage/backends/pathslicing.py
42

agreed

44–52

sure, why not

159–160

ah yes sure! this came to my mind a some point, but vanished before anything happened...

Better docstrings and kill a few map()

Build is green

Patch application report for D6442 (id=23416)

Rebasing onto c0bac6bc09...

Current branch diff-target is up to date.
Changes applied before test
commit be4094fb296df666d778f5969a9954485d16c04f
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 12:30:22 2021 +0200

    Extract the path slicing logic in a dedicated PathSlicer class
    
    so it can be reused in other backends, e.g. for cloud based storages,
    one may want to organise the objects in a pathslicing hierarchy.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/34/ for more details.

vlorentz added inline comments.
swh/objstorage/tests/test_pathslicer.py
50

forgot this one

This revision is now accepted and ready to land.Oct 12 2021, 11:44 AM
swh/objstorage/tests/test_pathslicer.py
50

thx a lot

forgotten print statement...

Build is green

Patch application report for D6442 (id=23479)

Rebasing onto c0bac6bc09...

Current branch diff-target is up to date.
Changes applied before test
commit 23b7f81c148383c788a0fffcf897fcf456980fa9
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 12:30:22 2021 +0200

    Extract the path slicing logic in a dedicated PathSlicer class
    
    so it can be reused in other backends, e.g. for cloud based storages,
    one may want to organise the objects in a pathslicing hierarchy.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/35/ for more details.