Page MenuHomeSoftware Heritage

Improve tests of the seaweedfs backend
ClosedPublic

Authored by douardda on Oct 20 2021, 12:11 PM.

Details

Summary

tests for this backend are now much more realistic,
mocking a seaweedfs Filer server instead of the WeedFiler directly.

And actually fix a bug in WeedFiler.get()...

Also remove tests for each compression algo, keeping only one
is enought.

Also improve a bit the seaweedfs backend

Diff Detail

Event Timeline

Build has FAILED

Patch application report for D6517 (id=23682)

Rebasing onto 23b7f81c14...

Current branch diff-target is up to date.
Changes applied before test
commit 78b1e82b8f87fbf7d6bb034204d5210ed71824cc
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Oct 20 10:34:41 2021 +0200

    Improve a bit the seaweedfs backend

commit 55ff4b95d3062421a7be6465c6ca07f8cd4909ab
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 14:01:24 2021 +0200

    Improve tests of the seaweedfs backend
    
    tests for this backend are now much more realistic,
    mocking a seaweedfs Filer server instead of the `WeedFiler` directly.
    
    And actually fix a bug in WeedFiler.get()...
    
    Also remove tests for each compression algo, keeping only one
    is enought.

Link to build: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/40/
See console output for more information: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/40/console

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 20 2021, 12:14 PM
Harbormaster failed remote builds in B24571: Diff 23682!

add a comment about DirMode value

Build has FAILED

Patch application report for D6517 (id=23684)

Rebasing onto 23b7f81c14...

Current branch diff-target is up to date.
Changes applied before test
commit be26482e93e15a2a699f9f35ce63286ccbf737b5
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Oct 20 10:34:41 2021 +0200

    Improve a bit the seaweedfs backend
    
    - use a persistent session object in WeedFiler,
    - better implementation of listing methods

commit 55ff4b95d3062421a7be6465c6ca07f8cd4909ab
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 14:01:24 2021 +0200

    Improve tests of the seaweedfs backend
    
    tests for this backend are now much more realistic,
    mocking a seaweedfs Filer server instead of the `WeedFiler` directly.
    
    And actually fix a bug in WeedFiler.get()...
    
    Also remove tests for each compression algo, keeping only one
    is enought.

Link to build: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/42/
See console output for more information: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/42/console

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 20 2021, 12:35 PM
Harbormaster failed remote builds in B24573: Diff 23684!

Build is green

Patch application report for D6517 (id=23682)

Rebasing onto 23b7f81c14...

Current branch diff-target is up to date.
Changes applied before test
commit 78b1e82b8f87fbf7d6bb034204d5210ed71824cc
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Oct 20 10:34:41 2021 +0200

    Improve a bit the seaweedfs backend

commit 55ff4b95d3062421a7be6465c6ca07f8cd4909ab
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 14:01:24 2021 +0200

    Improve tests of the seaweedfs backend
    
    tests for this backend are now much more realistic,
    mocking a seaweedfs Filer server instead of the `WeedFiler` directly.
    
    And actually fix a bug in WeedFiler.get()...
    
    Also remove tests for each compression algo, keeping only one
    is enought.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/44/ for more details.

vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/objstorage/backends/seaweed.py
74–114

would this work, as a minor simplification?

This revision is now accepted and ready to land.Oct 20 2021, 2:15 PM
swh/objstorage/backends/seaweed.py
74–114

need to check against actual seaweed filer

Build is green

Patch application report for D6517 (id=23689)

Rebasing onto 23b7f81c14...

Current branch diff-target is up to date.
Changes applied before test
commit be26482e93e15a2a699f9f35ce63286ccbf737b5
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Oct 20 10:34:41 2021 +0200

    Improve a bit the seaweedfs backend
    
    - use a persistent session object in WeedFiler,
    - better implementation of listing methods

commit 55ff4b95d3062421a7be6465c6ca07f8cd4909ab
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 14:01:24 2021 +0200

    Improve tests of the seaweedfs backend
    
    tests for this backend are now much more realistic,
    mocking a seaweedfs Filer server instead of the `WeedFiler` directly.
    
    And actually fix a bug in WeedFiler.get()...
    
    Also remove tests for each compression algo, keeping only one
    is enought.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/46/ for more details.

slight simplification as suggested by vlorentz

Build has FAILED

Patch application report for D6517 (id=23691)

Rebasing onto 23b7f81c14...

Current branch diff-target is up to date.
Changes applied before test
commit 96eadebd2d05f06bc87fb5461892e80fa98ebefe
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Oct 20 10:34:41 2021 +0200

    Improve a bit the seaweedfs backend
    
    - use a persistent session object in WeedFiler,
    - better implementation of listing methods

commit 55ff4b95d3062421a7be6465c6ca07f8cd4909ab
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 14:01:24 2021 +0200

    Improve tests of the seaweedfs backend
    
    tests for this backend are now much more realistic,
    mocking a seaweedfs Filer server instead of the `WeedFiler` directly.
    
    And actually fix a bug in WeedFiler.get()...
    
    Also remove tests for each compression algo, keeping only one
    is enought.

Link to build: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/48/
See console output for more information: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/48/console

remove a (comment) garbage line

Build is green

Patch application report for D6517 (id=23693)

Rebasing onto 23b7f81c14...

Current branch diff-target is up to date.
Changes applied before test
commit 6269067ca7b3599ecc7cc2436ee72a4f284fdd85
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Oct 20 10:34:41 2021 +0200

    Improve a bit the seaweedfs backend
    
    - use a persistent session object in WeedFiler,
    - better implementation of listing methods

commit 55ff4b95d3062421a7be6465c6ca07f8cd4909ab
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 8 14:01:24 2021 +0200

    Improve tests of the seaweedfs backend
    
    tests for this backend are now much more realistic,
    mocking a seaweedfs Filer server instead of the `WeedFiler` directly.
    
    And actually fix a bug in WeedFiler.get()...
    
    Also remove tests for each compression algo, keeping only one
    is enought.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/50/ for more details.