Page MenuHomeSoftware Heritage
Differential D6521

Add a simple read-only HTTP backend
ClosedPublic
Actions

Authored by douardda on Oct 20 2021, 4:51 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Commits
rDOBJS8ed5f4ebc915: Add a simple read-only HTTP backend
Summary

suitable for example to use for a S3 backend.

Diff Detail

Event Timeline

douardda created this revision.Oct 20 2021, 4:51 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptOct 20 2021, 4:51 PM
swh-public-ci added a comment.Oct 20 2021, 4:53 PM

Build is green

Patch application report for D6521 (id=23703)

Rebasing onto 6269067ca7...

Current branch diff-target is up to date.
Changes applied before test
commit b84674bb6ad757e04bd5efd6573dbb8427522da8
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Oct 4 09:48:02 2021 +0200

    Add a simple read-only HTTP backend
    
    suitable for example to use for a S3 backend.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/52/ for more details.

Harbormaster completed remote builds in B24594: Diff 23703.Oct 20 2021, 4:53 PM
douardda requested review of this revision.Oct 20 2021, 4:53 PM
douardda updated this revision to Diff 23704.Oct 20 2021, 4:53 PM

remove mistakenly commited mypy.ini file

douardda updated this revision to Diff 23705.Oct 20 2021, 4:55 PM

remove useless statement

swh-public-ci added a comment.Oct 20 2021, 4:55 PM

Build is green

Patch application report for D6521 (id=23704)

Rebasing onto 6269067ca7...

Current branch diff-target is up to date.
Changes applied before test
commit 6eb9b7637c08582f2ba11e08a5254909fb00721b
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Oct 4 09:48:02 2021 +0200

    Add a simple read-only HTTP backend
    
    suitable for example to use for a S3 backend.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/53/ for more details.

Harbormaster completed remote builds in B24595: Diff 23704.Oct 20 2021, 4:55 PM
swh-public-ci added a comment.Oct 20 2021, 4:57 PM

Build is green

Patch application report for D6521 (id=23705)

Rebasing onto 6269067ca7...

Current branch diff-target is up to date.
Changes applied before test
commit 31606f2354a3aaff945ed70654b6859963fcc39f
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Oct 4 09:48:02 2021 +0200

    Add a simple read-only HTTP backend
    
    suitable for example to use for a S3 backend.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/54/ for more details.

Harbormaster completed remote builds in B24596: Diff 23705.Oct 20 2021, 4:57 PM
douardda updated this revision to Diff 23707.Oct 20 2021, 5:00 PM

document the build_objstorage() test helper function

swh-public-ci added a comment.Oct 20 2021, 5:02 PM

Build is green

Patch application report for D6521 (id=23707)

Rebasing onto 6269067ca7...

Current branch diff-target is up to date.
Changes applied before test
commit 201b9ecc36d20b6ac7f0413dd922d9d6a2204ce8
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Oct 4 09:48:02 2021 +0200

    Add a simple read-only HTTP backend
    
    suitable for example to use for a S3 backend.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/55/ for more details.

Harbormaster completed remote builds in B24598: Diff 23707.Oct 20 2021, 5:02 PM
vlorentz requested changes to this revision.Oct 21 2021, 11:46 AM
vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/objstorage/backends/http.py
44

why would it?

51–61

could you add messages? Just "HTTPReadOnlyObjStorage does not implement xxx" would be fine. (the traceback isn't always preserved when the error goes through a serialization layer)

swh/objstorage/tests/test_objstorage_http.py
39

not covered?

This revision now requires changes to proceed.Oct 21 2021, 11:46 AM
douardda added inline comments.Oct 21 2021, 12:24 PM
swh/objstorage/backends/http.py
44

blind copy/paste... will remove

51–61

good thinking, will do

swh/objstorage/tests/test_objstorage_http.py
39

indeed, it's covered for the HEAD query but not the GET. Will fix

douardda updated this revision to Diff 23719.Oct 21 2021, 12:50 PM

Use ReadOnlyObjStorage and NonIterableObjStorage instead of NotImplementedError

and add a few tests.

swh-public-ci added a comment.Oct 21 2021, 12:52 PM

Build has FAILED

Patch application report for D6521 (id=23719)

Rebasing onto 6269067ca7...

Current branch diff-target is up to date.
Changes applied before test
commit 5b94c973119e1df221329837e0f3b9bef7961f09
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Oct 4 09:48:02 2021 +0200

    Add a simple read-only HTTP backend
    
    suitable for example to use for a S3 backend.

Link to build: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/56/
See console output for more information: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/56/console

Harbormaster failed remote builds in B24608: Diff 23719!Oct 21 2021, 12:52 PM
swh-public-ci added a comment.Oct 21 2021, 1:05 PM

Build is green

Patch application report for D6521 (id=23719)

Rebasing onto 6269067ca7...

Current branch diff-target is up to date.
Changes applied before test
commit 5b94c973119e1df221329837e0f3b9bef7961f09
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Oct 4 09:48:02 2021 +0200

    Add a simple read-only HTTP backend
    
    suitable for example to use for a S3 backend.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/57/ for more details.

Harbormaster completed remote builds in B24608: Diff 23719.Oct 21 2021, 1:05 PM
douardda updated this revision to Diff 23734.Oct 21 2021, 2:09 PM

rebase

swh-public-ci added a comment.Oct 21 2021, 2:11 PM

Build is green

Patch application report for D6521 (id=23734)

Rebasing onto bcbbfd4669...

Current branch diff-target is up to date.
Changes applied before test
commit 8ed5f4ebc91524639906d9866f7828660ccd9b06
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Oct 4 09:48:02 2021 +0200

    Add a simple read-only HTTP backend
    
    suitable for example to use for a S3 backend.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/63/ for more details.

Harbormaster completed remote builds in B24623: Diff 23734.Oct 21 2021, 2:11 PM
douardda marked 3 inline comments as done.Oct 21 2021, 2:12 PM
vlorentz accepted this revision.Oct 21 2021, 2:18 PM

even better, thx

This revision is now accepted and ready to land.Oct 21 2021, 2:18 PM
Closed by commit rDOBJS8ed5f4ebc915: Add a simple read-only HTTP backend (authored by douardda). · Explain WhyOct 21 2021, 2:23 PM
This revision was automatically updated to reflect the committed changes.
douardda added a commit: rDOBJS8ed5f4ebc915: Add a simple read-only HTTP backend.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
2 lines--
swh/
objstorage/
backends/
93 lines83%
Loading...
2 lines78%
Loading...
tests/
90 lines98%
Loading...

Diff 23703

View Options

mypy.ini

Loading...
View Options

swh/objstorage/backends/http.py

Loading...
View Options

swh/objstorage/factory.py

Loading...
View Options

swh/objstorage/tests/test_objstorage_http.py

Loading...
About · Developer information · Legal