Page MenuHomeSoftware Heritage

Add return type to get_objstorage + deduplicate its code
ClosedPublic

Authored by vlorentz on Jun 22 2022, 3:57 PM.

Diff Detail

Repository
rDOBJS Object storage
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D8024 (id=28895)

Could not rebase; Attempt merge onto 2c7e49136a...

Updating 2c7e491..847285a
Fast-forward
 swh/objstorage/api/client.py                       |  14 ++-
 swh/objstorage/backends/azure.py                   |  27 ++---
 swh/objstorage/backends/generator.py               |  10 +-
 swh/objstorage/backends/http.py                    |  18 ++--
 swh/objstorage/backends/in_memory.py               |  19 ++--
 swh/objstorage/backends/libcloud.py                |  20 ++--
 swh/objstorage/backends/noop.py                    |   2 +-
 swh/objstorage/backends/pathslicing.py             |  53 ++++-----
 swh/objstorage/backends/seaweedfs/objstorage.py    |  34 +++---
 swh/objstorage/backends/winery/objstorage.py       |  33 ++----
 swh/objstorage/cli.py                              |   8 +-
 swh/objstorage/constants.py                        |  17 +++
 swh/objstorage/factory.py                          | 119 ++++++++-------------
 swh/objstorage/interface.py                        |  64 ++++++-----
 swh/objstorage/multiplexer/filter/filter.py        |   4 +-
 swh/objstorage/multiplexer/filter/id_filter.py     |   9 +-
 .../multiplexer/multiplexer_objstorage.py          |  22 ++--
 swh/objstorage/objstorage.py                       | 112 +++++++++----------
 swh/objstorage/tests/objstorage_testing.py         |  26 ++---
 swh/objstorage/tests/test_multiplexer_filter.py    |  56 +++++-----
 swh/objstorage/tests/test_objstorage_api.py        |   2 +-
 swh/objstorage/tests/test_objstorage_azure.py      |  22 ++--
 swh/objstorage/tests/test_objstorage_http.py       |  17 +--
 swh/objstorage/tests/test_objstorage_in_memory.py  |   2 +-
 .../tests/test_objstorage_instantiation.py         |   4 +-
 .../tests/test_objstorage_multiplexer.py           |   4 +-
 .../tests/test_objstorage_pathslicing.py           |  10 +-
 .../tests/test_objstorage_random_generator.py      |   8 +-
 swh/objstorage/tests/test_objstorage_striping.py   |  37 +++----
 swh/objstorage/tests/test_objstorage_winery.py     |   5 +-
 swh/objstorage/tests/test_server.py                |  13 ++-
 31 files changed, 356 insertions(+), 435 deletions(-)
 create mode 100644 swh/objstorage/constants.py
Changes applied before test
commit 847285a28da8164a3ea618d9c14c14280058c4f8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 22 15:57:14 2022 +0200

    Add return type to get_objstorage

commit a3c57499fd98c16d3ac33fa0582083edda253733
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 22 15:56:37 2022 +0200

    Deduplicate get_objstorage ImportError handling
    
    And this is necessary to make the function type-check.

commit 18cec535b74cc13184168de282f593bbf7717fbf
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 22 15:44:45 2022 +0200

    Remove deprecated alias 'weed' of 'seaweedfs'.
    
    It is not used anymore.

commit 55fad44276b597b619a42ab61d80d388b9c6c06d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 22 11:38:53 2022 +0200

    tests: Remove explicit "args" arguments to fix deprecation warnings

commit 4184c0a4610c208597e3617c2d2d913aa0975f00
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 22 11:18:43 2022 +0200

    Make add() and restore() return None instead of ObjId
    
    The return value was useless, as the same ObjId has to be passed as argument.
    
    And as far as I can tell, no other package depends on this return value.

commit 68b2be94d8af30464350c0858fce5a55ef25daf8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 21 17:56:55 2022 +0200

    Add type annotations to all objstorage backends

commit 56216ab438cb0620ccc3fd5b6173d8d4e5aff4fa
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 21 18:05:44 2022 +0200

    libcloud: Default to gzip compression
    
    Like other backends.
    
    Allowing None will be an issue once we add type annotations to
    methods which use self.compression.

commit 6cdfe555fab7f75423bae7d820b2cd25ef4f4530
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 21 16:05:45 2022 +0200

    Make obj_id argument of ObjStorage.restore() required
    
    For consistency with ObjStorage.add()

commit 99e0d40bb0d2f30dc071ad23692a661b348171fc
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 21 16:00:43 2022 +0200

    Make obj_id argument of ObjStorage.add() required
    
    In practice, it is not very useful to recompute it in the objstorage,
    as callers already know it.

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/159/ for more details.

olasd added a subscriber: olasd.

We've run mypy on dependent modules and they seem to be happy enough.

This revision is now accepted and ready to land.Jun 22 2022, 4:03 PM