Page MenuHomeSoftware Heritage

objstorage: Return summary of batched object added to the store
ClosedPublic

Authored by ardumont on Feb 7 2020, 11:35 AM.

Details

Summary

This will simplify this computation done storage side.

Test Plan

tox

Diff Detail

Repository
rDOBJS Object storage
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 10492
Build 15635: tox-on-jenkinsJenkins
Build 15634: arc lint + arc unit

Event Timeline

vlorentz added inline comments.
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–263

you replaced a mean with the first result

swh/objstorage/objstorage.py
175–176

s/content/object/

swh/objstorage/tests/objstorage_testing.py
162–176

s/content/object/

tox.ini
8

you sure?

This revision now requires changes to proceed.Feb 7 2020, 11:57 AM
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–263

Thanks!

swh/objstorage/objstorage.py
175–176

yup, incoming updates.

tox.ini
8

yeah, you want me to remove it?

  • Rename content to object
  • Fix to compute correctly the mean
vlorentz added a subscriber: olasd.
vlorentz added inline comments.
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

@olasd mentionned using a Counter, to get shorter code. I think it could work like this:

results = self.wrap_call(
            write_threads, 'add_batch', contents,
            check_presence=check_presence,
        )

summed = sum(results)

return {
        'object:add': summed['object:add'] // len(results),
        'object:add:bytes': summed['object:add:bytes'] // len(results),
    }
This revision now requires changes to proceed.Feb 7 2020, 12:08 PM
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

also, s/result/results in the sum

Fix

swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

yeah...

that's the real issue here ;)

This revision is now accepted and ready to land.Feb 7 2020, 3:19 PM