Page MenuHomeSoftware Heritage

objstorage: Return summary of batched object added to the store
ClosedPublic

Authored by ardumont on Feb 7 2020, 11:35 AM.

Details

Summary

This will simplify this computation done storage side.

Test Plan

tox

Diff Detail

Repository
rDOBJS Object storage
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz added inline comments.
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

you replaced a mean with the first result

swh/objstorage/objstorage.py
173–174

s/content/object/

swh/objstorage/tests/objstorage_testing.py
162–176

s/content/object/

tox.ini
8

you sure?

This revision now requires changes to proceed.Feb 7 2020, 11:57 AM
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

Thanks!

swh/objstorage/objstorage.py
173–174

yup, incoming updates.

tox.ini
8

yeah, you want me to remove it?

  • Rename content to object
  • Fix to compute correctly the mean
vlorentz added a subscriber: olasd.
vlorentz added inline comments.
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

@olasd mentionned using a Counter, to get shorter code. I think it could work like this:

results = self.wrap_call(
            write_threads, 'add_batch', contents,
            check_presence=check_presence,
        )

summed = sum(results)

return {
        'object:add': summed['object:add'] // len(results),
        'object:add:bytes': summed['object:add:bytes'] // len(results),
    }
This revision now requires changes to proceed.Feb 7 2020, 12:08 PM
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

also, s/result/results in the sum

Fix

swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

yeah...

that's the real issue here ;)

This revision is now accepted and ready to land.Feb 7 2020, 3:19 PM