Page MenuHomeSoftware Heritage

objstorage: Return summary of batched object added to the store
ClosedPublic

Authored by ardumont on Feb 7 2020, 11:35 AM.

Details

Summary

This will simplify this computation done storage side.

Test Plan

tox

Diff Detail

Repository
rDOBJS Object storage
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 10490
Build 15632: tox-on-jenkinsJenkins
Build 15631: arc lint + arc unit

Event Timeline

vlorentz added inline comments.
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–252

you replaced a mean with the first result

swh/objstorage/objstorage.py
176–177

s/content/object/

swh/objstorage/tests/objstorage_testing.py
163–177

s/content/object/

tox.ini
9

you sure?

This revision now requires changes to proceed.Feb 7 2020, 11:57 AM
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–252

Thanks!

swh/objstorage/objstorage.py
176–177

yup, incoming updates.

tox.ini
9

yeah, you want me to remove it?

  • Rename content to object
  • Fix to compute correctly the mean
vlorentz added a subscriber: olasd.
vlorentz added inline comments.
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–253

@olasd mentionned using a Counter, to get shorter code. I think it could work like this:

results = self.wrap_call(
            write_threads, 'add_batch', contents,
            check_presence=check_presence,
        )

summed = sum(results)

return {
        'object:add': summed['object:add'] // len(results),
        'object:add:bytes': summed['object:add:bytes'] // len(results),
    }
This revision now requires changes to proceed.Feb 7 2020, 12:08 PM
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–253

also, s/result/results in the sum

Fix

swh/objstorage/multiplexer/multiplexer_objstorage.py
248–253

yeah...

that's the real issue here ;)

This revision is now accepted and ready to land.Feb 7 2020, 3:19 PM