Page MenuHomeSoftware Heritage

objstorage: Return summary of batched object added to the store
ClosedPublic

Authored by ardumont on Fri, Feb 7, 11:35 AM.

Details

Summary

This will simplify this computation done storage side.

Test Plan

tox

Diff Detail

Repository
rDOBJS Object storage
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Fri, Feb 7, 11:35 AM
vlorentz requested changes to this revision.Fri, Feb 7, 11:57 AM
vlorentz added inline comments.
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

you replaced a mean with the first result

swh/objstorage/objstorage.py
174–175

s/content/object/

swh/objstorage/tests/objstorage_testing.py
163–177

s/content/object/

tox.ini
9

you sure?

This revision now requires changes to proceed.Fri, Feb 7, 11:57 AM
ardumont added inline comments.Fri, Feb 7, 12:02 PM
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

Thanks!

swh/objstorage/objstorage.py
174–175

yup, incoming updates.

tox.ini
9

yeah, you want me to remove it?

ardumont updated this revision to Diff 9432.Fri, Feb 7, 12:06 PM
  • Rename content to object
  • Fix to compute correctly the mean
vlorentz requested changes to this revision.Fri, Feb 7, 12:08 PM
vlorentz added a subscriber: olasd.
vlorentz added inline comments.
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

@olasd mentionned using a Counter, to get shorter code. I think it could work like this:

results = self.wrap_call(
            write_threads, 'add_batch', contents,
            check_presence=check_presence,
        )

summed = sum(results)

return {
        'object:add': summed['object:add'] // len(results),
        'object:add:bytes': summed['object:add:bytes'] // len(results),
    }
This revision now requires changes to proceed.Fri, Feb 7, 12:08 PM
vlorentz added inline comments.Fri, Feb 7, 12:09 PM
swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

also, s/result/results in the sum

ardumont updated this revision to Diff 9433.Fri, Feb 7, 3:17 PM

Fix

swh/objstorage/multiplexer/multiplexer_objstorage.py
248–262

yeah...

that's the real issue here ;)

vlorentz accepted this revision.Fri, Feb 7, 3:19 PM
This revision is now accepted and ready to land.Fri, Feb 7, 3:19 PM