Page MenuHomeSoftware Heritage

cloud: add support for compression
ClosedPublic

Authored by douardda on Mar 19 2019, 12:31 PM.

Details

Summary

And use it in libcloud based backends.

Depends on D1274

Diff Detail

Repository
rDOBJS Object storage
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/objstorage/cloud/objstorage_cloud.py
174–179

Do we really need this?

This revision is now accepted and ready to land.Mar 19 2019, 1:44 PM
douardda added inline comments.
swh/objstorage/cloud/objstorage_cloud.py
174–179

easy hack to prevent from iterating a bytes one byte at a time.

Note that this implementation is a bit naive on the get side: we do uncompress the whole object.

Actually, this whole compression support should be discussed a bit more. This Diff is a stopgap

updated, remove smart stuff and add some tests

'smart' as 'do not compress if the blob is too small'...

This revision was automatically updated to reflect the committed changes.