Page MenuHomeSoftware Heritage

cloud: add support for compression
ClosedPublic

Authored by douardda on Mar 19 2019, 12:31 PM.

Details

Summary

And use it in libcloud based backends.

Depends on D1274

Diff Detail

Repository
rDOBJS Object storage
Branch
master
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 4724
Build 6295: tox-on-jenkinsJenkins
Build 6294: arc lint + arc unit

Event Timeline

vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/objstorage/cloud/objstorage_cloud.py
185–186

Do we really need this?

This revision is now accepted and ready to land.Mar 19 2019, 1:44 PM
douardda added inline comments.
swh/objstorage/cloud/objstorage_cloud.py
185–186

easy hack to prevent from iterating a bytes one byte at a time.

Note that this implementation is a bit naive on the get side: we do uncompress the whole object.

Actually, this whole compression support should be discussed a bit more. This Diff is a stopgap

updated, remove smart stuff and add some tests

'smart' as 'do not compress if the blob is too small'...

This revision was automatically updated to reflect the committed changes.