Page MenuHomeSoftware Heritage

swh.scheduler.celery_backend.config: Permit task bytes serialization
AbandonedPublic

Authored by ardumont on Mar 14 2018, 12:38 PM.

Details

Reviewers
olasd
Group Reviewers
Reviewers

Diff Detail

Repository
rDSCH Scheduling utilities
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 1183
Build 1527: arc lint + arc unit

Event Timeline

swh/scheduler/celery_backend/config.py
23

Simple wrapper around our json swh (en|de)coder

def swh_json_dumps(obj):
    """Specific swh json encoder"""
    return json.dumps(obj, cls=SWHJSONEncoder)


def swh_json_loads(obj):
    """Specific swh json decoder"""
    return json.loads(obj, cls=SWHJSONDecoder)

Discussion took place in irc:

13:17:49     +olasd | no, that's a very bad idea
13:18:03     +olasd | just set the serializer to msgpack or pickle
13:18:51  +ardumont | why is that a bad idea?
13:19:25     +olasd | we should stop piling hacks on top of hacks and use built-in things that work
13:19:42  +ardumont | i don't get why that is a hack
13:25:09     +olasd | it adds a non-default serialization format that nobody else in the world supports, when there are perfectly fine alternatives that work and are supported upstream
13:25:38     +seirl | ardumont: trying to shove in binary data in json is pretty much always a bad idea
13:26:04  +ardumont | well, yeah but i'm reusing stuff we use anyway
13:26:31  +ardumont | ok, as we are supposed to be away from pickle, i'm taking msgpack then
13:26:56     +seirl | (Y)
13:28:23     +olasd | yes please