Page MenuHomeSoftware Heritage

Use msgpack extension types instead of custom swh encoders/decoders
Closed, MigratedEdits Locked

Description

(At least in the context of the journal)
We currently use custom encoders for a few object types when serializing swh objects in msgpack: we currently use the encoding schema:

{
 b'swhtype': '<type>',
 b'd': <payload>
}

while the msgpack format allows custom extension types (which has recently been used to prevent integer overflows in msgpack dump/load).

We should make use of these extension types instead, it would make more sense, be more consistent and make the documentation of the kafka messages (T2818) easier.

Related Objects