Page MenuHomeSoftware Heritage

indexer storage crashes on kafka errors, because of integers in the key
Closed, MigratedEdits Locked

Description

https://sentry.softwareheritage.org/organizations/swh/issues/105388/?referrer=phabricator_plugin

KafkaDeliveryError: KafkaDeliveryError('flush() exceeded timeout (120s)', [DeliveryFailureInfo(object_type='content_fossology_license', key={'id': b'\x90\x80\x1c\xf8$n\xf0\xe0-\xc0\xf3_\xa2a\\\xf4\xa6\xba\x95\xe7', 'indexer_configuration_id': 511854352, 'license': 'No_license_found'}, message='No delivery before flush() timeout', code='SWH_FLUSH_TIMEOUT'), DeliveryFailureInfo(object_type='content_fossology_license', key={'id': b' \x7f\xc8\xcc^\xf9,\xf7Y\xf0A?\\\xf6\x92\xaaL\x07\xd5\x90', 'indexer_configuration_id': 51185435...
(7 additional frame(s) were not displayed)
...
  File "swh/core/db/common.py", line 73, in _meth
    return meth(self, *args, db=db, cur=cur, **kwargs)
  File "swh/indexer/storage/__init__.py", line 344, in content_fossology_license_add
    self.journal_writer.write_additions("content_fossology_license", licenses)
  File "swh/indexer/storage/writer.py", line 73, in write_additions
    self.journal.write_additions(obj_type, translated)
  File "swh/journal/writer/kafka.py", line 268, in write_additions
    self.flush()
  File "swh/journal/writer/kafka.py", line 229, in flush
    "flush() exceeded timeout (%ss)" % self.flush_timeout,

AttributeError: 'int' object has no attribute 'hex'
(7 additional frame(s) were not displayed)
...
  File "swh/journal/writer/kafka.py", line 50, in pretty_failures
    for f in self.delivery_failures
  File "swh/journal/writer/kafka.py", line 50, in <genexpr>
    for f in self.delivery_failures
  File "swh/journal/serializers.py", line 72, in pprint_key
    f"{k}: {stringify_key_item(k, v)}" for k, v in key.items()
  File "swh/journal/serializers.py", line 72, in <genexpr>
    f"{k}: {stringify_key_item(k, v)}" for k, v in key.items()
  File "swh/journal/serializers.py", line 64, in stringify_key_item
    return v.hex()