Page MenuHomeSoftware Heritage
Create Task
Maniphest T2239

storage: kafka issue: Can't pickle <class 'cimpl.KafkaException'>: import of module 'cimpl' failed
Closed, MigratedEdits Locked
Actions

Description

All _add entrypoints raise that issue at some point or another (not limited to but see [1] [2] [3] for example).

We need to find a way to circumvent this (that's noisy).

Either:

  • trap this exception and raise a wrapped swh one
  • stop using pickle as serialization?
  • other ideas?

[1] https://sentry.softwareheritage.org/share/issue/2a7f2df8dac3494fae5da930e27e4774/

[2] https://sentry.softwareheritage.org/share/issue/9bc7f0a095194c07b2b5a9f364c08cea/

[3] https://sentry.softwareheritage.org/share/issue/11e3f5a116134f469751321aab682775/

Revisions and Commits

rDCORE Foundations and core functionalities
ClosedD2627 In case of errors, return a simple dictionary instead of pickled exception.

Event Timeline

ardumont triaged this task as Normal priority.Jan 21 2020, 11:36 AM
ardumont created this task.
ardumont added a project: Storage manager.
ardumont updated the task description. (Show Details)
vlorentz added a subscriber: vlorentz.Feb 5 2020, 4:17 PM

Relatedly, errors raised by tenacity cannot be pickled because they contain a Lock: https://github.com/jd/tenacity/issues/147

vlorentz claimed this task.Feb 5 2020, 4:18 PM
vlorentz added a revision: D2627: In case of errors, return a simple dictionary instead of pickled exception..Feb 5 2020, 5:12 PM
vlorentz added a comment.Feb 17 2020, 12:14 PM

python-cassandra too has some un-unpicklable errors:

Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:   File "/home/vlorentz/swh-journal/swh/journal/replay.py", line 240, in _insert_objects
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:     storage.content_add_metadata([object_])
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:   File "/home/vlorentz/.local/lib/python3.7/site-packages/swh.core-0.0.86-py3.7.egg/swh/core/api/__init__.py", line 148, in meth_
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:     return self.post(meth._endpoint_path, post_data)
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:   File "/home/vlorentz/.local/lib/python3.7/site-packages/swh.core-0.0.86-py3.7.egg/swh/core/api/__init__.py", line 220, in post
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:     return self._decode_response(response)
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:   File "/home/vlorentz/.local/lib/python3.7/site-packages/swh.core-0.0.86-py3.7.egg/swh/core/api/__init__.py", line 283, in _decode_response
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:     self.raise_for_status(response)
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:   File "/home/vlorentz/.local/lib/python3.7/site-packages/swh.core-0.0.86-py3.7.egg/swh/core/api/__init__.py", line 258, in raise_for_status
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:     exception = pickle.loads(data)
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:   File "/usr/lib/python3/dist-packages/cassandra/__init__.py", line 472, in __init__
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:     Timeout.__init__(self, message, **kwargs)
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:   File "/usr/lib/python3/dist-packages/cassandra/__init__.py", line 423, in __init__
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]:     kwargs["write_type"] = WriteType.value_to_name[kwargs["write_type"]]
Feb 17 11:11:59 cassandra-replay01 replayer-12305[49281]: KeyError: None
olasd added a subscriber: olasd.Feb 25 2020, 11:16 AM

@vlorentz I guess this could be closed?

ardumont closed this task as Resolved.Feb 26 2020, 10:48 AM

I think so (latest storage version deployed).

Plus, checking the last occurences of those errors referenced in sentry, the most recent date back from 18/02/2020).

Closing.

And thanks ;)

gitlab-migration changed the task status from Resolved to Migrated.Jan 8 2023, 4:29 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal