HomeSoftware Heritage

Make the replayer not crash on kafka messages that fail to be converted as…

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

Make the replayer not crash on kafka messages that fail to be converted as model objects

for example, there are a few kafka directory messages in the current
production kafka cluster which entries contain the same name twice,
preventing the Directory model object from being created at all,
which makes the replayer crash.

This change makes the replayer able to handle such cases. When the model
object creation fails with a ValueError, the error is reported in the
(redis) error reporter, but the replaying process continue.

Since there is no model object, the error is reported with a crafted
error key of the form "{object_type}:{object_id}" if an object id is
present in the data structure, or "{object_type}:uuid:{uuid4}" if such
an id is not even present. For the record, the standard error key in
redis for a model object is it's swhid (if any).

Details

Commit No Longer Exists

This commit no longer exists in the repository.