Page MenuHomeSoftware Heritage

Replaying REMD topics is broken
Closed, ResolvedPublic

Description

Currently, REMD topics cannot be replayed because of errors like:

swh-graph-replayer_8          | Traceback (most recent call last):
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/bin/swh", line 8, in <module>
swh-graph-replayer_8          |     sys.exit(main())
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/cli/__init__.py", line 185, in main
swh-graph-replayer_8          |     return swh(auto_envvar_prefix="SWH")
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
swh-graph-replayer_8          |     return self.main(*args, **kwargs)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
swh-graph-replayer_8          |     rv = self.invoke(ctx)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
swh-graph-replayer_8          |     return _process_result(sub_ctx.command.invoke(sub_ctx))
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
swh-graph-replayer_8          |     return _process_result(sub_ctx.command.invoke(sub_ctx))
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
swh-graph-replayer_8          |     return ctx.invoke(self.callback, **ctx.params)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
swh-graph-replayer_8          |     return callback(*args, **kwargs)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
swh-graph-replayer_8          |     return f(get_current_context(), *args, **kwargs)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/cli.py", line 194, in replay
swh-graph-replayer_8          |     client.process(worker_fn)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/journal/client.py", line 265, in process
swh-graph-replayer_8          |     batch_processed, at_eof = self.handle_messages(messages, worker_fn)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/journal/client.py", line 292, in handle_messages
swh-graph-replayer_8          |     worker_fn(dict(objects))
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/replay.py", line 63, in process_replay_objects
swh-graph-replayer_8          |     _insert_objects(object_type, objects, storage)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/replay.py", line 157, in _insert_objects
swh-graph-replayer_8          |     storage.metadata_authority_add(authorities)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/metrics.py", line 24, in d
swh-graph-replayer_8          |     return f(*a, **kw)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/metrics.py", line 77, in d
swh-graph-replayer_8          |     r = f(*a, **kw)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/db/common.py", line 62, in _meth
swh-graph-replayer_8          |     return meth(self, *args, db=db, cur=cur, **kwargs)
swh-graph-replayer_8          |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/postgresql/storage.py", line 1440, in metadata_authority_add
swh-graph-replayer_8          |     "MetadataAuthority.metadata may not be None in "
swh-graph-replayer_8          | swh.storage.exc.StorageArgumentException: MetadataAuthority.metadata may not be None in metadata_authority_add.
swh-graph-replayer_8          | ERROR:swh.storage.replay:Failed to insert raw_extrinsic_metadata objects