Maniphest T2094

KeyError: 'content:add' in swh.loader.core.loader
Closed, MigratedEdits Locked
Actions

Assigned To

Authored By

	robguinness
	Nov 15 2019, 6:34 PM

Description

When submitting a "Create save request" on a git repository through the web app, the task fails with a KeyError in swh.loader.core.loader. The log and stacktrace is as follows:

2019-11-15 17:14:12,497: INFO/MainProcess] Received task: swh.loader.git.tasks.UpdateGitRepository[c1670151-ed4a-4544-a841-8ecf595ec5cb]  
[2019-11-15 17:14:12,502: INFO/ForkPoolWorker-1] Loading config file /loader.yml
[2019-11-15 17:14:12,519: DEBUG/ForkPoolWorker-1] Creating origin for https://github.com/OpenDiablo2/OpenDiablo2.git
[2019-11-15 17:14:12,545: DEBUG/ForkPoolWorker-1] Done creating origin for https://github.com/OpenDiablo2/OpenDiablo2.git
[2019-11-15 17:14:12,545: DEBUG/ForkPoolWorker-1] Creating origin_visit for origin https://github.com/OpenDiablo2/OpenDiablo2.git at time 2019-11-15 17:14:12.519239+00:00
[2019-11-15 17:14:12,556: DEBUG/ForkPoolWorker-1] Done Creating git origin_visit for origin https://github.com/OpenDiablo2/OpenDiablo2.git at time 2019-11-15 17:14:12.519239+00:00
Enumerating objects: 58, done.
Counting objects: 100% (58/58), done.
Compressing objects: 100% (45/45), done.
Total 5824 (delta 26), reused 36 (delta 13), pack-reused 5766
[2019-11-15 17:14:14,881: INFO/ForkPoolWorker-1] Listed 119 refs for repo https://github.com/OpenDiablo2/OpenDiablo2.git
[2019-11-15 17:14:20,658: DEBUG/ForkPoolWorker-1] Sending 2746 contents
[2019-11-15 17:14:21,355: ERROR/ForkPoolWorker-1] Loading failure, updating to `partial` status
Traceback (most recent call last):
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/loader/core/loader.py", line 870, in load
    self.store_data()
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/loader/core/loader.py", line 969, in store_data
    self.send_batch_contents(self.get_contents())
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/loader/core/loader.py", line 652, in send_batch_contents
    packet_size_bytes=packet_size_bytes)
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/loader/core/loader.py", line 48, in send_in_packets
    sender(formatted_objects)
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/six.py", line 696, in reraise
    raise value
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/loader/core/loader.py", line 404, in send_contents
    self.counters['contents'] += result['content:add']
KeyError: 'content:add'
[2019-11-15 17:14:21,357: DEBUG/ForkPoolWorker-1] Updating origin_visit for origin https://github.com/OpenDiablo2/OpenDiablo2.git with status partial
[2019-11-15 17:14:21,366: DEBUG/ForkPoolWorker-1] Done updating origin_visit for origin https://github.com/OpenDiablo2/OpenDiablo2.git with status partial

The environment is a freshly built docker swh/stack image built off the master branch.