Page MenuHomeSoftware Heritage

indexer: Issue on latest task changes
Closed, MigratedEdits Locked

Description

The new class task refactored is not entirely correct.
There are different result depending on the nature of the task (Indexer and Orchestrator diverge in their result).

Sample:

  • Orchestrator:
[2018-10-25 19:04:05,348: ERROR/MainProcess] Task swh.indexer.tasks.OrchestratorAllContents[959e9e95-62c7-408c-b084-9fac493ec1a3] raised unexpected: AttributeError("'list' object has no attribute 'results'",)
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 438, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/swh/scheduler/task.py", line 161, in run
    raise e from None
  File "/usr/lib/python3/dist-packages/swh/scheduler/task.py", line 158, in run
    result = self.run_task(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/swh/indexer/tasks.py", line 26, in run_task
    return indexer.results
AttributeError: 'list' object has no attribute 'results'
  • mimetype indexer:
[2018-10-25 19:02:24,160: ERROR/MainProcess] Task swh.indexer.tasks.ContentMimetype[d3c44a5d-e803-421d-8bef-f7f44ff46ef5] raised unexpected: AttributeError("'NoneType' object has no attribute 'results'",)
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 438, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/swh/scheduler/task.py", line 161, in run
    raise e from None
  File "/usr/lib/python3/dist-packages/swh/scheduler/task.py", line 158, in run
    result = self.run_task(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/swh/indexer/tasks.py", line 25, in run_task
    return indexer.results
AttributeError: 'NoneType' object has no attribute 'results'

dashboard: http://kibana0.internal.softwareheritage.org:5601/app/kibana#/dashboard/289ce780-d88d-11e8-b8ce-cf95f437ce37