Page MenuHomeSoftware Heritage

Enrich task return status to register partial completions
Closed, ResolvedPublic

Description

The return status of tasks is currently a three-state :

  • Exception => the task failed
  • True => the task succeeded and did something
  • False => the task succeeded and did nothing

We need to extend the return values to allow tasks to return a partial state.

One possible return value could be a dict with multiple values :

  • completion: full/partial
  • eventful: True/False
  • state: variable to pass to the next partial task run
  • delay: the delay before reexecution of a partial loading task

Event Timeline

We can also generalize this to improve behavior on transient, "expected" failures as well:

  • completion: full-eventful/full-uneventful/partial/transient-failure
  • state: to be passed on to the next task run
  • delay: to allow overriding the default retry delay