Page MenuHomeSoftware Heritage

Add a new SWHTask class to be used as base class for swh celery tasks
ClosedPublic

Authored by douardda on Jan 8 2019, 10:50 AM.

Details

Summary

It is meant to be used to declare swh tasks via the task decorator
instead of subclassing the (now deprecated) Task class.

It is typically used like:

from swh.scheduler.celery_backend.config import app
from swh.scheduler.tasks import SWHTask

@app.task(base=SWHTask)
def ping():
    return 'pong'

Also replace the TaskRouter class by a simple function
class-based router is a remainder of celery 3.

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

douardda created this revision.Jan 8 2019, 10:50 AM
ardumont accepted this revision.Jan 8 2019, 11:50 AM
This revision is now accepted and ready to land.Jan 8 2019, 11:50 AM
olasd added a subscriber: olasd.Jan 8 2019, 1:37 PM

Could we make this base class the default for all tasks registered through the @app.task decorator?

This would save us some typing on (all) the task definitions, and be a bit less error-prone.

In D886#19092, @olasd wrote:

Could we make this base class the default for all tasks registered through the @app.task decorator?

This would save us some typing on (all) the task definitions, and be a bit less error-prone.

Agreed, but I haven't find how to do that. I'll dig a bit deeper...

In D886#19092, @olasd wrote:

Could we make this base class the default for all tasks registered through the @app.task decorator?

This would save us some typing on (all) the task definitions, and be a bit less error-prone.

Agreed, but I haven't find how to do that. I'll dig a bit deeper...

It might be possible to use the task_cls argument of the Celery constructor for this. Need to check how to make this compatible with celery's pytest fixtures however...

olasd added a comment.Jan 9 2019, 5:47 PM
This comment was removed by olasd.
douardda retitled this revision from Add a new SHWTask class to be used as base class for swh celery tasks to Add a new SWHTask class to be used as base class for swh celery tasks.Jan 10 2019, 3:46 PM
douardda updated this revision to Diff 2882.Jan 10 2019, 3:47 PM

rebased and typo in commit msg

This revision was automatically updated to reflect the committed changes.