Page MenuHomeSoftware Heritage

Add a new SWHTask class to be used as base class for swh celery tasks
ClosedPublic

Authored by douardda on Tue, Jan 8, 10:50 AM.

Details

Summary

It is meant to be used to declare swh tasks via the task decorator
instead of subclassing the (now deprecated) Task class.

It is typically used like:

from swh.scheduler.celery_backend.config import app
from swh.scheduler.tasks import SWHTask

@app.task(base=SWHTask)
def ping():
    return 'pong'

Also replace the TaskRouter class by a simple function
class-based router is a remainder of celery 3.

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

douardda created this revision.Tue, Jan 8, 10:50 AM
ardumont accepted this revision.Tue, Jan 8, 11:50 AM
This revision is now accepted and ready to land.Tue, Jan 8, 11:50 AM
olasd added a subscriber: olasd.Tue, Jan 8, 1:37 PM

Could we make this base class the default for all tasks registered through the @app.task decorator?

This would save us some typing on (all) the task definitions, and be a bit less error-prone.

In D886#19092, @olasd wrote:

Could we make this base class the default for all tasks registered through the @app.task decorator?

This would save us some typing on (all) the task definitions, and be a bit less error-prone.

Agreed, but I haven't find how to do that. I'll dig a bit deeper...

In D886#19092, @olasd wrote:

Could we make this base class the default for all tasks registered through the @app.task decorator?

This would save us some typing on (all) the task definitions, and be a bit less error-prone.

Agreed, but I haven't find how to do that. I'll dig a bit deeper...

It might be possible to use the task_cls argument of the Celery constructor for this. Need to check how to make this compatible with celery's pytest fixtures however...

olasd added a comment.Wed, Jan 9, 5:47 PM
This comment was removed by olasd.
douardda retitled this revision from Add a new SHWTask class to be used as base class for swh celery tasks to Add a new SWHTask class to be used as base class for swh celery tasks.Thu, Jan 10, 3:46 PM
douardda updated this revision to Diff 2882.Thu, Jan 10, 3:47 PM

rebased and typo in commit msg

This revision was automatically updated to reflect the committed changes.