Page MenuHomeSoftware Heritage

Add a new SWHTask class to be used as base class for swh celery tasks
ClosedPublic

Authored by douardda on Jan 8 2019, 10:50 AM.

Details

Summary

It is meant to be used to declare swh tasks via the task decorator
instead of subclassing the (now deprecated) Task class.

It is typically used like:

from swh.scheduler.celery_backend.config import app
from swh.scheduler.tasks import SWHTask

@app.task(base=SWHTask)
def ping():
    return 'pong'

Also replace the TaskRouter class by a simple function
class-based router is a remainder of celery 3.

Diff Detail

Repository
rDSCH Scheduling utilities
Branch
wip
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 3269
Build 4206: tox-on-jenkinsJenkins
Build 4205: arc lint + arc unit

Event Timeline

This revision is now accepted and ready to land.Jan 8 2019, 11:50 AM

Could we make this base class the default for all tasks registered through the @app.task decorator?

This would save us some typing on (all) the task definitions, and be a bit less error-prone.

In D886#19092, @olasd wrote:

Could we make this base class the default for all tasks registered through the @app.task decorator?

This would save us some typing on (all) the task definitions, and be a bit less error-prone.

Agreed, but I haven't find how to do that. I'll dig a bit deeper...

In D886#19092, @olasd wrote:

Could we make this base class the default for all tasks registered through the @app.task decorator?

This would save us some typing on (all) the task definitions, and be a bit less error-prone.

Agreed, but I haven't find how to do that. I'll dig a bit deeper...

It might be possible to use the task_cls argument of the Celery constructor for this. Need to check how to make this compatible with celery's pytest fixtures however...

This comment was removed by olasd.
douardda retitled this revision from Add a new SHWTask class to be used as base class for swh celery tasks to Add a new SWHTask class to be used as base class for swh celery tasks.Jan 10 2019, 3:46 PM

rebased and typo in commit msg

This revision was automatically updated to reflect the committed changes.