Page MenuHomeSoftware Heritage

Improve cli and add 3 commands
ClosedPublic

Authored by douardda on Jan 8 2019, 10:52 AM.

Details

Summary

This is needed to be able to add more context objects (see following revisions).

Added cli commands are runner, listener and api-server

These commands do what they say, ie. start a runner, listener or API server
process.

Note that processes are not daemonized and run in front.

Typically used as:

swh-scheduler --cls local --database postgresql:///?service=swh api-server --host 127.0.0.1 --port 5008

swh-scheduler --cls remote --url http://127.0.0.1:5008 runner --period 10

swh-scheduler --cls remote --url http://127.0.0.1:5008 listener

Move the scheduler verification from the main cli group definition to subcommands

since it's actually the responsibility of each subcommand to decide whether it
can run without a properly configured scheduler instance.

This is also required so the user can run:

swh-scheduler subcommand --help

even with a non-properly configured scheduler.

Also move logging configuration into the cli group function
so that logging level config can be consistently set for all swh-scheduler
commands.

Diff Detail

Repository
rDSCH Scheduling utilities
Branch
wip
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 3357
Build 4327: tox-on-jenkinsJenkins
Build 4326: arc lint + arc unit

Event Timeline

olasd added inline comments.
swh/scheduler/cli.py
412

typo

ardumont added inline comments.
swh/scheduler/cli.py
384

which

412

listener

swh/scheduler/celery_backend/runner.py
111–115

Do we still need this as you expose this below (except for the __import__ instructions)?

swh/scheduler/cli.py
453

Same question as before, do we still need the functions starting the server defined in server.py? [1]

[1] https://forge.softwareheritage.org/source/swh-scheduler/browse/master/swh/scheduler/api/server.py$133-157

Sounds good.

Possibly some cleanup is needed (depending on the answer of my questions ;)

Also, not an issue, this can have impacts on the deployments manifests (swh-site).

douardda added inline comments.
swh/scheduler/cli.py
453

probably not, let's remove it

douardda added inline comments.
swh/scheduler/cli.py
453

In fact, we can delete the launch function but not the run_from_webserver() one since it's the entry point for prod wsgi servers (gunicorn here)

This revision is now accepted and ready to land.Jan 10 2019, 1:07 AM
This revision was automatically updated to reflect the committed changes.