Page MenuHomeSoftware Heritage

Improve cli and add 3 commands
ClosedPublic

Authored by douardda on Tue, Jan 8, 10:52 AM.

Details

Summary

This is needed to be able to add more context objects (see following revisions).

Added cli commands are runner, listener and api-server

These commands do what they say, ie. start a runner, listener or API server
process.

Note that processes are not daemonized and run in front.

Typically used as:

swh-scheduler --cls local --database postgresql:///?service=swh api-server --host 127.0.0.1 --port 5008

swh-scheduler --cls remote --url http://127.0.0.1:5008 runner --period 10

swh-scheduler --cls remote --url http://127.0.0.1:5008 listener

Move the scheduler verification from the main cli group definition to subcommands

since it's actually the responsibility of each subcommand to decide whether it
can run without a properly configured scheduler instance.

This is also required so the user can run:

swh-scheduler subcommand --help

even with a non-properly configured scheduler.

Also move logging configuration into the cli group function
so that logging level config can be consistently set for all swh-scheduler
commands.

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

douardda created this revision.Tue, Jan 8, 10:52 AM
olasd added a subscriber: olasd.Tue, Jan 8, 11:37 AM
olasd added inline comments.
swh/scheduler/cli.py
387

typo

ardumont added inline comments.
swh/scheduler/cli.py
360

which

387

listener

ardumont added inline comments.Tue, Jan 8, 11:38 AM
swh/scheduler/celery_backend/runner.py
111 ↗(On Diff #2820)

Do we still need this as you expose this below (except for the __import__ instructions)?

ardumont added inline comments.Tue, Jan 8, 11:43 AM
swh/scheduler/cli.py
428

Same question as before, do we still need the functions starting the server defined in server.py? [1]

[1] https://forge.softwareheritage.org/source/swh-scheduler/browse/master/swh/scheduler/api/server.py$133-157

Sounds good.

Possibly some cleanup is needed (depending on the answer of my questions ;)

Also, not an issue, this can have impacts on the deployments manifests (swh-site).

douardda marked an inline comment as done.Wed, Jan 9, 4:58 PM
douardda added inline comments.
swh/scheduler/cli.py
428

probably not, let's remove it

douardda updated this revision to Diff 2854.Wed, Jan 9, 5:05 PM

fix reported typo

douardda marked an inline comment as done.Wed, Jan 9, 5:22 PM
douardda added inline comments.
swh/scheduler/cli.py
428

In fact, we can delete the launch function but not the run_from_webserver() one since it's the entry point for prod wsgi servers (gunicorn here)

ardumont accepted this revision.Thu, Jan 10, 1:07 AM
This revision is now accepted and ready to land.Thu, Jan 10, 1:07 AM
This revision was automatically updated to reflect the committed changes.