Page MenuHomeSoftware Heritage

Install the scheduler-runner-priority in scheduler nodes
ClosedPublic

Authored by ardumont on Jun 11 2021, 11:21 AM.

Details

Summary

This got already deployed in production manually.

Related to T3367

Test Plan
$ bin/octocatalog-diff --to staging saatchi 2>&1
*******************************************
  File[/etc/softwareheritage/scheduler/listener-runner.yml] =>
   parameters =>
     notify =>
      - ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner]"]
      + ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]"]
*******************************************
- File[/etc/systemd/system/swh-scheduler-runner.service]
*******************************************
+ File[/etc/systemd/system/swh-scheduler-swh-scheduler-runner-priority.service] =>
   parameters =>
     "content": "# Scheduler Runner unit file\n# Managed by puppet class profile:...
     "ensure": "file",
     "group": "root",
     "mode": "0444",
     "notify": "Class[Systemd::Systemctl::Daemon_reload]",
     "owner": "root",
     "show_diff": true
*******************************************
+ File[/etc/systemd/system/swh-scheduler-swh-scheduler-runner.service] =>
   parameters =>
     "content": "# Scheduler Runner unit file\n# Managed by puppet class profile:...
     "ensure": "file",
     "group": "root",
     "mode": "0444",
     "notify": "Class[Systemd::Systemctl::Daemon_reload]",
     "owner": "root",
     "show_diff": true
*******************************************
+ Profile::Swh::Deploy::Scheduler::Runner[swh-scheduler-runner-priority] =>
   parameters =>
     "config_file": "/etc/softwareheritage/scheduler/listener-runner.yml",
     "group": "swhscheduler",
     "packages": [
       "python3-swh.scheduler"
     ],
     "priority": true,
     "sentry_dsn": "https://swh::deploy::scheduler::sentry_token@sentry.softwareh...
     "sentry_environment": "production",
     "sentry_swh_package": "swh.scheduler",
     "service_name": "swh-scheduler-runner-priority",
     "user": "swhscheduler"
*******************************************
+ Profile::Swh::Deploy::Scheduler::Runner[swh-scheduler-runner] =>
   parameters =>
     "config_file": "/etc/softwareheritage/scheduler/listener-runner.yml",
     "group": "swhscheduler",
     "packages": [
       "python3-swh.scheduler"
     ],
     "priority": false,
     "sentry_dsn": "https://swh::deploy::scheduler::sentry_token@sentry.softwareh...
     "sentry_environment": "production",
     "sentry_swh_package": "swh.scheduler",
     "service_name": "swh-scheduler-runner",
     "user": "swhscheduler"
*******************************************
+ Service[swh-scheduler-runner-priority] =>
   parameters =>
     "enable": true,
     "ensure": "running",
     "subscribe": [
       "Package[python3-swh.scheduler]"
     ]
*******************************************
- Systemd::Unit_file[swh-scheduler-runner.service]
*******************************************
+ Systemd::Unit_file[swh-scheduler-swh-scheduler-runner-priority.service] =>
   parameters =>
     "content": "# Scheduler Runner unit file\n# Managed by puppet class profile:...
     "ensure": "present",
     "group": "root",
     "mode": "0444",
     "notify": "Service[swh-scheduler-runner-priority]",
     "owner": "root",
     "path": "/etc/systemd/system",
     "show_diff": true
*******************************************
+ Systemd::Unit_file[swh-scheduler-swh-scheduler-runner.service] =>
   parameters =>
     "content": "# Scheduler Runner unit file\n# Managed by puppet class profile:...
     "ensure": "present",
     "group": "root",
     "mode": "0444",
     "notify": "Service[swh-scheduler-runner]",
     "owner": "root",
     "path": "/etc/systemd/system",
     "show_diff": true
*******************************************
*** End octocatalog-diff on saatchi.internal.softwareheritage.org
$ bin/octocatalog-diff --to staging scheduler0.internal.staging.swh.network
*******************************************
  File[/etc/softwareheritage/scheduler/listener-runner.yml] =>
   parameters =>
     notify =>
      - ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner]"]
      + ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]"]
*******************************************
- File[/etc/systemd/system/swh-scheduler-runner.service]
*******************************************
+ File[/etc/systemd/system/swh-scheduler-swh-scheduler-runner-priority.service] =>
   parameters =>
     "content": "# Scheduler Runner unit file\n# Managed by puppet class profile:...
     "ensure": "file",
     "group": "root",
     "mode": "0444",
     "notify": "Class[Systemd::Systemctl::Daemon_reload]",
     "owner": "root",
     "show_diff": true
*******************************************
+ File[/etc/systemd/system/swh-scheduler-swh-scheduler-runner.service] =>
   parameters =>
     "content": "# Scheduler Runner unit file\n# Managed by puppet class profile:...
     "ensure": "file",
     "group": "root",
     "mode": "0444",
     "notify": "Class[Systemd::Systemctl::Daemon_reload]",
     "owner": "root",
     "show_diff": true
*******************************************
+ Profile::Swh::Deploy::Scheduler::Runner[swh-scheduler-runner-priority] =>
   parameters =>
     "config_file": "/etc/softwareheritage/scheduler/listener-runner.yml",
     "group": "swhscheduler",
     "packages": [
       "python3-swh.scheduler"
     ],
     "priority": true,
     "sentry_dsn": "https://swh::deploy::scheduler::sentry_token@sentry.softwareh...
     "sentry_environment": "staging",
     "sentry_swh_package": "swh.scheduler",
     "service_name": "swh-scheduler-runner-priority",
     "user": "swhscheduler"
*******************************************
+ Profile::Swh::Deploy::Scheduler::Runner[swh-scheduler-runner] =>
   parameters =>
     "config_file": "/etc/softwareheritage/scheduler/listener-runner.yml",
     "group": "swhscheduler",
     "packages": [
       "python3-swh.scheduler"
     ],
     "priority": false,
     "sentry_dsn": "https://swh::deploy::scheduler::sentry_token@sentry.softwareh...
     "sentry_environment": "staging",
     "sentry_swh_package": "swh.scheduler",
     "service_name": "swh-scheduler-runner",
     "user": "swhscheduler"
*******************************************
+ Service[swh-scheduler-runner-priority] =>
   parameters =>
     "enable": true,
     "ensure": "running",
     "subscribe": [
       "Package[python3-swh.scheduler]"
     ]
*******************************************
- Systemd::Unit_file[swh-scheduler-runner.service]
*******************************************
+ Systemd::Unit_file[swh-scheduler-swh-scheduler-runner-priority.service] =>
   parameters =>
     "content": "# Scheduler Runner unit file\n# Managed by puppet class profile:...
     "ensure": "present",
     "group": "root",
     "mode": "0444",
     "notify": "Service[swh-scheduler-runner-priority]",
     "owner": "root",
     "path": "/etc/systemd/system",
     "show_diff": true
*******************************************
+ Systemd::Unit_file[swh-scheduler-swh-scheduler-runner.service] =>
   parameters =>
     "content": "# Scheduler Runner unit file\n# Managed by puppet class profile:...
     "ensure": "present",
     "group": "root",
     "mode": "0444",
     "notify": "Service[swh-scheduler-runner]",
     "owner": "root",
     "path": "/etc/systemd/system",
     "show_diff": true
*******************************************
*** End octocatalog-diff on scheduler0.internal.staging.swh.network

Diff Detail

Repository
rSPSITE puppet-swh-site
Branch
staging
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 21928
Build 34104: arc lint + arc unit

Event Timeline

ardumont created this revision.
site-modules/profile/manifests/swh/deploy/scheduler/runner.pp
42

See below to determine how the command is computed in the end (per runner).

runner (as the current one):

/usr/bin/swh --log-level INFO scheduler --config-file /etc/softwareheritage/scheduler/listener-runner.yml start-runner --period 10

runner-priority:

/usr/bin/swh --log-level INFO scheduler --config-file /etc/softwareheritage/scheduler/listener-runner.yml start-runner --period 10 --with-priority --task-type load-git --task-type load-svn --task-type load-archive-files --task-type load-hg
olasd added a subscriber: olasd.
olasd added inline comments.
site-modules/profile/manifests/swh/deploy/scheduler.pp
42–43

This could go away

site-modules/profile/manifests/swh/deploy/scheduler/runner.pp
64

Not sure that's useful, as puppet doesn't upgrade the packages (afaik)

This revision is now accepted and ready to land.Jun 11 2021, 11:26 AM
  • scheduler: Drop unneeded up instructions
site-modules/profile/manifests/swh/deploy/scheduler.pp
42–43

right!

Build is green

Patch application report for D5854 (id=20946)

Rebasing onto c7707b5c83...

First, rewinding head to replay your work on top of it...
Fast-forwarded diff-target to base-revision-364-D5854.
Changes applied before test

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/364/ for more details.