This removes duplication between swh-scheduler-listener and
swh-scheduler-runner*, and simplifies the growing complex parameters for the
runner class.
Details
Details
- Reviewers
ardumont - Group Reviewers
System administrators - Commits
- rSPSITE19f0fcc6bf91: Refactor swh scheduler services using a single unit file template
octocatalog-diff on saatchi and scheduler0:
*** Running octocatalog-diff on host saatchi.internal.softwareheritage.org
I, [2021-10-28T16:09:55.973968 #303493] INFO -- : Catalogs compiled for saatchi.internal.softwareheritage.org
I, [2021-10-28T16:09:56.292190 #303493] INFO -- : Diffs computed for saatchi.internal.softwareheritage.org
diff origin/production/saatchi.internal.softwareheritage.org current/saatchi.internal.softwareheritage.org
*******************************************
File[/etc/systemd/system/swh-scheduler-listener.service] =>
parameters =>
content =>
@@ -1,8 +1,8 @@
-# Scheduler Listener unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler celery event listener
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance listener)
# Changes will be overwritten
_
[Unit]
-Description=Software Heritage scheduler listener
+Description=Software Heritage scheduler celery event listener
Requires=network.target rabbitmq-server.service
After=network.target rabbitmq-server.service
*******************************************
File[/etc/systemd/system/swh-scheduler-runner-priority.service] =>
parameters =>
content =>
@@ -1,8 +1,8 @@
-# Scheduler Runner unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler runner for high priority tasks
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance runner-priority)
# Changes will be overwritten
_
[Unit]
-Description=Software Heritage scheduler runner
+Description=Software Heritage scheduler runner for high priority tasks
Requires=network.target rabbitmq-server.service
After=network.target rabbitmq-server.service
*******************************************
File[/etc/systemd/system/swh-scheduler-runner.service] =>
parameters =>
content =>
@@ -1,4 +1,4 @@
-# Scheduler Runner unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler runner
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance runner)
# Changes will be overwritten
_
*******************************************
Package[python3-swh.scheduler] =>
parameters =>
notify =>
+ ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]"]
*******************************************
- Profile::Swh::Deploy::Scheduler::Runner[swh-scheduler-runner-priority]
*******************************************
- Profile::Swh::Deploy::Scheduler::Runner[swh-scheduler-runner]
*******************************************
+ Profile::Swh::Deploy::Scheduler::Service[listener] =>
parameters =>
"service_args": ["start-listener"]
"service_description": "celery event listener"
"service_name": "swh-scheduler-listener"
*******************************************
+ Profile::Swh::Deploy::Scheduler::Service[runner-priority] =>
parameters =>
"service_args": ["start-runner", "--period", "10", "--with-priority", "--task-type", "load-git", "--task-type", "load-svn", "--task-type", "load-archive-files", "--task-type", "load-hg"]
"service_description": "runner for high priority tasks"
"service_name": "swh-scheduler-runner-priority"
*******************************************
+ Profile::Swh::Deploy::Scheduler::Service[runner] =>
parameters =>
"service_args": ["start-runner", "--period", "10"]
"service_description": "runner"
"service_name": "swh-scheduler-runner"
*******************************************
Service[swh-scheduler-listener] =>
parameters =>
subscribe =>
- ["Package[python3-swh.scheduler]"]
*******************************************
Service[swh-scheduler-runner-priority] =>
parameters =>
subscribe =>
- ["Package[python3-swh.scheduler]"]
*******************************************
Service[swh-scheduler-runner] =>
parameters =>
subscribe =>
- ["Package[python3-swh.scheduler]"]
*******************************************
Systemd::Unit_file[swh-scheduler-listener.service] =>
parameters =>
content =>
@@ -1,8 +1,8 @@
-# Scheduler Listener unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler celery event listener
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance listener)
# Changes will be overwritten
_
[Unit]
-Description=Software Heritage scheduler listener
+Description=Software Heritage scheduler celery event listener
Requires=network.target rabbitmq-server.service
After=network.target rabbitmq-server.service
notify =>
- Service[swh-scheduler-listener]
+ ["Service[swh-scheduler-listener]"]
*******************************************
Systemd::Unit_file[swh-scheduler-runner-priority.service] =>
parameters =>
content =>
@@ -1,8 +1,8 @@
-# Scheduler Runner unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler runner for high priority tasks
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance runner-priority)
# Changes will be overwritten
_
[Unit]
-Description=Software Heritage scheduler runner
+Description=Software Heritage scheduler runner for high priority tasks
Requires=network.target rabbitmq-server.service
After=network.target rabbitmq-server.service
notify =>
- Service[swh-scheduler-runner-priority]
+ ["Service[swh-scheduler-runner-priority]"]
*******************************************
Systemd::Unit_file[swh-scheduler-runner.service] =>
parameters =>
content =>
@@ -1,4 +1,4 @@
-# Scheduler Runner unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler runner
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance runner)
# Changes will be overwritten
_
notify =>
- Service[swh-scheduler-runner]
+ ["Service[swh-scheduler-runner]"]
*******************************************
*** End octocatalog-diff on saatchi.internal.softwareheritage.org
*** Running octocatalog-diff on host scheduler0.internal.staging.swh.network
I, [2021-10-28T16:10:00.259699 #303563] INFO -- : Catalogs compiled for scheduler0.internal.staging.swh.network
I, [2021-10-28T16:10:00.595162 #303563] INFO -- : Diffs computed for scheduler0.internal.staging.swh.network
diff origin/production/scheduler0.internal.staging.swh.network current/scheduler0.internal.staging.swh.network
*******************************************
File[/etc/systemd/system/swh-scheduler-listener.service] =>
parameters =>
content =>
@@ -1,8 +1,8 @@
-# Scheduler Listener unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler celery event listener
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance listener)
# Changes will be overwritten
_
[Unit]
-Description=Software Heritage scheduler listener
+Description=Software Heritage scheduler celery event listener
Requires=network.target rabbitmq-server.service
After=network.target rabbitmq-server.service
*******************************************
File[/etc/systemd/system/swh-scheduler-runner-priority.service] =>
parameters =>
content =>
@@ -1,8 +1,8 @@
-# Scheduler Runner unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler runner for high priority tasks
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance runner-priority)
# Changes will be overwritten
_
[Unit]
-Description=Software Heritage scheduler runner
+Description=Software Heritage scheduler runner for high priority tasks
Requires=network.target rabbitmq-server.service
After=network.target rabbitmq-server.service
*******************************************
File[/etc/systemd/system/swh-scheduler-runner.service] =>
parameters =>
content =>
@@ -1,4 +1,4 @@
-# Scheduler Runner unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler runner
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance runner)
# Changes will be overwritten
_
*******************************************
Package[python3-swh.scheduler] =>
parameters =>
notify =>
+ ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]"]
*******************************************
- Profile::Swh::Deploy::Scheduler::Runner[swh-scheduler-runner-priority]
*******************************************
- Profile::Swh::Deploy::Scheduler::Runner[swh-scheduler-runner]
*******************************************
+ Profile::Swh::Deploy::Scheduler::Service[listener] =>
parameters =>
"service_args": ["start-listener"]
"service_description": "celery event listener"
"service_name": "swh-scheduler-listener"
*******************************************
+ Profile::Swh::Deploy::Scheduler::Service[runner-priority] =>
parameters =>
"service_args": ["start-runner", "--period", "10", "--with-priority", "--task-type", "load-git", "--task-type", "load-svn", "--task-type", "load-archive-files", "--task-type", "load-hg"]
"service_description": "runner for high priority tasks"
"service_name": "swh-scheduler-runner-priority"
*******************************************
+ Profile::Swh::Deploy::Scheduler::Service[runner] =>
parameters =>
"service_args": ["start-runner", "--period", "10"]
"service_description": "runner"
"service_name": "swh-scheduler-runner"
*******************************************
Service[swh-scheduler-listener] =>
parameters =>
subscribe =>
- ["Package[python3-swh.scheduler]"]
*******************************************
Service[swh-scheduler-runner-priority] =>
parameters =>
subscribe =>
- ["Package[python3-swh.scheduler]"]
*******************************************
Service[swh-scheduler-runner] =>
parameters =>
subscribe =>
- ["Package[python3-swh.scheduler]"]
*******************************************
Systemd::Unit_file[swh-scheduler-listener.service] =>
parameters =>
content =>
@@ -1,8 +1,8 @@
-# Scheduler Listener unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler celery event listener
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance listener)
# Changes will be overwritten
_
[Unit]
-Description=Software Heritage scheduler listener
+Description=Software Heritage scheduler celery event listener
Requires=network.target rabbitmq-server.service
After=network.target rabbitmq-server.service
notify =>
- Service[swh-scheduler-listener]
+ ["Service[swh-scheduler-listener]"]
*******************************************
Systemd::Unit_file[swh-scheduler-runner-priority.service] =>
parameters =>
content =>
@@ -1,8 +1,8 @@
-# Scheduler Runner unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler runner for high priority tasks
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance runner-priority)
# Changes will be overwritten
_
[Unit]
-Description=Software Heritage scheduler runner
+Description=Software Heritage scheduler runner for high priority tasks
Requires=network.target rabbitmq-server.service
After=network.target rabbitmq-server.service
notify =>
- Service[swh-scheduler-runner-priority]
+ ["Service[swh-scheduler-runner-priority]"]
*******************************************
Systemd::Unit_file[swh-scheduler-runner.service] =>
parameters =>
content =>
@@ -1,4 +1,4 @@
-# Scheduler Runner unit file
-# Managed by puppet class profile::swh::deploy::scheduler
+# Scheduler runner
+# Managed by puppet class profile::swh::deploy::scheduler::service (instance runner)
# Changes will be overwritten
_
notify =>
- Service[swh-scheduler-runner]
+ ["Service[swh-scheduler-runner]"]
*******************************************
*** End octocatalog-diff on scheduler0.internal.staging.swh.networkIn practice, only comments are changed in unit files.
Diff Detail
Diff Detail
- Repository
- rSPSITE puppet-swh-site
- Branch
- scheduler-refactoring
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 24806 Build 38733: arc lint + arc unit