Page MenuHomeSoftware Heritage

Install save code now update routine as a service called every minute
ClosedPublic

Authored by ardumont on Aug 3 2021, 5:38 PM.

Details

Summary

As an implementation detail, the service is called every minute through a systemd timer.
This means that if a timer is already still running after 1 minute, this won't start and
avoid overlaping updates altogether.

Related to T3451

Test Plan

bin/octocatalog on various webapp nodes: webapp1, moma, webapp0.staging

Expectedly, this installs:

  • the timer and service routine to update regularly save code now statuses
  • drops the old script shell triggered regularly by the cron when installed (webapp1 has no cron installed).
  • (on moma and webapp0.staging) this drops the old cron
bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to staging moma
Found host moma.softwareheritage.org
WARN     -> Environment "wip-pg-hba-rules-in-yaml" contained non-word characters, correcting name to wip_pg_hba_rules_in_yaml
Cloning into '/tmp/swh-ocd.HEtxBgfy/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.HEtxBgfy/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host moma.softwareheritage.org
I, [2021-08-03T17:35:26.555892 #24787]  INFO -- : Catalogs compiled for moma.softwareheritage.org
I, [2021-08-03T17:35:30.660001 #24787]  INFO -- : Diffs computed for moma.softwareheritage.org
diff origin/production/moma.softwareheritage.org current/moma.softwareheritage.org
*******************************************
- Concat_file[profile::cron::refresh-savecodenow-statuses]
*******************************************
- Concat_fragment[profile::cron::refresh-savecodenow-statuses::_header]
*******************************************
- Concat_fragment[profile::cron::refresh-savecodenow-statuses]
*******************************************
- File[/etc/cron.d/puppet-refresh-savecodenow-statuses]
*******************************************
+ File[/etc/systemd/system/swh-webapp-update-savecodenow-statuses.service] =>
   parameters =>
      "ensure": "file"
      "group": "root"
      "mode": "0444"
      "notify": "Class[Systemd::Systemctl::Daemon_reload]"
      "owner": "root"
      "show_diff": true
      "content": >>>
# Webapp Update SaveCodeNow status unit file
# Managed by puppet class profile::swh::deploy::webapp
# Changes will be overwritten

[Unit]
Description=Software Heritage Webapp Update SaveCodeNow Statuses
Requires=network.target
After=network.target

[Service]
User=swhwebapp
Group=swhwebapp
Environment=DJANGO_SETTINGS_MODULE=swh.web.settings.production
Type=simple
ExecStart=/usr/bin/django-admin refresh_savecodenow_statuses

[Install]
WantedBy=multi-user.target
<<<
*******************************************
+ File[/etc/systemd/system/swh-webapp-update-savecodenow-statuses.timer] =>
   parameters =>
      "ensure": "file"
      "group": "root"
      "mode": "0444"
      "notify": ["Class[Systemd::Systemctl::Daemon_reload]", "Service[swh-webapp-update-savecodenow-statuses.timer]"]
      "owner": "root"
      "show_diff": true
      "content": >>>
[Install]
WantedBy=timers.target

[Timer]
OnCalendar=*:0/1
Unit=swh-webapp-update-savecodenow-statuses.service

[Unit]
Description=regular webapp update savecodenow statuses
<<<
*******************************************
  File[/usr/local/bin/refresh-savecodenow-statuses] =>
   parameters =>
     ensure =>
      - present
      + absent
*******************************************
- Profile::Cron::D[refresh-savecodenow-statuses]
*******************************************
- Profile::Cron::File[refresh-savecodenow-statuses]
*******************************************
+ Service[swh-webapp-update-savecodenow-statuses.timer] =>
   parameters =>
      "enable": true
      "ensure": true
      "provider": "systemd"
*******************************************
+ Systemd::Timer[swh-webapp-update-savecodenow-statuses.timer] =>
   parameters =>
      "active": true
      "enable": true
      "ensure": "present"
      "group": "root"
      "mode": "0444"
      "owner": "root"
      "path": "/etc/systemd/system"
      "show_diff": true
      "service_content": >>>
# Webapp Update SaveCodeNow status unit file
# Managed by puppet class profile::swh::deploy::webapp
# Changes will be overwritten

[Unit]
Description=Software Heritage Webapp Update SaveCodeNow Statuses
Requires=network.target
After=network.target

[Service]
User=swhwebapp
Group=swhwebapp
Environment=DJANGO_SETTINGS_MODULE=swh.web.settings.production
Type=simple
ExecStart=/usr/bin/django-admin refresh_savecodenow_statuses

[Install]
WantedBy=multi-user.target
<<<
      "timer_content": >>>
[Install]
WantedBy=timers.target

[Timer]
OnCalendar=*:0/1
Unit=swh-webapp-update-savecodenow-statuses.service

[Unit]
Description=regular webapp update savecodenow statuses
<<<
*******************************************
+ Systemd::Unit_file[swh-webapp-update-savecodenow-statuses.service] =>
   parameters =>
      "ensure": "present"
      "group": "root"
      "mode": "0444"
      "owner": "root"
      "path": "/etc/systemd/system"
      "show_diff": true
      "content": >>>
# Webapp Update SaveCodeNow status unit file
# Managed by puppet class profile::swh::deploy::webapp
# Changes will be overwritten

[Unit]
Description=Software Heritage Webapp Update SaveCodeNow Statuses
Requires=network.target
After=network.target

[Service]
User=swhwebapp
Group=swhwebapp
Environment=DJANGO_SETTINGS_MODULE=swh.web.settings.production
Type=simple
ExecStart=/usr/bin/django-admin refresh_savecodenow_statuses

[Install]
WantedBy=multi-user.target
<<<
*******************************************
+ Systemd::Unit_file[swh-webapp-update-savecodenow-statuses.timer] =>
   parameters =>
      "active": true
      "enable": true
      "ensure": "present"
      "group": "root"
      "mode": "0444"
      "owner": "root"
      "path": "/etc/systemd/system"
      "show_diff": true
      "content": >>>
[Install]
WantedBy=timers.target

[Timer]
OnCalendar=*:0/1
Unit=swh-webapp-update-savecodenow-statuses.service

[Unit]
Description=regular webapp update savecodenow statuses
<<<
*******************************************
*** End octocatalog-diff on moma.softwareheritage.org

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont edited the summary of this revision. (Show Details)
vlorentz added inline comments.
site-modules/profile/templates/swh/deploy/webapp/swh-webapp-update-savecodenow-statuses.service.erb
7–8

shouldn't it also require postgresql to be up?

site-modules/profile/templates/swh/deploy/webapp/swh-webapp-update-savecodenow-statuses.timer.erb
9
ardumont added inline comments.
site-modules/profile/templates/swh/deploy/webapp/swh-webapp-update-savecodenow-statuses.service.erb
7–8

It's not running on the same node.
So i think it's fine enough as it is.

ardumont edited the summary of this revision. (Show Details)
ardumont edited the test plan for this revision. (Show Details)
  • Adapt according to review (update description)
  • Rework commit message
This revision is now accepted and ready to land.Aug 3 2021, 6:00 PM