Page MenuHomeSoftware Heritage

Adapt logrotate configuration so extra directory is also logrotated
ClosedPublic

Authored by ardumont on Oct 5 2021, 9:13 AM.

Details

Summary

On moma, the current logrotate stanza won't do anything:

root@moma:~# file /var/log/softwareheritage/webapp/swh-web.log
/var/log/softwareheritage/webapp/swh-web.log: ASCII text, with very long lines

And current size is a tad big:

root@moma:~# ls -lah /var/log/softwareheritage/webapp/swh-web.log
-rwxrwx--- 1 swhwebapp swhwebapp 5.7G Oct  5 07:13 /var/log/softwareheritage/webapp/swh-web.log

There is a warning icinga alert about it [1]:

02:01 <+swhbot`> icinga RECOVERY: service disk / on moma.softwareheritage.org is OK: DISK OK - free space: / 4452 MB (23% inode=86%);

[1] Noticed this week end but it's still a thing

Related to T3313

Test Plan

bin/octocatalog diff (on webapp nodes):

diff origin/production/webapp.internal.staging.swh.network current/webapp.internal.staging.swh.network
*******************************************
+ File[/etc/logrotate.d/swh-webapp] =>
   parameters =>
      "ensure": "file"
      "group": "swhwebapp"
      "mode": "0644"
      "owner": "swhwebapp"
      "content": >>>
# Software Heritage logrotate configuration
# Managed by puppet. Changes will be overwritten.

/var/log/softwareheritage/webapp/swh-web.log {
    weekly
    missingok
    rotate 5
    compress
    notifempty
    su swhwebapp swhwebapp
    postrotate
      kill -USR1 $(cat /var/run/gunicorn/swh-webapp/pidfile)
    endscript
}
<<<
*******************************************
*** End octocatalog-diff on webapp.internal.staging.swh.network

Diff Detail

Repository
rSPSITE puppet-swh-site
Branch
staging
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24207
Build 37786: arc lint + arc unit

Event Timeline

ardumont created this revision.
This revision is now accepted and ready to land.Oct 5 2021, 10:50 AM
ardumont edited the test plan for this revision. (Show Details)
ardumont added a subscriber: vsellier.

Adapt after discussing with @vsellier

Use copytruncate to avoid having to deal with a gunicor-swh-webapp reloading. When the
file is compressed, a new one is created and then the webapp no longer logs. This
copytruncate should avoid this ^ as the existing file remains after the logrotation
happened.

ardumont edited the test plan for this revision. (Show Details)

Push variables at the top level manifest

site-modules/profile/templates/swh/logrotate-webapp.conf.erb
8

this postrotate script should force the webapp to recreate the logfile instead of copying it

ardumont added inline comments.
site-modules/profile/templates/swh/logrotate-webapp.conf.erb
8

awesome, thx

Adapt according to suggestion

Update after the correct rebase

LGTM thanks

It will allow to wait for the change of logging paradigm in T3313