Page MenuHomeSoftware Heritage
Differential D6407

Adapt logrotate configuration so extra directory is also logrotated
ClosedPublic
Actions

Authored by ardumont on Oct 5 2021, 9:13 AM.

Details

Reviewers
anlambert
vsellier
Group Reviewers
System administration
Maniphest Tasks
T3313: Web API: per-user accounting
Commits
rSPSITE2a007a23921c: Adapt logrotate configuration for webapp instances
Summary

On moma, the current logrotate stanza won't do anything:

root@moma:~# file /var/log/softwareheritage/webapp/swh-web.log
/var/log/softwareheritage/webapp/swh-web.log: ASCII text, with very long lines

And current size is a tad big:

root@moma:~# ls -lah /var/log/softwareheritage/webapp/swh-web.log
-rwxrwx--- 1 swhwebapp swhwebapp 5.7G Oct  5 07:13 /var/log/softwareheritage/webapp/swh-web.log

There is a warning icinga alert about it [1]:

02:01 <+swhbot`> icinga RECOVERY: service disk / on moma.softwareheritage.org is OK: DISK OK - free space: / 4452 MB (23% inode=86%);

[1] Noticed this week end but it's still a thing

Related to T3313

Test Plan

bin/octocatalog diff (on webapp nodes):

diff origin/production/webapp.internal.staging.swh.network current/webapp.internal.staging.swh.network
*******************************************
+ File[/etc/logrotate.d/swh-webapp] =>
   parameters =>
      "ensure": "file"
      "group": "swhwebapp"
      "mode": "0644"
      "owner": "swhwebapp"
      "content": >>>
# Software Heritage logrotate configuration
# Managed by puppet. Changes will be overwritten.

/var/log/softwareheritage/webapp/swh-web.log {
    weekly
    missingok
    rotate 5
    compress
    notifempty
    su swhwebapp swhwebapp
    postrotate
      kill -USR1 $(cat /var/run/gunicorn/swh-webapp/pidfile)
    endscript
}
<<<
*******************************************
*** End octocatalog-diff on webapp.internal.staging.swh.network

Diff Detail

Repository
rSPSITE puppet-swh-site
Branch
staging
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 24209
Build 37788: arc lint + arc unit

Event Timeline

ardumont requested review of this revision.Oct 5 2021, 9:13 AM
ardumont created this revision.
Harbormaster completed remote builds in B24201: Diff 23284.Oct 5 2021, 9:13 AM
ardumont edited the test plan for this revision. (Show Details)Oct 5 2021, 9:14 AM
ardumont added a reviewer: System administration.
ardumont edited the summary of this revision. (Show Details)Oct 5 2021, 9:21 AM
ardumont edited the summary of this revision. (Show Details)Oct 5 2021, 10:47 AM
ardumont added a task: T3313: Web API: per-user accounting.
anlambert accepted this revision.Oct 5 2021, 10:50 AM
anlambert added a subscriber: anlambert.

+1, thanks !

This revision is now accepted and ready to land.Oct 5 2021, 10:50 AM
ardumont edited the test plan for this revision. (Show Details)Oct 5 2021, 10:54 AM
ardumont updated this revision to Diff 23292.Oct 5 2021, 10:56 AM
ardumont edited the test plan for this revision. (Show Details)
ardumont added a subscriber: vsellier.

Adapt after discussing with @vsellier

Use copytruncate to avoid having to deal with a gunicor-swh-webapp reloading. When the
file is compressed, a new one is created and then the webapp no longer logs. This
copytruncate should avoid this ^ as the existing file remains after the logrotation
happened.

Harbormaster completed remote builds in B24206: Diff 23292.Oct 5 2021, 10:56 AM
ardumont updated this revision to Diff 23293.Oct 5 2021, 10:57 AM

Use correct repository

Harbormaster completed remote builds in B24207: Diff 23293.Oct 5 2021, 10:57 AM
ardumont edited the test plan for this revision. (Show Details)Oct 5 2021, 11:00 AM
ardumont updated this revision to Diff 23294.Oct 5 2021, 11:00 AM
ardumont edited the test plan for this revision. (Show Details)

Push variables at the top level manifest

Harbormaster completed remote builds in B24208: Diff 23294.Oct 5 2021, 11:01 AM
ardumont updated this revision to Diff 23295.Oct 5 2021, 11:01 AM

Rebase

Harbormaster completed remote builds in B24209: Diff 23295.Oct 5 2021, 11:01 AM
vsellier added inline comments.Oct 5 2021, 11:11 AM
site-modules/profile/templates/swh/logrotate-webapp.conf.erb
8

this postrotate script should force the webapp to recreate the logfile instead of copying it

ardumont edited the test plan for this revision. (Show Details)Oct 5 2021, 11:21 AM
ardumont added inline comments.
site-modules/profile/templates/swh/logrotate-webapp.conf.erb
8

awesome, thx

ardumont updated this revision to Diff 23296.Oct 5 2021, 11:21 AM

Use postrotate stanza

Harbormaster completed remote builds in B24210: Diff 23296.Oct 5 2021, 11:21 AM
ardumont updated this revision to Diff 23297.Oct 5 2021, 11:22 AM

Adapt according to suggestion

Harbormaster completed remote builds in B24211: Diff 23297.Oct 5 2021, 11:22 AM
ardumont updated this revision to Diff 23299.Oct 5 2021, 11:23 AM

Update after the correct rebase

Harbormaster completed remote builds in B24213: Diff 23299.Oct 5 2021, 11:23 AM
vsellier accepted this revision.Oct 5 2021, 11:26 AM

LGTM thanks

It will allow to wait for the change of logging paradigm in T3313

Closed by commit rSPSITE2a007a23921c: Adapt logrotate configuration for webapp instances (authored by ardumont). · Explain WhyOct 5 2021, 11:32 AM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rSPSITE2a007a23921c: Adapt logrotate configuration for webapp instances.

Revision Contents
Changeset List

PathSize
site-modules/
profile/
manifests/
swh/
deploy/
16 lines
templates/
swh/
12 lines

Diff 23295

View Options

site-modules/profile/manifests/swh/deploy/webapp.pp

Loading...
View Options

site-modules/profile/templates/swh/logrotate-webapp.conf.erb

Loading...
About · Developer information · Legal