Page MenuHomeSoftware Heritage

Declare icinga checks in the yaml configuration
ClosedPublic

Authored by ardumont on Apr 29 2021, 2:40 PM.

Details

Summary

This avoids hard-coding the checks into the manifest.

This in the end goal of fixing the current failing end-to-end snapshot check we have in
staging (ack for a while now).

Test Plan

vagrant up --provision prod-moma
vagrant up --provision staging-webapp

and check the notice logs are of correct output. They do:

moma:

{
  counters => {
    http_uri => /api/1/stat/counters/,
    http_string => "content":
  },
  content end to end => {
    http_uri => /browse/content/4dfc4478b1d5f7388b298fdfc06802485bdeae0c/,
    http_string => PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
  },
  content_known => {
    http_uri => /api/1/content/known/search/,
    http_post => q=8624bcdae55baeef00cd11d5dfcfa60f68710a02,
    http_string => "found":true
  },
  directory end to end => {
    http_uri => /browse/directory/977fc4b98c0e85816348cebd3b12026407c368b6/,
    http_linespan => true,
    http_expect_body_regex => -:"Doc.*Grammar.*Include.*Lib.*Mac.*Misc.*Modules.*Objects.*PC.*PCbuild.*LICENSE.*README.rst"
  },
  revision end to end => {
    http_uri => /browse/revision/f1b94134a4b879bc55c3dacdb496690c8ebdc03f/,
    http_linespan => true,
    http_expect_body_regex => -:"Allocate the output vlc pictures with dimensions padded,.*as requested by the decoder \\\\(for alignments\\\\)."
  },
  revision log end to end => {
    http_uri => /browse/revision/b9b0ecd1e2f9db10335383651f8317ed8cec8296/log/,
    http_linespan => true,
    http_expect_body_regex => -:"/browse/revision/b9b0ecd1e2f9db10335383651f8317ed8cec8296/.*Roberto Di Cosmo.*Moved to github"
  },
  release end to end => {
    http_uri => /browse/release/a9b7e3f1eada90250a6b2ab2ef3e0a846cb16831/,
    http_linespan => true,
    http_expect_body_regex => -:"Linux 4.9-rc8.*/revision/3e5de27e940d00d8d504dfb96625fb654f641509/"
  },
  snapshot end to end => {
    http_uri => /browse/snapshot/baebc2109e4a2ec22a1129a3859647e191d04df4/branches/,
    http_linespan => true,
    http_expect_body_regex => -:"buster/main/4.13.13-1.*buster/main/4.14.12-2.*buster/main/4.14.13-1.*buster/main/4.14.17-1.*buster/main/4.15.4-1.*buster/main/4.9.65-3.*experimental/main/4.10~rc6-1~exp2.*jessie-backports/main/3.16.39-1.*jessie-backports/main/4.7.8-1~bpo8\\\\+1.*jessie-backports/main/4.9.18-1~bpo8\\\\+1.*jessie-backports/main/4.9.65-3\\\\+deb9u1~bpo8\\\\+1.*jessie-backports/main/4.9.65-3\\\\+deb9u2~bpo8\\\\+1.*jessie-kfreebsd/main/3.16.7-ckt9-2.*jessie-proposed-updates/main/3.16.51-3.*jessie-proposed-updates/main/3.16.51-3\\\\+deb8u1.*jessie-updates/main/3.16.51-3.*jessie/main/3.16.43-1.*jessie/main/3.16.51-2.*jessie/main/3.16.7-ckt2-1.*jessie/main/3.16.7-ckt20-1\\\\+deb8u3"
  }
}

staging webapp (only the last snapshot check differs):

{
  counters => {
    http_uri => /api/1/stat/counters/,
    http_string => "content":
  },
  content end to end => {
    http_uri => /browse/content/4dfc4478b1d5f7388b298fdfc06802485bdeae0c/,
    http_string => PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2},
  content_known => {
    http_uri => /api/1/content/known/search/,
    http_post => q=8624bcdae55baeef00cd11d5dfcfa60f68710a02,
    http_string => "found":true},
  directory end to end => {
    http_uri => /browse/directory/977fc4b98c0e85816348cebd3b12026407c368b6/,
    http_linespan => true,
    http_expect_body_regex => -:"Doc.*Grammar.*Include.*Lib.*Mac.*Misc.*Modules.*Objects.*PC.*PCbuild.*LICENSE.*README.rst"},
  revision end to end => {
    http_uri => /browse/revision/f1b94134a4b879bc55c3dacdb496690c8ebdc03f/,
    http_linespan => true,
    http_expect_body_regex => -:"Allocate the output vlc pictures with dimensions padded,.*as requested by the decoder \\\\(for alignments\\\\)."
  },
  revision log end to end => {
    http_uri => /browse/revision/b9b0ecd1e2f9db10335383651f8317ed8cec8296/log/,
    http_linespan => true,
    http_expect_body_regex => -:"/browse/revision/b9b0ecd1e2f9db10335383651f8317ed8cec8296/.*Roberto Di Cosmo.*Moved to github"},
  release end to end => {
    http_uri => /browse/release/a9b7e3f1eada90250a6b2ab2ef3e0a846cb16831/,
    http_linespan => true,
    http_expect_body_regex => -:"Linux 4.9-rc8.*/revision/3e5de27e940d00d8d504dfb96625fb654f641509/"
  },
  snapshot end to end => {
    http_uri => /browse/snapshot/baebc2109e4a2ec22a1129a3859647e191d04df4/branches/,
    http_linespan => true,
    http_expect_body_regex => -:"refs/tags/syslinux-3.20-pre2.*refs/tags/syslinux-3.20-pre3.*"
  }
}

(snapshot chosen randomly in the staging db) [1]

[1]

swh=> select *
                    from origin_visit ov
                    inner join origin o on ov.origin=o.id
                    inner join origin_visit_status ovs using (origin, visit)
                    where ovs.status='full'
                      and ov.type='git'
                      and ov.date > now() - '3 months'::interval
                      and random() < 0.1
                    limit 1
;
 origin | visit |             date              | type |   id   |                            url                            |             date              | status | metadata |                  snapshot                  | type
--------+-------+-------------------------------+------+--------+-----------------------------------------------------------+-------------------------------+--------+----------+--------------------------------------------+------
 243673 |     3 | 2021-01-29 17:34:11.555383+00 | git  | 243673 | https://git.kernel.org/pub/scm/boot/syslinux/syslinux.git | 2021-01-29 17:34:14.626338+00 | full   |          | \x48dcf76ec1a3bd57ec117b1dace633691fdfd70d | git
(1 row)

Diff Detail

Repository
rSPSITE puppet-swh-site
Branch
staging
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 21139
Build 32806: arc lint + arc unit

Event Timeline

ardumont created this revision.
olasd added a subscriber: olasd.

Don't forget to drop the notice calls ;)

This revision is now accepted and ready to land.Apr 29 2021, 2:45 PM

Don't forget to drop the notice calls ;)

Yep, thanks. I initially let them hang so it showed how i kinda debugged it.