Differential D5256

(v2) e2e-checks: Install vault and deposit checks for production and staging
AbandonedPublic
Actions

Authored by ardumont on Mar 16 2021, 12:14 PM.

Details

Reviewers

None

Group Reviewers

System administrators

Maniphest Tasks

T3133: deposit: Add equivalent production integration test to staging

Summary

It was only installed for production so far.

v2 because there is a v1.

I don't know which is best.

Related to T3133

[1] v1: D5257

Test Plan

bin/octocatalog-diff deposit.internal.staging.swh.network|vault.internal.staging.swh.network -> nothing as expected.
bin/octocatalog-diff pergamon (drop old check-deposit and check-vault for {production-check-deposit, staging-check-deposit} and {production-check-vault, staging-check-vault}:

$ bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to staging pergamon
Found host pergamon.softwareheritage.org
WARN     -> Environment "wip-pg-hba-rules-in-yaml" contained non-word characters, correcting name to wip_pg_hba_rules_in_yaml
Cloning into '/tmp/swh-ocd.yU9AR6Np/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.yU9AR6Np/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host pergamon.softwareheritage.org
I, [2021-03-16T13:57:12.757110 #17120]  INFO -- : Catalogs compiled for pergamon.softwareheritage.org
W, [2021-03-16T13:57:18.273603 #17120]  WARN -- : Resource File[/tmp/ocd-ipc-20210316-17120-tfxq4o/ocd-builddir-20210316-17129-16w9ung/routes.yaml] appears to depend on catalog compilation directory. Suppressed from results.
W, [2021-03-16T13:57:18.274119 #17120]  WARN -- : Resource File[/tmp/ocd-ipc-20210316-17120-tfxq4o/ocd-builddir-20210316-17126-13gkeyw/routes.yaml] appears to depend on catalog compilation directory. Suppressed from results.
W, [2021-03-16T13:57:18.274766 #17120]  WARN -- : Resource key Ini_setting[puppetdbserver_urls] parameters => path may depend on catalog compilation directory, but there may be differences. This is included in results for now, but please verify.
W, [2021-03-16T13:57:18.274783 #17120]  WARN -- : Resource key Ini_setting[puppetdbserver_urls] parameters => path appears to depend on catalog compilation directory. Suppressed from results.
W, [2021-03-16T13:57:18.274818 #17120]  WARN -- : Resource key Ini_setting[soft_write_failure] parameters => path may depend on catalog compilation directory, but there may be differences. This is included in results for now, but please verify.
W, [2021-03-16T13:57:18.274829 #17120]  WARN -- : Resource key Ini_setting[soft_write_failure] parameters => path appears to depend on catalog compilation directory. Suppressed from results.
I, [2021-03-16T13:57:18.275977 #17120]  INFO -- : Diffs computed for pergamon.softwareheritage.org
diff origin/production/pergamon.softwareheritage.org current/pergamon.softwareheritage.org
*******************************************
- Concat::Fragment[icinga2::object::CheckCommand::check-deposit-cmd]
*******************************************
- Concat::Fragment[icinga2::object::CheckCommand::check-vault-cmd]
*******************************************
- Concat::Fragment[icinga2::object::Service::check-deposit]
*******************************************
- Concat::Fragment[icinga2::object::Service::check-vault]
*******************************************
+ Concat::Fragment[icinga2::object::Service::production-check-deposit] =>
   parameters =>
      "order": 60
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "content": >>>

object Service "(production) Check deposit end-to-end" {
  import "generic-service-check-e2e"

  host_name = "pergamon.softwareheritage.org"
  check_command = "production-deposit-check-cmd"
}
<<<
*******************************************
+ Concat::Fragment[icinga2::object::Service::production-check-vault] =>
   parameters =>
      "order": 60
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "content": >>>

object Service "(production) Check vault end-to-end" {
  import "generic-service-check-e2e"

  host_name = "pergamon.softwareheritage.org"
  check_command = "production-vault-check-cmd"
}
<<<
*******************************************
+ Concat::Fragment[icinga2::object::Service::staging-check-deposit] =>
   parameters =>
      "order": 60
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "content": >>>

object Service "(staging) Check deposit end-to-end" {
  import "generic-service-check-e2e"

  host_name = "pergamon.softwareheritage.org"
  check_command = "staging-deposit-check-cmd"
}
<<<
*******************************************
+ Concat::Fragment[icinga2::object::Service::staging-check-vault] =>
   parameters =>
      "order": 60
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "content": >>>

object Service "(staging) Check vault end-to-end" {
  import "generic-service-check-e2e"

  host_name = "pergamon.softwareheritage.org"
  check_command = "staging-vault-check-cmd"
}
<<<
*******************************************
- Concat[/etc/icinga2/conf.d/e2e-checks.conf]
*******************************************
- Concat_file[/etc/icinga2/conf.d/e2e-checks.conf]
*******************************************
- Concat_fragment[/etc/icinga2/conf.d/e2e-checks.conf_header]
*******************************************
- Concat_fragment[icinga2::object::CheckCommand::check-deposit-cmd]
*******************************************
- Concat_fragment[icinga2::object::CheckCommand::check-vault-cmd]
*******************************************
- Concat_fragment[icinga2::object::Service::check-deposit]
*******************************************
- Concat_fragment[icinga2::object::Service::check-vault]
*******************************************
+ Concat_fragment[icinga2::object::Service::production-check-deposit] =>
   parameters =>
      "order": 60
      "tag": "_etc_icinga2_zones.d_master_pergamon.softwareheritage.org.conf"
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "content": >>>

object Service "(production) Check deposit end-to-end" {
  import "generic-service-check-e2e"

  host_name = "pergamon.softwareheritage.org"
  check_command = "production-deposit-check-cmd"
}
<<<
*******************************************
+ Concat_fragment[icinga2::object::Service::production-check-vault] =>
   parameters =>
      "order": 60
      "tag": "_etc_icinga2_zones.d_master_pergamon.softwareheritage.org.conf"
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "content": >>>

object Service "(production) Check vault end-to-end" {
  import "generic-service-check-e2e"

  host_name = "pergamon.softwareheritage.org"
  check_command = "production-vault-check-cmd"
}
<<<
*******************************************
+ Concat_fragment[icinga2::object::Service::staging-check-deposit] =>
   parameters =>
      "order": 60
      "tag": "_etc_icinga2_zones.d_master_pergamon.softwareheritage.org.conf"
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "content": >>>

object Service "(staging) Check deposit end-to-end" {
  import "generic-service-check-e2e"

  host_name = "pergamon.softwareheritage.org"
  check_command = "staging-deposit-check-cmd"
}
<<<
*******************************************
+ Concat_fragment[icinga2::object::Service::staging-check-vault] =>
   parameters =>
      "order": 60
      "tag": "_etc_icinga2_zones.d_master_pergamon.softwareheritage.org.conf"
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "content": >>>

object Service "(staging) Check vault end-to-end" {
  import "generic-service-check-e2e"

  host_name = "pergamon.softwareheritage.org"
  check_command = "staging-vault-check-cmd"
}
<<<
*******************************************
  File[/etc/bind/keys/local-update] =>
   parameters =>
     content =>
      @@ -2,4 +2,4 @@
       key local-update {
        algorithm hmac-sha256;
      - secret "TO78xNiuuC4MpB3zxxpmkR9hKO+o2CxjlUPqi0ZOsciuI58tjYF1oSMm+0WElmLvysVhXl9clkK/u58RWFsYwQ==";
      + secret "ExenYLGVF7VWd7zlrngQMD0UhsVozCUdS/qcMPUM+K4u4B/GoFRPxdTGIQpGotrNs3vpRUeokdT4bjAC4yN59A==";
       };
*******************************************
  File[/etc/bind/rndc.key] =>
   parameters =>
     content =>
      @@ -2,4 +2,4 @@
       key rndc-key {
        algorithm hmac-md5;
      - secret "MIOjweXr429TEPBrKyElhtvKjorqdRcik5VPrHLS0b9FzFS28yaVr/I4za0Owp2KLfs6EQGSom6aoWzXB5P/BA==";
      + secret "uwokh/N3OJB3Art45jSreNhU0ooijK5uuq7Qp1rOxH1gIaMBDycWDP8E6nReqXXqIiwnFuKm8EBSV1r94OPSdQ==";
       };
*******************************************
- Icinga2::Object::Checkcommand[check-deposit-cmd]
*******************************************
- Icinga2::Object::Checkcommand[check-vault-cmd]
*******************************************
- Icinga2::Object::Service[check-deposit]
*******************************************
- Icinga2::Object::Service[check-vault]
*******************************************
+ Icinga2::Object::Service[production-check-deposit] =>
   parameters =>
      "apply": false
      "assign": []
      "check_command": "production-deposit-check-cmd"
      "ensure": "present"
      "host_name": "pergamon.softwareheritage.org"
      "ignore": []
      "import": ["generic-service-check-e2e"]
      "order": 60
      "prefix": false
      "service_name": "(production) Check deposit end-to-end"
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "template": false
*******************************************
+ Icinga2::Object::Service[production-check-vault] =>
   parameters =>
      "apply": false
      "assign": []
      "check_command": "production-vault-check-cmd"
      "ensure": "present"
      "host_name": "pergamon.softwareheritage.org"
      "ignore": []
      "import": ["generic-service-check-e2e"]
      "order": 60
      "prefix": false
      "service_name": "(production) Check vault end-to-end"
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "template": false
*******************************************
+ Icinga2::Object::Service[staging-check-deposit] =>
   parameters =>
      "apply": false
      "assign": []
      "check_command": "staging-deposit-check-cmd"
      "ensure": "present"
      "host_name": "pergamon.softwareheritage.org"
      "ignore": []
      "import": ["generic-service-check-e2e"]
      "order": 60
      "prefix": false
      "service_name": "(staging) Check deposit end-to-end"
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "template": false
*******************************************
+ Icinga2::Object::Service[staging-check-vault] =>
   parameters =>
      "apply": false
      "assign": []
      "check_command": "staging-vault-check-cmd"
      "ensure": "present"
      "host_name": "pergamon.softwareheritage.org"
      "ignore": []
      "import": ["generic-service-check-e2e"]
      "order": 60
      "prefix": false
      "service_name": "(staging) Check vault end-to-end"
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "template": false
*******************************************
- Icinga2::Object[icinga2::object::CheckCommand::check-deposit-cmd]
*******************************************
- Icinga2::Object[icinga2::object::CheckCommand::check-vault-cmd]
*******************************************
- Icinga2::Object[icinga2::object::Service::check-deposit]
*******************************************
- Icinga2::Object[icinga2::object::Service::check-vault]
*******************************************
+ Icinga2::Object[icinga2::object::Service::production-check-deposit] =>
   parameters =>
      "apply": false
      "assign": []
      "attrs": {"host_name"=>"pergamon.softwareheritage.org", "check_command"=>"production-deposit-check-cmd"}
      "attrs_list": ["display_name", "host_name", "check_command", "check_timeout", "check_interval", "check_period", "retry_interval", "max_check_attempts", "groups", "enable_notifications", "enable_active_checks", "enable_passive_checks", "enable_event_handler", "enable_flapping", "enable_perfdata", "event_command", "flapping_threshold_low", "flapping_threshold_high", "volatile", "zone", "command_endpoint", "notes", "notes_url", "action_url", "icon_image", "icon_image_alt", "vars", "Acknowledgement", "ApiBindHost", "ApiBindPort", "ApiEnvironment", "ApplicationType", "AttachDebugger", "BuildCompilerName", "BuildCompilerVersion", "BuildHostName", "Concurrency", "Critical", "Custom", "Deprecated", "Down", "DowntimeEnd", "DowntimeRemoved", "DowntimeStart", "Environment", "FlappingEnd", "FlappingStart", "HostDown", "HostUp", "IncludeConfDir", "Internal", "Json", "LocalStateDir", "LogCritical", "LogDebug", "LogInformation", "LogNotice", "LogWarning", "Math", "MaxConcurrentChecks", "ModAttrPath", "NodeName", "OK", "ObjectsPath", "PidPath", "PkgDataDir", "PlatformArchitecture", "PlatformKernel", "PlatformKernelVersion", "PlatformName", "PlatformVersion", "PrefixDir", "Problem", "Recovery", "RunAsGroup", "RunAsUser", "RunDir", "ServiceCritical", "ServiceOK", "ServiceUnknown", "ServiceWarning", "StatePath", "SysconfDir", "System", "Types", "Unknown", "Up", "UseVfork", "VarsPath", "Warning", "ZonesDir", "NodeName", "ZoneName", "TicketSalt", "PluginDir", "PluginContribDir", "ManubulonPluginDir", "name", "NodeName", "ZoneName", "TicketSalt", "PluginDir", "PluginContribDir", "ManubulonPluginDir", "name"]
      "ensure": "present"
      "ignore": []
      "import": ["generic-service-check-e2e"]
      "object_name": "(production) Check deposit end-to-end"
      "object_type": "Service"
      "order": 60
      "prefix": false
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "template": false
*******************************************
+ Icinga2::Object[icinga2::object::Service::production-check-vault] =>
   parameters =>
      "apply": false
      "assign": []
      "attrs": {"host_name"=>"pergamon.softwareheritage.org", "check_command"=>"production-vault-check-cmd"}
      "attrs_list": ["display_name", "host_name", "check_command", "check_timeout", "check_interval", "check_period", "retry_interval", "max_check_attempts", "groups", "enable_notifications", "enable_active_checks", "enable_passive_checks", "enable_event_handler", "enable_flapping", "enable_perfdata", "event_command", "flapping_threshold_low", "flapping_threshold_high", "volatile", "zone", "command_endpoint", "notes", "notes_url", "action_url", "icon_image", "icon_image_alt", "vars", "Acknowledgement", "ApiBindHost", "ApiBindPort", "ApiEnvironment", "ApplicationType", "AttachDebugger", "BuildCompilerName", "BuildCompilerVersion", "BuildHostName", "Concurrency", "Critical", "Custom", "Deprecated", "Down", "DowntimeEnd", "DowntimeRemoved", "DowntimeStart", "Environment", "FlappingEnd", "FlappingStart", "HostDown", "HostUp", "IncludeConfDir", "Internal", "Json", "LocalStateDir", "LogCritical", "LogDebug", "LogInformation", "LogNotice", "LogWarning", "Math", "MaxConcurrentChecks", "ModAttrPath", "NodeName", "OK", "ObjectsPath", "PidPath", "PkgDataDir", "PlatformArchitecture", "PlatformKernel", "PlatformKernelVersion", "PlatformName", "PlatformVersion", "PrefixDir", "Problem", "Recovery", "RunAsGroup", "RunAsUser", "RunDir", "ServiceCritical", "ServiceOK", "ServiceUnknown", "ServiceWarning", "StatePath", "SysconfDir", "System", "Types", "Unknown", "Up", "UseVfork", "VarsPath", "Warning", "ZonesDir", "NodeName", "ZoneName", "TicketSalt", "PluginDir", "PluginContribDir", "ManubulonPluginDir", "name", "NodeName", "ZoneName", "TicketSalt", "PluginDir", "PluginContribDir", "ManubulonPluginDir", "name"]
      "ensure": "present"
      "ignore": []
      "import": ["generic-service-check-e2e"]
      "object_name": "(production) Check vault end-to-end"
      "object_type": "Service"
      "order": 60
      "prefix": false
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "template": false
*******************************************
+ Icinga2::Object[icinga2::object::Service::staging-check-deposit] =>
   parameters =>
      "apply": false
      "assign": []
      "attrs": {"host_name"=>"pergamon.softwareheritage.org", "check_command"=>"staging-deposit-check-cmd"}
      "attrs_list": ["display_name", "host_name", "check_command", "check_timeout", "check_interval", "check_period", "retry_interval", "max_check_attempts", "groups", "enable_notifications", "enable_active_checks", "enable_passive_checks", "enable_event_handler", "enable_flapping", "enable_perfdata", "event_command", "flapping_threshold_low", "flapping_threshold_high", "volatile", "zone", "command_endpoint", "notes", "notes_url", "action_url", "icon_image", "icon_image_alt", "vars", "Acknowledgement", "ApiBindHost", "ApiBindPort", "ApiEnvironment", "ApplicationType", "AttachDebugger", "BuildCompilerName", "BuildCompilerVersion", "BuildHostName", "Concurrency", "Critical", "Custom", "Deprecated", "Down", "DowntimeEnd", "DowntimeRemoved", "DowntimeStart", "Environment", "FlappingEnd", "FlappingStart", "HostDown", "HostUp", "IncludeConfDir", "Internal", "Json", "LocalStateDir", "LogCritical", "LogDebug", "LogInformation", "LogNotice", "LogWarning", "Math", "MaxConcurrentChecks", "ModAttrPath", "NodeName", "OK", "ObjectsPath", "PidPath", "PkgDataDir", "PlatformArchitecture", "PlatformKernel", "PlatformKernelVersion", "PlatformName", "PlatformVersion", "PrefixDir", "Problem", "Recovery", "RunAsGroup", "RunAsUser", "RunDir", "ServiceCritical", "ServiceOK", "ServiceUnknown", "ServiceWarning", "StatePath", "SysconfDir", "System", "Types", "Unknown", "Up", "UseVfork", "VarsPath", "Warning", "ZonesDir", "NodeName", "ZoneName", "TicketSalt", "PluginDir", "PluginContribDir", "ManubulonPluginDir", "name", "NodeName", "ZoneName", "TicketSalt", "PluginDir", "PluginContribDir", "ManubulonPluginDir", "name"]
      "ensure": "present"
      "ignore": []
      "import": ["generic-service-check-e2e"]
      "object_name": "(staging) Check deposit end-to-end"
      "object_type": "Service"
      "order": 60
      "prefix": false
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "template": false
*******************************************
+ Icinga2::Object[icinga2::object::Service::staging-check-vault] =>
   parameters =>
      "apply": false
      "assign": []
      "attrs": {"host_name"=>"pergamon.softwareheritage.org", "check_command"=>"staging-vault-check-cmd"}
      "attrs_list": ["display_name", "host_name", "check_command", "check_timeout", "check_interval", "check_period", "retry_interval", "max_check_attempts", "groups", "enable_notifications", "enable_active_checks", "enable_passive_checks", "enable_event_handler", "enable_flapping", "enable_perfdata", "event_command", "flapping_threshold_low", "flapping_threshold_high", "volatile", "zone", "command_endpoint", "notes", "notes_url", "action_url", "icon_image", "icon_image_alt", "vars", "Acknowledgement", "ApiBindHost", "ApiBindPort", "ApiEnvironment", "ApplicationType", "AttachDebugger", "BuildCompilerName", "BuildCompilerVersion", "BuildHostName", "Concurrency", "Critical", "Custom", "Deprecated", "Down", "DowntimeEnd", "DowntimeRemoved", "DowntimeStart", "Environment", "FlappingEnd", "FlappingStart", "HostDown", "HostUp", "IncludeConfDir", "Internal", "Json", "LocalStateDir", "LogCritical", "LogDebug", "LogInformation", "LogNotice", "LogWarning", "Math", "MaxConcurrentChecks", "ModAttrPath", "NodeName", "OK", "ObjectsPath", "PidPath", "PkgDataDir", "PlatformArchitecture", "PlatformKernel", "PlatformKernelVersion", "PlatformName", "PlatformVersion", "PrefixDir", "Problem", "Recovery", "RunAsGroup", "RunAsUser", "RunDir", "ServiceCritical", "ServiceOK", "ServiceUnknown", "ServiceWarning", "StatePath", "SysconfDir", "System", "Types", "Unknown", "Up", "UseVfork", "VarsPath", "Warning", "ZonesDir", "NodeName", "ZoneName", "TicketSalt", "PluginDir", "PluginContribDir", "ManubulonPluginDir", "name", "NodeName", "ZoneName", "TicketSalt", "PluginDir", "PluginContribDir", "ManubulonPluginDir", "name"]
      "ensure": "present"
      "ignore": []
      "import": ["generic-service-check-e2e"]
      "object_name": "(staging) Check vault end-to-end"
      "object_type": "Service"
      "order": 60
      "prefix": false
      "target": "/etc/icinga2/zones.d/master/pergamon.softwareheritage.org.conf"
      "template": false
*******************************************
- Package[python3-swh.icingaplugins]
*******************************************
*** End octocatalog-diff on pergamon.softwareheritage.org

Diff Detail

Repository

rSPSITE puppet-swh-site

Branch

staging

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 19942
Build 30969: arc lint + arc unit

Event Timeline

ardumont requested review of this revision.Mar 16 2021, 12:14 PM

ardumont created this revision.

Harbormaster completed remote builds in B19935: Diff 18841.Mar 16 2021, 12:14 PM

ardumont retitled this revision from e2e-checks: Install checks for staging and production machines to wip: e2e-checks: Install checks for staging and production machines.Mar 16 2021, 12:20 PM

ardumont edited the test plan for this revision. (Show Details)

ardumont edited the summary of this revision. (Show Details)Mar 16 2021, 1:59 PM

ardumont edited the test plan for this revision. (Show Details)

Install service check in the master as before

Harbormaster completed remote builds in B19940: Diff 18845.Mar 16 2021, 1:59 PM

ardumont added a reviewer: System administrators.Mar 16 2021, 2:00 PM

ardumont added inline comments.

site-modules/profile/manifests/icinga2/master.pp
66–81	I have yet to determine how to avoid hard-coding this.

ardumont retitled this revision from wip: e2e-checks: Install checks for staging and production machines to e2e-checks: Install vault and deposit checks for production and staging.Mar 16 2021, 2:01 PM

ardumont edited the summary of this revision. (Show Details)

ardumont edited the test plan for this revision. (Show Details)Mar 16 2021, 2:06 PM

Rework commit message

Harbormaster completed remote builds in B19941: Diff 18846.Mar 16 2021, 2:06 PM

simplify a bit (drop spurious line, drop intermediary variable)

Harbormaster completed remote builds in B19942: Diff 18847.Mar 16 2021, 2:11 PM

ardumont retitled this revision from e2e-checks: Install vault and deposit checks for production and staging to (v2) e2e-checks: Install vault and deposit checks for production and staging.Mar 16 2021, 2:26 PM

ardumont edited the summary of this revision. (Show Details)

ardumont edited the summary of this revision. (Show Details)Mar 16 2021, 2:29 PM

I'll give v1 a short first.

ardumont edited the summary of this revision. (Show Details)Mar 16 2021, 3:33 PM

ardumont mentioned this in T3133: deposit: Add equivalent production integration test to staging.Mar 16 2021, 4:43 PM