Page MenuHomeSoftware Heritage
Differential D7045

Migrate sentry node to admin vlan
ClosedPublic
Actions

Authored by ardumont on Jan 27 2022, 9:44 AM.

Details

Reviewers
vsellier
Group Reviewers
System administrators
Maniphest Tasks
T3891: Migrate sentry to admin vlan
Commits
rSPSITE36667e66d210: Migrate sentry node to admin vlan
Summary

This:

  • Actually installs the reverse proxy part to serve sentry request
  • Update varnish reverse proxy to allow specific icinga checks (to conserve actual sentry checks as is).
  • No impact on riverside node regarding sentry (besides its vagrant ip change)
  • Installs a rewrite rule on the pergamon reverse proxy to conserve the sentry.s.o resolution as is the time the ttl expiry happens [1]

[1] sentry.s.o will change from targetting pergamon to targetting swh-rproxy3.inria.fr

Related to T3891

Test Plan

riverside (but cannot octo-diff it as the fqdn changes)

pergamon (reverse-proxy, dns) impacted:

$ $SWH_PUPPET_ENVIRONMENT_HOME/bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to staging pergamon
...
diff origin/production/pergamon.softwareheritage.org current/pergamon.softwareheritage.org
*******************************************
- Apache::Mod[proxy]
*******************************************
- Apache::Mod[proxy_http]
*******************************************
  Apache::Vhost[sentry.softwareheritage.org_non-ssl] =>
   parameters =>
     docroot =>
      - /var/www/html
      + /var/www
     manage_docroot =>
      - false
      + true
*******************************************
  Apache::Vhost[sentry.softwareheritage.org_ssl] =>
   parameters =>
     docroot =>
      - /var/www/html
      + /var/www
     manage_docroot =>
      - false
      + true
     proxy_pass =>
      - [{"path"=>"/", "url"=>"http://riverside.internal.softwareheritage.org:9000/"}]
     proxy_preserve_host =>
      - true
      + false
     request_headers =>
      - ["set X-Forwarded-Proto \"https\"", "set X-Forwarded-Port \"443\""]
     rewrites =>
      + [{"rewrite_rule"=>["^.*$ http://riverside.internal.admin.swh.network"]}]
     ssl_cert =>
      - /etc/ssl/certs/letsencrypt/sentry/cert.pem
      + /etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/cert.pem
     ssl_chain =>
      - /etc/ssl/certs/letsencrypt/sentry/chain.pem
      + /etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/chain.pem
     ssl_key =>
      - /etc/ssl/certs/letsencrypt/sentry/privkey.pem
      + /etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/privkey.pem
*******************************************
  Concat::Fragment[sentry.softwareheritage.org_non-ssl-directories] =>
   parameters =>
     content =>
      @@ -1,6 +1,6 @@
      _
      -  ## Directories, there should at least be a declaration for /var/www/html
      +  ## Directories, there should at least be a declaration for /var/www
      _
      -  <Directory "/var/www/html">
      +  <Directory "/var/www">
           Options Indexes FollowSymLinks MultiViews
           AllowOverride None
*******************************************
  Concat::Fragment[sentry.softwareheritage.org_non-ssl-docroot] =>
   parameters =>
     content =>
      @@ -1,3 +1,3 @@
      _
         ## Vhost docroot
      -  DocumentRoot "/var/www/html"
      +  DocumentRoot "/var/www"
*******************************************
  Concat::Fragment[sentry.softwareheritage.org_ssl-directories] =>
   parameters =>
     content =>
      @@ -1,6 +1,6 @@
      _
      -  ## Directories, there should at least be a declaration for /var/www/html
      +  ## Directories, there should at least be a declaration for /var/www
      _
      -  <Directory "/var/www/html">
      +  <Directory "/var/www">
           Options Indexes FollowSymLinks MultiViews
           AllowOverride None
*******************************************
  Concat::Fragment[sentry.softwareheritage.org_ssl-docroot] =>
   parameters =>
     content =>
      @@ -1,3 +1,3 @@
      _
         ## Vhost docroot
      -  DocumentRoot "/var/www/html"
      +  DocumentRoot "/var/www"
*******************************************
- Concat::Fragment[sentry.softwareheritage.org_ssl-proxy]
*******************************************
- Concat::Fragment[sentry.softwareheritage.org_ssl-requestheader]
*******************************************
+ Concat::Fragment[sentry.softwareheritage.org_ssl-rewrite] =>
   parameters =>
      "order": 190
      "target": "25-sentry.softwareheritage.org_ssl.conf"
      "content": >>>
  ## Rewrite rules
  RewriteEngine On

  RewriteRule ^.*$ http://riverside.internal.admin.swh.network
<<<
*******************************************
  Concat::Fragment[sentry.softwareheritage.org_ssl-ssl] =>
   parameters =>
     content =>
      @@ -2,7 +2,7 @@
         ## SSL directives
         SSLEngine on
      -  SSLCertificateFile      "/etc/ssl/certs/letsencrypt/sentry/cert.pem"
      -  SSLCertificateKeyFile   "/etc/ssl/certs/letsencrypt/sentry/privkey.pem"
      -  SSLCertificateChainFile "/etc/ssl/certs/letsencrypt/sentry/chain.pem"
      +  SSLCertificateFile      "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/cert.pem"
      +  SSLCertificateKeyFile   "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/privkey.pem"
      +  SSLCertificateChainFile "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/chain.pem"
         SSLProtocol             all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
         SSLCipherSuite          ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
*******************************************
  Concat_fragment[sentry.softwareheritage.org_non-ssl-directories] =>
   parameters =>
     content =>
      @@ -1,6 +1,6 @@
      _
      -  ## Directories, there should at least be a declaration for /var/www/html
      +  ## Directories, there should at least be a declaration for /var/www
      _
      -  <Directory "/var/www/html">
      +  <Directory "/var/www">
           Options Indexes FollowSymLinks MultiViews
           AllowOverride None
*******************************************
  Concat_fragment[sentry.softwareheritage.org_non-ssl-docroot] =>
   parameters =>
     content =>
      @@ -1,3 +1,3 @@
      _
         ## Vhost docroot
      -  DocumentRoot "/var/www/html"
      +  DocumentRoot "/var/www"
*******************************************
  Concat_fragment[sentry.softwareheritage.org_ssl-directories] =>
   parameters =>
     content =>
      @@ -1,6 +1,6 @@
      _
      -  ## Directories, there should at least be a declaration for /var/www/html
      +  ## Directories, there should at least be a declaration for /var/www
      _
      -  <Directory "/var/www/html">
      +  <Directory "/var/www">
           Options Indexes FollowSymLinks MultiViews
           AllowOverride None
*******************************************
  Concat_fragment[sentry.softwareheritage.org_ssl-docroot] =>
   parameters =>
     content =>
      @@ -1,3 +1,3 @@
      _
         ## Vhost docroot
      -  DocumentRoot "/var/www/html"
      +  DocumentRoot "/var/www"
*******************************************
- Concat_fragment[sentry.softwareheritage.org_ssl-proxy]
*******************************************
- Concat_fragment[sentry.softwareheritage.org_ssl-requestheader]
*******************************************
+ Concat_fragment[sentry.softwareheritage.org_ssl-rewrite] =>
   parameters =>
      "order": 190
      "tag": "25-sentry.softwareheritage.org_ssl.conf"
      "target": "25-sentry.softwareheritage.org_ssl.conf"
      "content": >>>
  ## Rewrite rules
  RewriteEngine On

  RewriteRule ^.*$ http://riverside.internal.admin.swh.network
<<<
*******************************************
  Concat_fragment[sentry.softwareheritage.org_ssl-ssl] =>
   parameters =>
     content =>
      @@ -2,7 +2,7 @@
         ## SSL directives
         SSLEngine on
      -  SSLCertificateFile      "/etc/ssl/certs/letsencrypt/sentry/cert.pem"
      -  SSLCertificateKeyFile   "/etc/ssl/certs/letsencrypt/sentry/privkey.pem"
      -  SSLCertificateChainFile "/etc/ssl/certs/letsencrypt/sentry/chain.pem"
      +  SSLCertificateFile      "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/cert.pem"
      +  SSLCertificateKeyFile   "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/privkey.pem"
      +  SSLCertificateChainFile "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/chain.pem"
         SSLProtocol             all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
         SSLCipherSuite          ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
*******************************************
+ Exec[letsencrypt certonly sentry.softwareheritage.org] =>
   parameters =>
      "command": "certbot --text --agree-tos --non-interactive certonly --rsa-key-size 4096 --cert-name 'sentry.softwareheritage.org' -d 'sentry.softwareheritage.org' --authenticator manual --preferred-challenges dns --manual-public-ip-logging-ok --manual-auth-hook '/usr/local/bin/letsencrypt_gandi_livedns auth' --manual-cleanup-hook '/usr/local/bin/letsencrypt_gandi_livedns cleanup' --deploy-hook '/usr/local/bin/letsencrypt_puppet_export'"
      "environment": []
      "path": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
      "provider": "shell"
      "unless": "/usr/local/sbin/letsencrypt-domain-validation /etc/letsencrypt/live/sentry.softwareheritage.org/cert.pem 'sentry.softwareheritage.org'"
*******************************************
- Exec[letsencrypt certonly sentry]
*******************************************
- File[/etc/apache2/mods-available/proxy.conf]
*******************************************
- File[/etc/apache2/mods-available/proxy.load]
*******************************************
- File[/etc/apache2/mods-available/proxy_http.load]
*******************************************
- File[/etc/apache2/mods-enabled/proxy.conf]
*******************************************
- File[/etc/apache2/mods-enabled/proxy.load]
*******************************************
- File[/etc/apache2/mods-enabled/proxy_http.load]
*******************************************
...
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/cert.pem] =>
   parameters =>
      "ensure": "present"
      "group": "root"
      "mode": "0644"
      "notify": ["Class[Apache::Service]"]
      "owner": "root"
      "source": "puppet:///le_certs/sentry.softwareheritage.org/cert.pem"
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/chain.pem] =>
   parameters =>
      "ensure": "present"
      "group": "root"
      "mode": "0644"
      "notify": ["Class[Apache::Service]"]
      "owner": "root"
      "source": "puppet:///le_certs/sentry.softwareheritage.org/chain.pem"
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/fullchain.pem] =>
   parameters =>
      "ensure": "present"
      "group": "root"
      "mode": "0644"
      "owner": "root"
      "source": "puppet:///le_certs/sentry.softwareheritage.org/fullchain.pem"
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/privkey.pem] =>
   parameters =>
      "ensure": "present"
      "group": "root"
      "mode": "0600"
      "notify": ["Class[Apache::Service]"]
      "owner": "root"
      "source": "puppet:///le_certs/sentry.softwareheritage.org/privkey.pem"
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org] =>
   parameters =>
      "ensure": "directory"
      "group": "root"
      "mode": "0755"
      "owner": "root"
*******************************************
- File[/etc/ssl/certs/letsencrypt/sentry/cert.pem]
*******************************************
- File[/etc/ssl/certs/letsencrypt/sentry/chain.pem]
*******************************************
- File[/etc/ssl/certs/letsencrypt/sentry/fullchain.pem]
*******************************************
- File[/etc/ssl/certs/letsencrypt/sentry/privkey.pem]
*******************************************
- File[/etc/ssl/certs/letsencrypt/sentry]
*******************************************
+ File[/var/www] =>
   parameters =>
      "ensure": "directory"
      "group": "root"
      "owner": "root"
*******************************************
+ Letsencrypt::Certonly[sentry.softwareheritage.org] =>
   parameters =>
      "additional_args": ["--authenticator manual", "--preferred-challenges dns", "--manual-public-ip-logging-ok", "--manual-auth-hook '/usr/local/bin/letsencrypt_gandi_livedns auth'", "--manual-cleanup-hook '/usr/local/bin/letsencrypt_gandi_livedns cleanup'", "--deploy-hook '/usr/local/bin/letsencrypt_puppet_export'"]
      "cert_name": "sentry.softwareheritage.org"
      "config_dir": "/etc/letsencrypt"
      "cron_hour": 4
      "cron_minute": 15
      "cron_monthday": ["*"]
      "custom_plugin": true
      "deploy_hook_commands": []
      "domains": ["sentry.softwareheritage.org"]
      "ensure": "present"
      "environment": []
      "key_size": 4096
      "letsencrypt_command": "certbot"
      "manage_cron": false
      "plugin": "standalone"
      "post_hook_commands": []
      "pre_hook_commands": []
      "suppress_cron_output": false
      "webroot_paths": []
*******************************************
- Letsencrypt::Certonly[sentry]
*******************************************
+ Profile::Letsencrypt::Certificate[sentry.softwareheritage.org] =>
   parameters =>
      "basename": "sentry.softwareheritage.org"
      "privkey_group": "root"
      "privkey_mode": "0600"
      "privkey_owner": "root"
*******************************************
- Profile::Letsencrypt::Certificate[sentry]
*******************************************
- Profile::Reverse_proxy[sentry]
*******************************************
*** End octocatalog-diff on pergamon.softwareheritage.org

rp1:

$ $SWH_PUPPET_ENVIRONMENT_HOME/bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to staging rp1.internal.admin.swh.network
...
diff origin/production/rp1.internal.admin.swh.network current/rp1.internal.admin.swh.network
*******************************************
+ Concat::Fragment[/etc/varnish/includes.vcl:sentry] =>
   parameters =>
      "content": "include \"includes/01_sentry.vcl\";"
      "order": "01"
      "target": "/etc/varnish/includes.vcl"
*******************************************
+ Concat::Fragment[/etc/varnish/includes.vcl:vhost_sentry.softwareheritage.org] =>
   parameters =>
      "content": "include \"includes/50_vhost_sentry.softwareheritage.org.vcl\";"
      "order": "50"
      "target": "/etc/varnish/includes.vcl"
*******************************************
+ Concat::Fragment[hitch::domain sentry.softwareheritage.org] =>
   parameters =>
      "notify": "Class[Hitch::Service]"
      "order": "10"
      "target": "/etc/hitch/hitch.conf"
      "content": >>>
pem-file = "/etc/hitch/sentry.softwareheritage.org.pem"
<<<
*******************************************
+ Concat::Fragment[sentry.softwareheritage.org cacert] =>
   parameters =>
      "notify": "Class[Hitch::Service]"
      "order": "03"
      "source": "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/chain.pem"
      "target": "/etc/hitch/sentry.softwareheritage.org.pem"
*******************************************
+ Concat::Fragment[sentry.softwareheritage.org cert] =>
   parameters =>
      "notify": "Class[Hitch::Service]"
      "order": "02"
      "source": "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/cert.pem"
      "target": "/etc/hitch/sentry.softwareheritage.org.pem"
*******************************************
+ Concat::Fragment[sentry.softwareheritage.org dhparams] =>
   parameters =>
      "notify": "Class[Hitch::Service]"
      "order": "04"
      "source": "/etc/hitch/dhparams.pem"
      "target": "/etc/hitch/sentry.softwareheritage.org.pem"
*******************************************
+ Concat::Fragment[sentry.softwareheritage.org key] =>
   parameters =>
      "notify": "Class[Hitch::Service]"
      "order": "01"
      "source": "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/privkey.pem"
      "target": "/etc/hitch/sentry.softwareheritage.org.pem"
*******************************************
+ Concat[/etc/hitch/sentry.softwareheritage.org.pem] =>
   parameters =>
      "backup": "puppet"
      "ensure": "present"
      "ensure_newline": false
      "force": false
      "format": "plain"
      "group": "_hitch"
      "mode": "0640"
      "notify": "Class[Hitch::Service]"
      "order": "alpha"
      "owner": "root"
      "path": "/etc/hitch/sentry.softwareheritage.org.pem"
      "replace": true
      "show_diff": true
      "warn": false
*******************************************
+ Concat_file[/etc/hitch/sentry.softwareheritage.org.pem] =>
   parameters =>
      "backup": "puppet"
      "ensure_newline": false
      "force": false
      "format": "plain"
      "group": "_hitch"
      "mode": "0640"
      "order": "alpha"
      "owner": "root"
      "replace": true
      "show_diff": true
      "tag": "_etc_hitch_sentry.softwareheritage.org.pem"
*******************************************
+ Concat_fragment[/etc/varnish/includes.vcl:sentry] =>
   parameters =>
      "content": "include \"includes/01_sentry.vcl\";"
      "order": "01"
      "tag": "_etc_varnish_includes.vcl"
      "target": "/etc/varnish/includes.vcl"
*******************************************
+ Concat_fragment[/etc/varnish/includes.vcl:vhost_sentry.softwareheritage.org] =>
   parameters =>
      "content": "include \"includes/50_vhost_sentry.softwareheritage.org.vcl\";"
      "order": "50"
      "tag": "_etc_varnish_includes.vcl"
      "target": "/etc/varnish/includes.vcl"
*******************************************
+ Concat_fragment[hitch::domain sentry.softwareheritage.org] =>
   parameters =>
      "order": "10"
      "tag": "_etc_hitch_hitch.conf"
      "target": "/etc/hitch/hitch.conf"
      "content": >>>
pem-file = "/etc/hitch/sentry.softwareheritage.org.pem"
<<<
*******************************************
+ Concat_fragment[sentry.softwareheritage.org cacert] =>
   parameters =>
      "order": "03"
      "source": "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/chain.pem"
      "tag": "_etc_hitch_sentry.softwareheritage.org.pem"
      "target": "/etc/hitch/sentry.softwareheritage.org.pem"
*******************************************
+ Concat_fragment[sentry.softwareheritage.org cert] =>
   parameters =>
      "order": "02"
      "source": "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/cert.pem"
      "tag": "_etc_hitch_sentry.softwareheritage.org.pem"
      "target": "/etc/hitch/sentry.softwareheritage.org.pem"
*******************************************
+ Concat_fragment[sentry.softwareheritage.org dhparams] =>
   parameters =>
      "order": "04"
      "source": "/etc/hitch/dhparams.pem"
      "tag": "_etc_hitch_sentry.softwareheritage.org.pem"
      "target": "/etc/hitch/sentry.softwareheritage.org.pem"
*******************************************
+ Concat_fragment[sentry.softwareheritage.org key] =>
   parameters =>
      "order": "01"
      "source": "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/privkey.pem"
      "tag": "_etc_hitch_sentry.softwareheritage.org.pem"
      "target": "/etc/hitch/sentry.softwareheritage.org.pem"
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/cert.pem] =>
   parameters =>
      "ensure": "present"
      "group": "root"
      "mode": "0644"
      "owner": "root"
      "source": "puppet:///le_certs/sentry.softwareheritage.org/cert.pem"
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/chain.pem] =>
   parameters =>
      "ensure": "present"
      "group": "root"
      "mode": "0644"
      "owner": "root"
      "source": "puppet:///le_certs/sentry.softwareheritage.org/chain.pem"
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/fullchain.pem] =>
   parameters =>
      "ensure": "present"
      "group": "root"
      "mode": "0644"
      "owner": "root"
      "source": "puppet:///le_certs/sentry.softwareheritage.org/fullchain.pem"
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/privkey.pem] =>
   parameters =>
      "ensure": "present"
      "group": "root"
      "mode": "0600"
      "owner": "root"
      "source": "puppet:///le_certs/sentry.softwareheritage.org/privkey.pem"
*******************************************
+ File[/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org] =>
   parameters =>
      "ensure": "directory"
      "group": "root"
      "mode": "0755"
      "owner": "root"
*******************************************
+ File[/etc/varnish/includes/01_sentry.vcl] =>
   parameters =>
      "group": "root"
      "mode": "0644"
      "notify": "Exec[vcl_reload]"
      "owner": "root"
      "content": >>>
# backend_default.vcl
#
# Default backend definition.
#
# File managed by puppet. All modifications will be lost.

backend sentry
{
    .host = "riverside.internal.admin.swh.network";
    .port = "80";
}
<<<
*******************************************
+ File[/etc/varnish/includes/50_vhost_sentry.softwareheritage.org.vcl] =>
   parameters =>
      "group": "root"
      "mode": "0644"
      "notify": "Exec[vcl_reload]"
      "owner": "root"
      "content": >>>
# vhost_sentry.softwareheritage.org.vcl
#
# Settings for the sentry.softwareheritage.org vhost
#
# File managed by puppet. All modifications will be lost.

sub vcl_recv {
    if (
        req.http.host ~ "^(?i)sentry\.softwareheritage\.org(:[0-9]+)?$"
    ) {
        var.set("known-vhost", "yes");
        if (std.port(server.ip) == 80) {
            set req.http.x-redir = "https://" + req.http.host + req.url;
            return(synth(850, "Moved permanently"));
        } else {
            set req.http.X-Forwarded-Proto = "https";
            set req.backend_hint = sentry;
        }
    }
}

sub vcl_deliver {
    if (
        req.http.host ~ "^(?i)sentry\.softwareheritage\.org(:[0-9]+)?$"
    ) {
        if (std.port(server.ip) != 80) {
            set resp.http.Strict-Transport-Security = "max-age=15768000;";
        }
    }
}

sub vcl_synth {
    if (
        req.http.host ~ "^(?i)sentry\.softwareheritage\.org(:[0-9]+)?$"
    ) {
        if (resp.status == 401) {
            set resp.http.WWW-Authenticate = "Basic";
            return(deliver);
        }
    }
}
<<<
*******************************************
+ Hitch::Domain[sentry.softwareheritage.org] =>
   parameters =>
      "cacert_source": "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/chain.pem"
      "cert_source": "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/cert.pem"
      "default": false
      "ensure": "present"
      "key_source": "/etc/ssl/certs/letsencrypt/sentry.softwareheritage.org/privkey.pem"
*******************************************
+ Profile::Hitch::Ssl_cert[sentry.softwareheritage.org] =>
   parameters =>
      "ssl_cert_name": "sentry.softwareheritage.org"
*******************************************
+ Profile::Letsencrypt::Certificate[sentry.softwareheritage.org] =>
   parameters =>
      "basename": "sentry.softwareheritage.org"
      "privkey_group": "root"
      "privkey_mode": "0600"
      "privkey_owner": "root"
*******************************************
+ Profile::Varnish::Vcl_include[sentry] =>
   parameters =>
      "basename": "sentry"
      "order": "01"
      "content": >>>
# backend_default.vcl
#
# Default backend definition.
#
# File managed by puppet. All modifications will be lost.

backend sentry
{
    .host = "riverside.internal.admin.swh.network";
    .port = "80";
}
<<<
*******************************************
+ Profile::Varnish::Vcl_include[vhost_sentry.softwareheritage.org] =>
   parameters =>
      "basename": "vhost_sentry.softwareheritage.org"
      "order": "50"
      "content": >>>
# vhost_sentry.softwareheritage.org.vcl
#
# Settings for the sentry.softwareheritage.org vhost
#
# File managed by puppet. All modifications will be lost.

sub vcl_recv {
    if (
        req.http.host ~ "^(?i)sentry\.softwareheritage\.org(:[0-9]+)?$"
    ) {
        var.set("known-vhost", "yes");
        if (std.port(server.ip) == 80) {
            set req.http.x-redir = "https://" + req.http.host + req.url;
            return(synth(850, "Moved permanently"));
        } else {
            set req.http.X-Forwarded-Proto = "https";
            set req.backend_hint = sentry;
        }
    }
}

sub vcl_deliver {
    if (
        req.http.host ~ "^(?i)sentry\.softwareheritage\.org(:[0-9]+)?$"
    ) {
        if (std.port(server.ip) != 80) {
            set resp.http.Strict-Transport-Security = "max-age=15768000;";
        }
    }
}

sub vcl_synth {
    if (
        req.http.host ~ "^(?i)sentry\.softwareheritage\.org(:[0-9]+)?$"
    ) {
        if (resp.status == 401) {
            set resp.http.WWW-Authenticate = "Basic";
            return(deliver);
        }
    }
}
<<<
*******************************************
+ Profile::Varnish::Vhost[sentry.softwareheritage.org] =>
   parameters =>
      "aliases": []
      "backend_http_host": "riverside.internal.admin.swh.network"
      "backend_http_port": "80"
      "backend_name": "sentry"
      "basic_auth": false
      "hsts_max_age": 15768000
      "order": "50"
      "servername": "sentry.softwareheritage.org"
      "websocket_support": false
*******************************************
+ Varnish::Vcl[/etc/varnish/includes/01_sentry.vcl] =>
   parameters =>
      "file": "/etc/varnish/includes/01_sentry.vcl"
      "content": >>>
# backend_default.vcl
#
# Default backend definition.
#
# File managed by puppet. All modifications will be lost.

backend sentry
{
    .host = "riverside.internal.admin.swh.network";
    .port = "80";
}
<<<
*******************************************
+ Varnish::Vcl[/etc/varnish/includes/50_vhost_sentry.softwareheritage.org.vcl] =>
   parameters =>
      "file": "/etc/varnish/includes/50_vhost_sentry.softwareheritage.org.vcl"
      "content": >>>
# vhost_sentry.softwareheritage.org.vcl
#
# Settings for the sentry.softwareheritage.org vhost
#
# File managed by puppet. All modifications will be lost.

sub vcl_recv {
    if (
        req.http.host ~ "^(?i)sentry\.softwareheritage\.org(:[0-9]+)?$"
    ) {
        var.set("known-vhost", "yes");
        if (std.port(server.ip) == 80) {
            set req.http.x-redir = "https://" + req.http.host + req.url;
            return(synth(850, "Moved permanently"));
        } else {
            set req.http.X-Forwarded-Proto = "https";
            set req.backend_hint = sentry;
        }
    }
}

sub vcl_deliver {
    if (
        req.http.host ~ "^(?i)sentry\.softwareheritage\.org(:[0-9]+)?$"
    ) {
        if (std.port(server.ip) != 80) {
            set resp.http.Strict-Transport-Security = "max-age=15768000;";
        }
    }
}

sub vcl_synth {
    if (
        req.http.host ~ "^(?i)sentry\.softwareheritage\.org(:[0-9]+)?$"
    ) {
        if (resp.status == 401) {
            set resp.http.WWW-Authenticate = "Basic";
            return(deliver);
        }
    }
}
<<<
*******************************************
*** End octocatalog-diff on rp1.internal.admin.swh.network

Diff Detail

Repository
rSPSITE puppet-swh-site
Branch
staging
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 26431
Build 41318: arc lint + arc unit

Event Timeline

ardumont created this revision.Jan 27 2022, 9:44 AM
Herald added a reviewer: System administrators. · View Herald TranscriptJan 27 2022, 9:44 AM
Harbormaster completed remote builds in B26370: Diff 25538.Jan 27 2022, 9:44 AM
ardumont requested review of this revision.Jan 27 2022, 9:44 AM
ardumont edited the test plan for this revision. (Show Details)Jan 27 2022, 11:36 AM
vsellier added a subscriber: vsellier.Jan 27 2022, 11:38 AM
vsellier added inline comments.
data/deployments/admin/common.yaml
39

The certificate entry should be renamed too on common.yaml from sentry to sentry.s.o
The renaming will be also needed for the rewitre_domains class

41

it seems the nginx is listening on the port 80 on riverside

data/hostname/pergamon.softwareheritage.org.yaml
81

should be http

ardumont marked 3 inline comments as done.Jan 27 2022, 11:46 AM
ardumont added inline comments.
data/common/common.yaml
1047

no need for this as this creates indirection.
let's arkham razor this, no indirection, the simpler the better.

ardumont updated this revision to Diff 25539.Jan 27 2022, 11:51 AM
ardumont edited the test plan for this revision. (Show Details)

Adapt according to review

Harbormaster completed remote builds in B26371: Diff 25539.Jan 27 2022, 11:51 AM
ardumont edited the test plan for this revision. (Show Details)Jan 27 2022, 11:54 AM
ardumont edited the test plan for this revision. (Show Details)Jan 27 2022, 2:02 PM
ardumont updated this revision to Diff 25540.Jan 27 2022, 2:03 PM
ardumont edited the test plan for this revision. (Show Details)

Use correct range of commits

Harbormaster completed remote builds in B26372: Diff 25540.Jan 27 2022, 2:04 PM
ardumont updated this revision to Diff 25541.Jan 27 2022, 2:07 PM

Add misses changes

Harbormaster completed remote builds in B26373: Diff 25541.Jan 27 2022, 2:07 PM
ardumont added inline comments.Jan 27 2022, 2:08 PM
data/deployments/admin/common.yaml
39

!

ardumont edited the test plan for this revision. (Show Details)Jan 27 2022, 2:09 PM
ardumont added a comment.Jan 27 2022, 3:43 PM

What's currently missing [1] in the state of the diff is the icinga check on the sentry service (moved to rp1).
With the proper means to define the http_uri value.

[1]

-object Service "sentry https certificate" {
-  import "generic-service"
-
-  host_name = "pergamon.softwareheritage.org"
-  check_command = "http"
-  vars.http_address = "sentry.softwareheritage.org"
-  vars.http_vhost = "sentry.softwareheritage.org"
-  vars.http_ssl = true
-  vars.http_sni = true
-  vars.http_certificate = 25
-}
-
-object Service "sentry https" {
-  import "generic-service"
-
-  host_name = "pergamon.softwareheritage.org"
-  check_command = "http"
-  vars.http_address = "sentry.softwareheritage.org"
-  vars.http_vhost = "sentry.softwareheritage.org"
-  vars.http_ssl = true
-  vars.http_sni = true
-  vars.http_uri = "/auth/login/swh/"
-  vars.http_string = "Sentry"
-}
data/deployments/admin/common.yaml
43

see D7045#183182

ardumont updated this revision to Diff 25548.Jan 27 2022, 4:10 PM
ardumont edited the test plan for this revision. (Show Details)

Rebase

Harbormaster completed remote builds in B26379: Diff 25548.Jan 27 2022, 4:10 PM
ardumont added a comment.Jan 28 2022, 9:56 AM

And here we go, diff update on its way:

--- /etc/icinga2/zones.d/master/exported-checks.conf    2022-01-27 15:15:29.132000000 +0000
+++ /tmp/puppet-file20220128-9315-qdoy9 2022-01-28 08:55:12.136000000 +0000
@@ -429,16 +429,6 @@
   vars.http_onredirect = "sticky"
 }

-object Service "sentry http redirect" {
-  import "generic-service"
-
-  host_name = "pergamon.softwareheritage.org"
-  check_command = "http"
-  vars.http_address = "sentry.softwareheritage.org"
-  vars.http_vhost = "sentry.softwareheritage.org"
-  vars.http_uri = "/"
-}
-
 object Service "swh sentry http redirect" {
   import "generic-service"

@@ -450,31 +440,6 @@
   vars.http_address = "sentry.softwareheritage.org"
 }

-object Service "sentry https certificate" {
-  import "generic-service"
-
-  host_name = "pergamon.softwareheritage.org"
-  check_command = "http"
-  vars.http_address = "sentry.softwareheritage.org"
-  vars.http_vhost = "sentry.softwareheritage.org"
-  vars.http_ssl = true
-  vars.http_sni = true
-  vars.http_certificate = 25
-}
-
-object Service "sentry https" {
-  import "generic-service"
-
-  host_name = "pergamon.softwareheritage.org"
-  check_command = "http"
-  vars.http_address = "sentry.softwareheritage.org"
-  vars.http_vhost = "sentry.softwareheritage.org"
-  vars.http_ssl = true
-  vars.http_sni = true
-  vars.http_uri = "/auth/login/swh/"
-  vars.http_string = "Sentry"
-}
-
 object Service "swh grafana https certificate" {
   import "generic-service"

@@ -498,6 +463,7 @@
   vars.http_uri = "/"
   vars.http_port = 443
   vars.http_vhost = "grafana.softwareheritage.org"
+  vars.http_string = "Grafana"
   vars.http_address = "grafana.softwareheritage.org"
   vars.http_onredirect = "sticky"
 }
@@ -525,6 +491,7 @@
   vars.http_uri = "/"
   vars.http_port = 443
   vars.http_vhost = "hedgedoc.softwareheritage.org"
+  vars.http_string = "Hedgedoc"
   vars.http_address = "hedgedoc.softwareheritage.org"
   vars.http_onredirect = "sticky"
 }
@@ -549,9 +516,10 @@
   check_command = "http"
   vars.http_sni = true
   vars.http_ssl = true
-  vars.http_uri = "/"
+  vars.http_uri = "/auth/login/swh/"
   vars.http_port = 443
   vars.http_vhost = "sentry.softwareheritage.org"
+  vars.http_string = "Sentry"
   vars.http_address = "sentry.softwareheritage.org"
   vars.http_onredirect = "sticky"
 }
ardumont added a comment.Jan 28 2022, 10:19 AM
  1. TODO: Don't forget to implement
  2. - check proxy_preserve_host => true is the default varnish behavior

It does, request received on riverside shows the host is the expected one (sentry.s.o) through a proxified request [1]

[1]

root@riverside:~# nc -l 0.0.0.0 80
GET /auth/login/swh/ HTTP/1.1
User-Agent: curl/7.64.0
Accept: */*
X-Forwarded-For: 10.168.100.29
X-Forwarded-Proto: https
host: sentry.softwareheritage.org
Accept-Encoding: gzip
X-Varnish: 32774
ardumont updated this revision to Diff 25549.Jan 28 2022, 10:20 AM
  • Update existing reverse proxy to allow specific icinga checks (check_uri, check_string)
  • nothing to do about the preserve_host setup as it's the default varnish behavior
Harbormaster completed remote builds in B26380: Diff 25549.Jan 28 2022, 10:20 AM
ardumont edited the test plan for this revision. (Show Details)Jan 28 2022, 10:23 AM
ardumont retitled this revision from wip: Migrate sentry node to admin vlan to Migrate sentry node to admin vlan.Jan 28 2022, 10:28 AM
ardumont edited the summary of this revision. (Show Details)
ardumont updated this revision to Diff 25550.Jan 28 2022, 10:38 AM
ardumont edited the summary of this revision. (Show Details)
ardumont edited the test plan for this revision. (Show Details)

Fix service port from 80 to 9000 (80 was actually the container service's port), the
exposed port on the machine is 9000.

Harbormaster completed remote builds in B26381: Diff 25550.Jan 28 2022, 10:38 AM
ardumont added inline comments.Jan 28 2022, 4:20 PM
data/common/common.yaml
1047

occam*

/me *grins*

ardumont mentioned this in T3891: Migrate sentry to admin vlan.Jan 28 2022, 5:34 PM
ardumont updated this revision to Diff 25598.Feb 1 2022, 2:35 PM

Rebase

Harbormaster completed remote builds in B26428: Diff 25598.Feb 1 2022, 2:35 PM
vsellier requested changes to this revision.Feb 1 2022, 2:37 PM
vsellier added inline comments.
data/hostname/pergamon.softwareheritage.org.yaml
81

it will need a new temporary firewall hole

This revision now requires changes to proceed.Feb 1 2022, 2:37 PM
ardumont updated this revision to Diff 25601.Feb 1 2022, 3:09 PM

Adapt according to review

Harbormaster completed remote builds in B26431: Diff 25601.Feb 1 2022, 3:09 PM
ardumont marked an inline comment as done.Feb 1 2022, 3:09 PM
ardumont added inline comments.
data/deployments/admin/common.yaml
43

no longer relevant comment (fixed)

vsellier accepted this revision.Feb 1 2022, 3:31 PM
This revision is now accepted and ready to land.Feb 1 2022, 3:31 PM
Closed by commit rSPSITE36667e66d210: Migrate sentry node to admin vlan (authored by ardumont). · Explain WhyFeb 1 2022, 3:32 PM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rSPSITE36667e66d210: Migrate sentry node to admin vlan.

Revision Contents
Changeset List

PathSize
data/
common/
6 lines
deployments/
admin/
16 lines
hostname/
2 lines
8 lines
subnets/
7 lines
manifests/
2 lines
site-modules/
profile/
manifests/
sentry/
swh/
deploy/
10 lines
role/
manifests/
3 lines

Diff 25601

View Options

data/common/common.yaml

Loading...
View Options

data/deployments/admin/common.yaml

Loading...
View Options

data/hostname/bojimans.internal.admin.swh.network.yaml

Loading...
View Options

data/hostname/pergamon.softwareheritage.org.yaml

Loading...
View Options

data/subnets/vagrant.yaml

Loading...
View Options

manifests/site.pp

Loading...
View Options

site-modules/profile/manifests/sentry/reverse_proxy.pp

Loading...
View Options

site-modules/profile/manifests/swh/deploy/reverse_proxy.pp

Loading...
View Options

site-modules/role/manifests/swh_sysadmin.pp

Loading...
About · Developer information · Legal