Page MenuHomeSoftware Heritage

Migrate inventory to the admin vlan
Closed, MigratedEdits Locked

Description

Plan:

  • Update the inventory accordingly (new ip, depreciate the old ip)
  • D7018: Update the puppet configuration
  • On pergamon:
    • Stop puppet agent
  • On bojimans:
    • Stop puppet agent
    • Update the ip to the new vlan442 ip (192.168.50.60)
      • either terraform manifest if possible or directly through the proxmox ui
      • connect through ssh and adapt /etc/network/interfaces
      • also adapt hardware entry about network (proxmox ui) to change from vmbr0 to vmbr442
    • Update the hostname to bojimans.i.a.s.n
    • Remove the puppet certificates
    • Update the facts /etc/facter/facts.d/deployment.yaml to admin
    • poweroff + start back
    • Run puppet with puppet agent --test --fqdn bojimans.internal.admin.swh.network
  • on pergamon:
    • Run puppet agent
    • Decommission bojimans.i.s.o certificate
  • open firewall rule to allow traffic to the inventory node
  • Install necessary fact for cloud-init to stop tampering with /etc/hosts
  • Update inventory with reserved ip (rattach such ip to bojimans) [1]

Optionally:

  • Reference bojimans in our sysadm terraform admin manifest.

[1] https://inventory.internal.admin.swh.network/virtualization/virtual-machines/74/

Event Timeline

vsellier changed the task status from Open to Work in Progress.Jan 21 2022, 3:13 PM
vsellier triaged this task as Normal priority.
vsellier created this task.
ardumont updated the task description. (Show Details)
ardumont updated the task description. (Show Details)
ardumont updated the task description. (Show Details)

pergamon puppet agent --test appliance:

root@pergamon:~# puppet agent --test
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for pergamon.softwareheritage.org
Info: Applying configuration version '1643035704'
Notice: /Stage[main]/Profile::Ssh::Server/Concat[/etc/ssh/puppet_known_hosts/bojimans.internal.admin.swh.network.keys]/File[/etc/ssh/puppet_known_hosts/bojimans.internal.admin.swh.network.keys]/ensure: defined content as '{md5}8d4836a69bc23e3e88e81da7aff3486a'
Info: Concat[/etc/ssh/puppet_known_hosts/bojimans.internal.admin.swh.network.keys]: Scheduling refresh of Exec[update ssh_known_hosts]
Notice: /Stage[main]/Profile::Ssh::Server/Exec[update ssh_known_hosts]: Triggered 'refresh' from 1 event
Notice: /Stage[main]/Profile::Bind_server::Primary/Resource_record[bojimans.internal.admin.swh.network/PTR]/ensure: created
Notice: /Stage[main]/Profile::Bind_server::Primary/Resource_record[bojimans.internal.admin.swh.network/A]/ensure: created
Notice: /Stage[main]/Profile::Prometheus::Server/Profile::Prometheus::Scrape_config[bojimans.internal.admin.swh.network_statsd]/File[/etc/prometheus/exported-configs/bojimans.internal.admin.swh.network_statsd.yaml]/ensure: defined content
as '{md5}1e7ed70291bdc7917ddb084a106c2cd5'
Info: /Stage[main]/Profile::Prometheus::Server/Profile::Prometheus::Scrape_config[bojimans.internal.admin.swh.network_statsd]/File[/etc/prometheus/exported-configs/bojimans.internal.admin.swh.network_statsd.yaml]: Scheduling refresh of Exec[update-prometheus-config]
Notice: /Stage[main]/Profile::Prometheus::Server/Profile::Prometheus::Scrape_config[bojimans.internal.admin.swh.network_node]/File[/etc/prometheus/exported-configs/bojimans.internal.admin.swh.network_node.yaml]/ensure: defined content as '{md5}0a51d137ccdc32a3bafc4916937fd283'
Info: /Stage[main]/Profile::Prometheus::Server/Profile::Prometheus::Scrape_config[bojimans.internal.admin.swh.network_node]/File[/etc/prometheus/exported-configs/bojimans.internal.admin.swh.network_node.yaml]: Scheduling refresh of Exec[update-prometheus-config]
Notice: /Stage[main]/Profile::Prometheus::Server/Exec[update-prometheus-config]: Triggered 'refresh' from 2 events
Info: /Stage[main]/Profile::Prometheus::Server/Exec[update-prometheus-config]: Scheduling refresh of Service[prometheus]
Notice: /Stage[main]/Profile::Prometheus::Server/Service[prometheus]: Triggered 'refresh' from 1 event
Notice: /Stage[main]/Profile::Icinga2::Master/Icinga2::Object::Host[bojimans.internal.admin.swh.network]/Icinga2::Object[icinga2::object::Host::bojimans.internal.admin.swh.network]/Concat[/etc/icinga2/zones.d/master/bojimans.internal.admin.swh.network.conf]/File[/etc/icinga2/zones.d/master/bojimans.internal.admin.swh.network.conf]/ensure: defined content as '{md5}fd5d66314de8f7bf0124ce440ed0cded'
Info: Concat[/etc/icinga2/zones.d/master/bojimans.internal.admin.swh.network.conf]: Scheduling refresh of Class[Icinga2::Service]
Info: Class[Icinga2::Service]: Scheduling refresh of Service[icinga2]
Notice: /Stage[main]/Icinga2::Service/Service[icinga2]: Triggered 'refresh' from 1 event
Notice: Applied catalog in 48.16 seconds

If we do not forget about deploying the manifest changes... That should fairly help as well. ¯\_(ツ)_/¯

root@pergamon:~# /usr/local/bin/deploy.sh
HEAD is now at 1e47805 Add loader cvs sentry project token
Already up to date.
HEAD is now at 1e47805 Add loader cvs sentry project token
Already up to date.
root@pergamon:~# puppet agent --test
,Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for pergamon.softwareheritage.org
Info: Applying configuration version '1643037113'
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[netbox]/Exec[letsencrypt certonly netbox]/returns: executed successfully
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[inventory.internal.softwareheritage.org]/Exec[letsencrypt certonly inventory.internal.softwareheritage.org]/returns: executed successfully
Notice: /Stage[main]/Profile::Bind_server::Primary/Resource_record[inventory/CNAME]/data: data changed ['bojimans.internal.softwareheritage.org.'] to ['bojimans.internal.admin.swh.network.']
Notice: /Stage[main]/Profile::Bind_server::Primary/Resource_record[inventory-admin/CNAME]/ensure: created
Notice: Applied catalog in 69.12 seconds

Decomission bojimans.i.s.o

root@pergamon:~# swh-puppet-master-decommission bojimans.internal.softwareheritage.org
+ puppet node deactivate bojimans.internal.softwareheritage.org
Submitted 'deactivate node' for bojimans.internal.softwareheritage.org with UUID 93b95cdf-b723-46f8-a955-fb64001ce891
+ puppet node clean bojimans.internal.softwareheritage.org
Notice: Revoked certificate with serial 242
Notice: Removing file Puppet::SSL::Certificate bojimans.internal.softwareheritage.org at '/var/lib/puppet/ssl/ca/signed/bojimans.internal.softwareheritage.org.pem'
bojimans.internal.softwareheritage.org
+ puppet cert clean bojimans.internal.softwareheritage.org
Warning: `puppet cert` is deprecated and will be removed in a future release.
   (location: /usr/lib/ruby/vendor_ruby/puppet/application.rb:370:in `run')
Notice: Revoked certificate with serial 242
+ systemctl restart apache2
ardumont updated the task description. (Show Details)
ardumont claimed this task.
ardumont updated the task description. (Show Details)
ardumont moved this task from deployed/landed/monitoring to done on the System administration board.