Closing this task as all the possible upgrade are done.
The delayed upgrades will be followed in dedicated task as it will be integradated in a more global task relative to the elastic infrastructure or the pergamon splitting task

Dec 21 2021, 4:06 PM · System administration (Component upgrades)

vsellier updated the task description for T3579: Meta-task: upgrade infrastructure to Debian Bullseye.

Dec 21 2021, 4:03 PM · System administration (Component upgrades)

vsellier closed T3807: Migrate kelvingrove (keycloak) to bullseye as Resolved.

root@kelvingrove:~# task=T3807
root@kelvingrove:/etc# puppet agent --disable "$task: dist-upgrade to bullseye"
root@kelvingrove:/etc# sed -i -e 's/buster/bullseye/;s,bullseye/updates,bullseye-security,' /etc/apt/sources.list.d/*

Dec 21 2021, 4:03 PM · System administration (Component upgrades)

vsellier closed T3807: Migrate kelvingrove (keycloak) to bullseye, a subtask of T3579: Meta-task: upgrade infrastructure to Debian Bullseye, as Resolved.

Dec 21 2021, 4:03 PM · System administration (Component upgrades)

vsellier added a comment to T3807: Migrate kelvingrove (keycloak) to bullseye.

The tests with vagrant are not showing any issue with puppet / keycloak after the upgrade so let's proceed to the upgrade.

Dec 21 2021, 3:38 PM · System administration (Component upgrades)

vsellier changed the status of T3807: Migrate kelvingrove (keycloak) to bullseye, a subtask of T3579: Meta-task: upgrade infrastructure to Debian Bullseye, from Open to Work in Progress.

Dec 21 2021, 2:13 PM · System administration (Component upgrades)

vsellier changed the status of T3807: Migrate kelvingrove (keycloak) to bullseye from Open to Work in Progress.

Dec 21 2021, 2:13 PM · System administration (Component upgrades)

vsellier updated the task description for T3579: Meta-task: upgrade infrastructure to Debian Bullseye.

Dec 21 2021, 12:13 PM · System administration (Component upgrades)

vsellier updated the task description for T3579: Meta-task: upgrade infrastructure to Debian Bullseye.

Dec 21 2021, 12:13 PM · System administration (Component upgrades)

vsellier closed T3801: Migrate production database servers to bullseye as Resolved.

Dec 21 2021, 12:12 PM · System administration (Component upgrades)

vsellier closed T3801: Migrate production database servers to bullseye, a subtask of T3579: Meta-task: upgrade infrastructure to Debian Bullseye, as Resolved.

Dec 21 2021, 12:12 PM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 21 2021, 12:09 PM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 21 2021, 12:09 PM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 21 2021, 11:49 AM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 21 2021, 10:41 AM · System administration (Component upgrades)

vsellier added a comment to T3801: Migrate production database servers to bullseye.

A memory alert is logged on the idrac

	Correctable memory error logging disabled for a memory device at location DIMM_A9. 	Fri 17 Dec 2021 16:15:39

We will have to monitor in the future to check if this memory dimm has some weaknesses

Dec 21 2021, 10:13 AM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 21 2021, 9:59 AM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 21 2021, 9:34 AM · System administration (Component upgrades)

vsellier added a comment to T3801: Migrate production database servers to bullseye.

on moma:

puppet disabled

root@moma:/etc/softwareheritage/storage# puppet agent --disable 'T3801 upgrade database servers'

storage configuration update to use belvedere database and service restarted

Dec 21 2021, 9:24 AM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 21 2021, 9:16 AM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 21 2021, 9:06 AM · System administration (Component upgrades)

vsellier renamed T3801: Migrate production database servers to bullseye from Migrate database servers to bullseye to Migrate production database servers to bullseye.

Dec 21 2021, 9:04 AM · System administration (Component upgrades)

vsellier changed the status of T3801: Migrate production database servers to bullseye, a subtask of T3579: Meta-task: upgrade infrastructure to Debian Bullseye, from Open to Work in Progress.

Dec 21 2021, 9:04 AM · System administration (Component upgrades)

vsellier changed the status of T3801: Migrate production database servers to bullseye from Open to Work in Progress.

Dec 21 2021, 9:04 AM · System administration (Component upgrades)

Dec 17 2021

vsellier closed T3813: Migrate the staging database server to bullseye as Resolved.

workers:

Before the migration

root@pergamon:~# clush -b -w @staging-workers 'set -e; puppet agent --disable "T3812"; puppet agent --disable T3771; cd /etc/systemd/system/multi-user.target.wants; for unit in swh-worker@*; do systemctl disable $unit; done; systemctl stop --no-block swh-worker@*; sleep 300; systemctl kill swh-worker@* -s 9'

Dec 17 2021, 4:14 PM · System administration (Component upgrades)

vsellier closed T3813: Migrate the staging database server to bullseye, a subtask of T3801: Migrate production database servers to bullseye, as Resolved.

Dec 17 2021, 4:14 PM · System administration (Component upgrades)

vsellier updated the task description for T3813: Migrate the staging database server to bullseye.

Dec 17 2021, 2:25 PM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 17 2021, 2:17 PM · System administration (Component upgrades)

vsellier updated the task description for T3801: Migrate production database servers to bullseye.

Dec 17 2021, 2:17 PM · System administration (Component upgrades)

vsellier changed the status of T3813: Migrate the staging database server to bullseye from Open to Work in Progress.

Dec 17 2021, 2:16 PM · System administration (Component upgrades)

Dec 15 2021

olasd added a comment to T3808: Migrate pergamon to bullseye.

It has a bunch of reverse proxies and direct service https hosts (jenkins, debian repo, annex.softwareheritage.org, bitbucket-archive, ...), as well, which could move to a, well, reverse proxy :-)

Dec 15 2021, 10:44 AM · System administration (Component upgrades)

olasd added a comment to T3808: Migrate pergamon to bullseye.

It'd be nice to move grafana (which is a public-access service with a fairly large attack surface) to a separate VM that's not our (internal) puppet master.

Dec 15 2021, 10:40 AM · System administration (Component upgrades)

ardumont updated the task description for T3579: Meta-task: upgrade infrastructure to Debian Bullseye.

Dec 15 2021, 10:18 AM · System administration (Component upgrades)

ardumont added a subtask for T3579: Meta-task: upgrade infrastructure to Debian Bullseye: T3706: (Minor) upgrade Opnsense to 21.1.9.

Dec 15 2021, 10:17 AM · System administration (Component upgrades)

ardumont added a parent task for T3706: (Minor) upgrade Opnsense to 21.1.9: T3579: Meta-task: upgrade infrastructure to Debian Bullseye.

Dec 15 2021, 10:17 AM · System administration (Component upgrades)

ardumont added a subtask for T3579: Meta-task: upgrade infrastructure to Debian Bullseye: T3760: (Major) opnsense upgrade to 21.1.6.

Dec 15 2021, 10:17 AM · System administration (Component upgrades)

ardumont added a parent task for T3760: (Major) opnsense upgrade to 21.1.6: T3579: Meta-task: upgrade infrastructure to Debian Bullseye.

Dec 15 2021, 10:17 AM · System administration (Component upgrades)

ardumont updated the task description for T3579: Meta-task: upgrade infrastructure to Debian Bullseye.