Page MenuHomeSoftware Heritage

Update rancher cluster to kubernetes 1.22
Closed, MigratedEdits Locked

Description

Subtask dedicated to the rancher's kubernetes cluster upgrade

Event Timeline

vsellier triaged this task as Normal priority.Jun 29 2022, 9:55 AM
vsellier created this task.
vsellier renamed this task from Update AKS cluster to kubernetes 1.22 to Update rancher cluster to kubernetes 1.22.Jul 5 2022, 5:38 PM
vsellier changed the task status from Open to Work in Progress.Jul 7 2022, 6:22 PM
vsellier moved this task from Backlog to in-progress on the System administration board.
vsellier moved this task from in-progress to done on the System administration board.

The kubernetes upgrade was launched through the azure portal (it's also possible to trigger it with the az command line)
Everything looks fine:

  • A new node with the version 1.22.6 was triggerd
kubectl get pods -o wide; echo; kubectl get nodes -o wide
NAME                               READY   STATUS    RESTARTS      AGE   IP            NODE                              NOMINATED NODE   READINESS GATES
debian                             1/1     Running   1 (23m ago)   27m   10.244.0.63   aks-default-36212332-vmss000000   <none>           <none>
rancher-59f4c74c6f-5vlq6           1/1     Running   0             91m   10.244.0.59   aks-default-36212332-vmss000000   <none>           <none>
rancher-59f4c74c6f-92txx           1/1     Running   0             90m   10.244.0.60   aks-default-36212332-vmss000000   <none>           <none>
rancher-59f4c74c6f-cfshs           1/1     Running   0             91m   10.244.0.58   aks-default-36212332-vmss000000   <none>           <none>
rancher-webhook-6958cfcddf-2gjwn   1/1     Running   0             85d   10.244.0.26   aks-default-36212332-vmss000000   <none>           <none>

NAME                              STATUS     ROLES   AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
aks-default-36212332-vmss000000   Ready      agent   85d   v1.21.9   10.240.0.5    <none>        Ubuntu 18.04.6 LTS   5.4.0-1073-azure   containerd://1.4.12+azure-3
aks-default-36212332-vmss000001   NotReady   agent   19s   v1.22.6   10.240.0.7    <none>        Ubuntu 18.04.6 LTS   5.4.0-1085-azure   containerd://1.5.11+azure-2
  • The old 1.21.9 node was drained and all the load moved to the new node
kubectl get pods -o wide; echo; kubectl get nodes -o wide                                                       18:24:53
NAME                               READY   STATUS    RESTARTS   AGE     IP            NODE                              NOMINATED NODE   READINESS GATES
rancher-59f4c74c6f-dxxh4           1/1     Running   0          3m22s   10.244.1.8    aks-default-36212332-vmss000001   <none>           <none>
rancher-59f4c74c6f-wdrrq           1/1     Running   0          3m22s   10.244.1.6    aks-default-36212332-vmss000001   <none>           <none>
rancher-59f4c74c6f-zhwdn           1/1     Running   0          3m21s   10.244.1.3    aks-default-36212332-vmss000001   <none>           <none>
rancher-webhook-6958cfcddf-fcp75   1/1     Running   0          3m20s   10.244.1.16   aks-default-36212332-vmss000001   <none>           <none>

NAME                              STATUS                     ROLES   AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
aks-default-36212332-vmss000000   Ready,SchedulingDisabled   agent   85d     v1.21.9   10.240.0.5    <none>        Ubuntu 18.04.6 LTS   5.4.0-1073-azure   containerd://1.4.12+azure-3
aks-default-36212332-vmss000001   Ready                      agent   3m56s   v1.22.6   10.240.0.7    <none>        Ubuntu 18.04.6 LTS   5.4.0-1085-azure   containerd://1.5.11+azure-2
  • unexpected A new 1.22.6 node was recreate, I think to reset the node pool to 0 and the first one deleted
kubectl get pods -o wide; echo; kubectl get nodes -o wide                                                    18:30:35
NAME                               READY   STATUS              RESTARTS   AGE   IP           NODE                              NOMINATED NODE   READINESS GATES
rancher-59f4c74c6f-7rksq           0/1     ContainerCreating   0          41s   <none>       aks-default-36212332-vmss000000   <none>           <none>
rancher-59f4c74c6f-ptxbh           0/1     ContainerCreating   0          40s   <none>       aks-default-36212332-vmss000000   <none>           <none>
rancher-59f4c74c6f-xlz4j           0/1     ContainerCreating   0          39s   <none>       aks-default-36212332-vmss000000   <none>           <none>
rancher-webhook-6958cfcddf-cqz42   1/1     Running             0          40s   10.244.2.5   aks-default-36212332-vmss000000   <none>           <none>

NAME                              STATUS   ROLES   AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
aks-default-36212332-vmss000000   Ready    agent   98s   v1.22.6   10.240.0.5    <none>        Ubuntu 18.04.6 LTS   5.4.0-1085-azure   containerd://1.5.11+azure-2

At the end, everything is ok:

kubectl get pods -o wide; echo; kubectl get nodes -o wide                                                           18:30:37
NAME                               READY   STATUS    RESTARTS   AGE     IP            NODE                              NOMINATED NODE   READINESS GATES
rancher-59f4c74c6f-7rksq           1/1     Running   0          7m32s   10.244.2.9    aks-default-36212332-vmss000000   <none>           <none>
rancher-59f4c74c6f-ptxbh           1/1     Running   0          7m31s   10.244.2.6    aks-default-36212332-vmss000000   <none>           <none>
rancher-59f4c74c6f-xlz4j           1/1     Running   0          7m30s   10.244.2.11   aks-default-36212332-vmss000000   <none>           <none>
rancher-webhook-6958cfcddf-cqz42   1/1     Running   0          7m31s   10.244.2.5    aks-default-36212332-vmss000000   <none>           <none>

NAME                              STATUS   ROLES   AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
aks-default-36212332-vmss000000   Ready    agent   8m29s   v1.22.6   10.240.0.5    <none>        Ubuntu 18.04.6 LTS   5.4.0-1085-azure   containerd://1.5.11+azure-2