Page MenuHomeSoftware Heritage
Files F7123747

D8809.id31747.diff
No OneTemporary
Actions

D8809.id31747.diff
View Options

diff --git a/sysadm/images/gitlab-infrastructure.png b/sysadm/images/gitlab-infrastructure.png
new file mode 100644
index 0000000000000000000000000000000000000000..0000000000000000000000000000000000000000
GIT binary patch
literal 0
Hc$@<O00001
literal 0
Hc$@<O00001
diff --git a/sysadm/support-services/gitlab/index.rst b/sysadm/support-services/gitlab/index.rst
new file mode 100644
--- /dev/null
+++ b/sysadm/support-services/gitlab/index.rst
@@ -0,0 +1,10 @@
+.. _gitlab:
+
+Gitlab
+======
+
+.. toctree::
+ :maxdepth: 1
+
+ installation
+
diff --git a/sysadm/support-services/gitlab/installation.rst b/sysadm/support-services/gitlab/installation.rst
new file mode 100644
--- /dev/null
+++ b/sysadm/support-services/gitlab/installation.rst
@@ -0,0 +1,109 @@
+.. _gitlab_installation:
+
+.. admonition:: Intended audience
+ :class: important
+
+ sysadm staff members
+
+
+Gitlab installation
+===================
+
+The deployment of the gitlab instances is managed by:
+ - terraform for the the infrastructure part
+ - ArgoCd for the Gitlab deployment
+
+Each deployment relies on a dedicated `AKS` instance and a dedicated blob storage.
+
+Infrastructure
+--------------
+
+The `terraform code <https://gitlab.softwareheritage.org/infra/swh-sysadmin-provisioning/-/tree/master/azure/terraform/modules/gitlab>`__ is deploying
+and configuring a full, dedicated Azure Kubernetes Service matching this environment:
+
+.. figure:: ../../images/gitlab-infrastructure.png
+ :alt: Gitlab Infrastructure
+
+(`source <https://archive.softwareheritage.org/swh:1:dir:713a9ee3a2191aa2a5f386798442bc321c0b4747;origin=https://forge.softwareheritage.org/source/snippets.git;visit=swh:1:snp:7336142b5e1a03be4eedfa3f9edc4552297abd82;anchor=swh:1:rev:3045f3c9cca4ff0611725bae143c0ccc5027ed4c;path=/sysadmin/docs/gitlab/>`__)
+
+The output of terraform displays all the necessary information to connect to the kubernetes.
+It's also possible to get the information by executing those commands:
+
+.. code:: bash
+
+ terraform output gitlab-<instance>_aks_summary
+ terraform output --raw gitlab-<instance>_storage_summary
+
+Once the infrastructure is created and available, Gitlab is deployed by ArgoCD.
+It's done by deploying the `Gitlab Operator <https://gitlab.com/gitlab-org/cloud-native/gitlab-operator>`__
+and a gitlab configuration (for example, the
+`staging configuration <https://gitlab.softwareheritage.org/infra/ci-cd/k8s-swh-private-data/-/blob/master/gitlab-staging/gitlab-staging.yaml>`__ ).
+
+A couple of other components are also deployed in parallel to manage the monitoring, an additional ingress controller and cert-manager.
+The applications are available in the `k8s-clusters-conf repository <https://gitlab.softwareheritage.org/infra/ci-cd/k8s-clusters-conf/-/tree/master/argocd/applications/gitlab-staging>`__.
+
+Important components
+~~~~~~~~~~~~~~~~~~~~
+
+- The gitlab transactional data is stored on a couple of kubernetes PVs associated to their relative Azure volumes
+
+.. code:: bash
+
+ kubectl -n gitlab-system get pv 17:36:19
+ NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
+ pvc-38afb357-36e4-4661-9be6-011f58a24c95 64Gi RWO Retain Bound gitlab-system/repo-data-gitlab-gitaly-0 default 20d
+ pvc-46544290-0278-4b41-b616-7466cd5140f9 8Gi RWO Retain Bound gitlab-system/data-gitlab-postgresql-0 default 20d
+ pvc-988b8047-2ac9-426b-8792-37d749141657 8Gi RWO Retain Bound gitlab-system/redis-data-gitlab-redis-master-0 default 20d
+ pvc-c55bdefa-089b-4cda-b0d0-a1daafb6f37f 16Gi RWO Retain Bound monitoring/prometheus-gitlab-production-promethe-prometheus-db-prometheus-gitlab-production-promethe-prometheus-0 default 19d
+
+By default, the operator use a `Delete` reclaim policy which is risky. After the initial installation, it was manually changed to switch to a `Retain` policy
+
+- The gitlab assets (attachments, container registry data, ...) are stored in a dedicated object storage account
+
+Backups
+-------
+
+<Not yet available>
+
+.. note:: The backup will be stored on a minio instance installed on the Rocquencourt infrastructure.
+ see `#4645 <https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4645>`__
+
+Upgrades
+--------
+
+An upgrade of gitlab is done in two major steps (**always test in staging before upgrading the production environment**)
+
+It's important to follow the operatore releases to not be more than 3 minor versions late, or the migration will not be supported be the operator.
+
+- Step 1 `Upgrade the operator <https://gitlab.com/gitlab-org/cloud-native/gitlab-operator/-/blob/master/doc/operator_upgrades.md>`__
+
+ - Check the last `gitlab operator release <https://gitlab.com/gitlab-org/cloud-native/gitlab-operator/-/releases>`__ and the chart versions mapping
+ - Update the operator version in the `application declaration <https://gitlab.softwareheritage.org/infra/ci-cd/k8s-clusters-conf/-/blob/master/argocd/applications/gitlab-staging/gitlab-operator.yaml>`__. `Example of upgrade <https://gitlab.softwareheritage.org/infra/ci-cd/k8s-clusters-conf/-/commit/1a349433d45623128deabe7ca4ca57b740e16cba>`__
+ - Commit and push the change.
+ - Wait a couple of minutes to let ArgoCD applies the change
+ - A restart of the operator pod should happen (check the gitlab-controller-manager pod).
+ - Something like this should be visible in the operator logs:
+
+.. code:: shell
+
+ > kubernetes logs gitlab-controller-manager-xxxx-xxxx
+ Configuration error detected: chart version 6.5.1 not supported; please use one of the following: 6.5.2, 6.4.4, 6.3.5
+
+At this point, The gitlab operator is updated but the Gitlab upgrade itself is not yet started.
+
+- Step 2 `Upgrade the gitlab version <https://gitlab.com/gitlab-org/cloud-native/gitlab-operator/-/blob/master/doc/gitlab_upgrades.md>`__
+
+ - Follow the recommended upgrade path depending of the current deployed version of the gitlab instance and the allowed versions of the operator
+ - Update the chart version matching the desired gitlab version (should be present on the release page of the operator) in the
+ `gitlab resource declaration <https://gitlab.softwareheritage.org/infra/ci-cd/k8s-swh-private-data/-/blob/master/gitlab-staging/gitlab-staging.yaml>`__
+ - Commit and push the change
+ - Wait a couple of minutes to let ArgoCD applies the change
+ - A upgrade of the gitlab services should happen and the launch of a migration pod should be done
+ - After a couple of minutes (could be as long as an hour), the cluster should stabilize if everything is ok (no pending pods, no pods in error, ...)
+ - Check the deployed version and the operator logs to check if nothing is wrong
+
+.. code:: shell
+
+ > kubectl -n gitlab-system get gitlab 17:33:39
+ NAME STATUS VERSION
+ gitlab Running 6.5.2
diff --git a/sysadm/support-services/index.rst b/sysadm/support-services/index.rst
--- a/sysadm/support-services/index.rst
+++ b/sysadm/support-services/index.rst
@@ -1,7 +1,7 @@
Support services
================
+.. toctree::
+ :maxdepth: 2
-
-.. todo::
- This page is a work in progress.
+ gitlab/index

File Metadata

Mime Type
text/plain
Expires
Dec 19 2024, 11:30 PM (11 w, 3 d ago)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
3217599

Event Timeline

About · Developer information · Legal