Page MenuHomeSoftware Heritage
Create Task
Maniphest T4135

staging: Deploy graphql service
Closed, MigratedEdits Locked
Actions

Description

  • (vse) create a dedicated repository sysadm-environment/k8s-clusters-config:argo-workflows:/poc-argoworkflow (repo:branch:/directory)
  • (vse) Publish workflow code in ^ directory: "/workflows" [1]
  • (vse) Deployment doc on argoWorkflow
  • (ath) Deployment doc on argoCD
  • (ath) Publish argocd deployment code related to graphql in sysadm-environment/k8s-clusters-config:argocd:/
  • (ard) Deploy graphql locally (using minikube as cluster)
    • Create dockerhub image softwareheritage/graphql:latest
    • Deploy it from the swh-charts/swh-graphql helm chart
  • (ard) Deploy staging kubernetes cluster
    • Iinventory planification on new nodes and ip
    • D8152: Prepare puppet manifest to add correct role to new cluster nodes
    • D8155: Prepare terraform manifest for cluster and nodes creation
    • Deploy new graphql rancher cluster ^
  • (ard) Deploy graphql on the cluster
    • Manually Deploy graphql from the swh-charts/swh-graphql helm chart
    • D8159: Expose graphql instance through reverse proxy to one cluster member
    • 703bd22: Fix with varnish configuration so the graphql application is reached [2]
    • T4135#88666: Fight with graphql-worker{1,2} so they properly register on the cluster rancher
  • (ard/ath) T4413 Deploy argocd on admin vlan
  • (ard/ath) Decomission cluster-graphql to the benefit of cluster-graphql3
    • Register graphql-worker{3,2,1} on cluster-graphql3
    • Unregister graphql-worker0 from cluster-graphql (and clean up docker leftovers)
    • Register graphql-worker0 to cluster-graphql3
    • Destroy "cluster-graphql" (terraform)
  • rSPSITE0921c2b: Remove http basic auth currently configured which does not work [4]
  • Install load-balancer to access the graphql backend instead of a hard-coded member of the ingress (implementation needs to be determined first)
  • D8429: Monitoring / Alerting: Add a puppet icinga check to raise if the service is down
  • D8423: Update the documentation with new service url

[1] rSKCONFf89d3a7a6bc43290829b62cf139ebb54f31a1873

[2] https://graphql.staging.swh.network/

[3] https://argo-worker01.internal.admin.swh.network/

[4] The main page is served ok (with the proper creds). But after that, the actual

queries to the backend are failing to fetch the data through POST without
authentication (so pretty much graphql is useless with that http auth). Currently,
puppet is deactivated on rp0.staging with a hot-patched varnish to deactivate http
auth for graphql.

Revisions and Commits

rSPRE sysadm-provisioning
D8155rSPRE71e31c0fea42 Add new staging graphql-worker nodes
R260 Helm charts for swh packages
D8363R260:a3ebadbd9a17 staging/graphql/ingress: Fix double if instruction
D8352R260:5aed44119bb7 staging/graphql: Push issues to sentry
rSPSITE puppet-swh-site
D8450rSPSITEaf3bd6139e3e icinga: fix a typo on the graphql host declaration
D8432rSPSITEfa7e071f4f7f Add static check on the staging graphql instance 2/2
D8429rSPSITEdc47d24839ee Add static check on the staging graphql instance
D8159rSPSITE96f1f0574f2d graphql: Deploy reverse proxy to access the graphql instance
D8152rSPSITEa55f7245e268 Add new staging graphql-worker node with rancher_node role
rDDOC Development documentation
D8423rDDOC3108fa1e33fa service-urls: Update public with graphql instance

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
ardumont raised the priority of this task from Low to Normal.Jul 20 2022, 2:51 PM
ardumont mentioned this in rCJSWHe1dd4f976b9b: jobs/swh-packages: Install new build-and-push job to graphql project.Jul 20 2022, 2:52 PM
ardumont mentioned this in rDGQL2890f3480dbe: Makefile.local: Add targets to allow image build and push through ci.Jul 20 2022, 2:55 PM
ardumont mentioned this in rDGQL639ef3590f65: Makefile.local: Add missing clean target.Jul 20 2022, 3:01 PM
ardumont mentioned this in rDGQLa2d431be723f: Makefile.local: Add missing test target.Jul 20 2022, 3:15 PM
ardumont mentioned this in rDGQL3a1be9a2123c: Makefile: Revert unneeded docker targets.Jul 20 2022, 3:50 PM
ardumont mentioned this in rCJSWH7ee05db9554f: Revert "jobs/swh-packages: Install new build-and-push job to graphql project".
ardumont updated the task description. (Show Details)Jul 21 2022, 11:13 AM
ardumont updated the task description. (Show Details)
ardumont added a revision: D8152: Add new staging graphql-worker node with rancher_node role.Jul 21 2022, 12:49 PM
ardumont updated the task description. (Show Details)Jul 21 2022, 12:51 PM
ardumont added a commit: rSPSITEa55f7245e268: Add new staging graphql-worker node with rancher_node role.Jul 21 2022, 12:51 PM
ardumont updated the task description. (Show Details)Jul 21 2022, 12:52 PM
ardumont added a revision: D8155: Add new staging graphql-worker nodes.Jul 21 2022, 2:48 PM
ardumont updated the task description. (Show Details)Jul 21 2022, 2:54 PM
ardumont added a commit: rSPRE71e31c0fea42: Add new staging graphql-worker nodes.Jul 21 2022, 10:07 PM
ardumont updated the task description. (Show Details)Jul 21 2022, 10:08 PM
ardumont updated the task description. (Show Details)Jul 22 2022, 9:34 AM
ardumont added a revision: D8159: graphql: Deploy reverse proxy to access the graphql instance.Jul 22 2022, 5:34 PM
ardumont updated the task description. (Show Details)Jul 22 2022, 5:42 PM
ardumont mentioned this in rSENV1badaaae4ed5: Install new letsencrypt cert for vagrant test purpose.Jul 22 2022, 6:07 PM
ardumont added a commit: rSPSITE96f1f0574f2d: graphql: Deploy reverse proxy to access the graphql instance.Jul 25 2022, 9:54 AM
ardumont mentioned this in rSPSITEd41df6a65307: graphql: Add missing configuration keys.Jul 25 2022, 10:39 AM
ardumont mentioned this in R260:8fa68523c223: ingress: Fix missing namespace configuration.Jul 25 2022, 12:17 PM
ardumont updated the task description. (Show Details)Jul 25 2022, 3:04 PM
ardumont mentioned this in R260:0879c7ba2d64: Migrate values to values folder with proper names.Jul 26 2022, 12:09 PM
ardumont mentioned this in rSKCONF81301ca09c10: Update staging application to use the staging infra.Jul 26 2022, 12:27 PM
ardumont updated the task description. (Show Details)Jul 26 2022, 12:39 PM
ardumont mentioned this in R260:703bd229f90e: ingress: Allow empty host definition.Jul 26 2022, 4:57 PM
ardumont updated the task description. (Show Details)Jul 26 2022, 4:58 PM
ardumont updated the task description. (Show Details)Jul 27 2022, 9:26 AM
ardumont mentioned this in rSPSITE6e67c09fff24: Install argo-worker admin nodes as rancher node role.Jul 28 2022, 12:38 PM
ardumont added a comment.Jul 28 2022, 3:40 PM

Fight with graphql-worker{1,2} so they properly register on the cluster rancher

Status on the graphql cluster mess.
Only one node registered properly within the "cluster-graphql".
No logs whatsoever helps in determining why others are just stuck in registering and doing nothing but wait.

I've created another cluster "cluster-graphql2" (through terraform).
I've tried after cleaning up the graphql-worker{1,2,3} to register it to that new cluster.
Again to the same result without much clues to go on to debug.

I've then tried to remove that cluster altogether and that failed poorly as well.
It's just plainly refuse to destroy that cluster be it through terraform or through the rancher ui.

So in the end, fed up with banging my head against the wall, i've removed that state from terraform [1]

[1]

terraform state rm rancher2_cluster.cluster-graphql2
Removed rancher2_cluster.cluster-graphql2
Successfully removed 1 resource instance(s).
ardumont added a comment.Jul 28 2022, 3:43 PM

(w/ amadou) we've finally managed to create a new cluster graphql "cluster-graphql3" with a specific version (matching the one from the internship) [1]
(commits on their way)

We'll decomission the previous stuck "cluster-graphql" and migrate the node attached to that cluster to the one that works.
^ Once we are done with the argocd installation.

[1] kubernetes_version = "v1.21.12-rancher1-1"

ardumont updated the task description. (Show Details)Jul 28 2022, 3:47 PM
ardumont mentioned this in rSPRE57a18349ede9: Provision cluster-graphql2 cluster + register graphql-worker3 to it.Jul 28 2022, 3:52 PM
ardumont mentioned this in rSPRE471406d9354f: Provision cluster-graphql3 cluster.
ardumont mentioned this in rSPRE046cf0b6b0d9: rancher: Prepare reuse of rancher definition across vlans.
ardumont mentioned this in rSPRE15ab751f1b13: cluster-argo: Register new cluster and worker for argo tools.
ardumont mentioned this in rSPRE8d58591f60e0: cluster-graphql3: Move unregistered graphql-worker{1,2}.
ardumont mentioned this in rSPRE525ff43b7770: rancher: Unify config summary and register command output names.
ardumont mentioned this in rSPRE131a009a6e96: cluster-argo: Add new worker.
ardumont mentioned this in P1414 Ingress-dashboard-conf.Jul 28 2022, 4:55 PM
ardumont updated the task description. (Show Details)Jul 28 2022, 5:16 PM
ardumont updated the task description. (Show Details)Jul 28 2022, 5:19 PM
ardumont mentioned this in R260:91b772bbcd1d: staging: Scale to 3 replicas.Jul 28 2022, 5:20 PM
ardumont mentioned this in rSPRE3b233b002bfe: staging: Register graphql-worker0 with cluster-graphql3.Jul 28 2022, 5:27 PM
ardumont added a comment.EditedJul 28 2022, 5:29 PM

ArgoCD deployed graphql to staging [1]

TODO:

  • Install argocd through the admin rp (instead of the opened rule to allow access to it through vpn)
  • Make argocd deal with itself now that the bootstrap is done
  • Install load-balancer to access the graphql backend instead of a hard-coded member of the ingress (actually unsure whether < is necessary or not). [2]

[1]

[2] https://forge.softwareheritage.org/source/puppet-swh-site/browse/production/data/deployments/staging/common.yaml$152

ardumont mentioned this in rSKCONF6f3cfee25746: README: Update instructions for actual deployment on vlan admin.Jul 28 2022, 5:37 PM
ardumont mentioned this in rSKCONF3d4d61264761: Migrate staging graphql deployment to cluster-graphql3.
ardumont mentioned this in rSKCONF9879a46cdf89: Migration application to more explicit path names.
ardumont mentioned this in rSPREfd2108935b25: cluster-graphql: Destroy unresponsive cluster.Jul 28 2022, 5:40 PM
ardumont updated the task description. (Show Details)Jul 28 2022, 5:43 PM
ardumont mentioned this in rSKCONFaa5daeef8d2c: Clean up redundant files from swh-charts repository.Jul 28 2022, 5:58 PM
ardumont mentioned this in T4134: Package the graphql service.Jul 29 2022, 9:10 AM
ardumont updated the task description. (Show Details)Jul 29 2022, 2:29 PM
ardumont updated the task description. (Show Details)Jul 29 2022, 2:32 PM
ardumont mentioned this in rSPSITE0921c2b80207: staging/graphql: Deactivate basic authentication on graphql instance.Jul 29 2022, 3:12 PM
ardumont updated the task description. (Show Details)Jul 29 2022, 3:14 PM
ardumont changed the status of subtask T4413: Deploy argocd on admin vlan from Open to Work in Progress.Jul 29 2022, 6:20 PM
ardumont added a parent task: T4103: Setup a staging environment for GraphQL APIs.Aug 2 2022, 3:30 PM
jayeshv added a subscriber: jayeshv.Aug 2 2022, 3:32 PM
ardumont moved this task from in-progress to code-review/await-feedback/pause on the System administration board.Aug 2 2022, 3:34 PM
ardumont closed subtask T4413: Deploy argocd on admin vlan as Resolved.Aug 23 2022, 5:15 PM
ardumont added a revision: D8352: staging/graphql: Push issues to sentry.Aug 31 2022, 11:14 AM
ardumont mentioned this in R260:1c57606da1fc: staging: Declare ingress host.Aug 31 2022, 12:05 PM
ardumont added a revision: D8363: staging/graphql/ingress: Fix double if instruction.Aug 31 2022, 4:00 PM
ardumont added a commit: R260:5aed44119bb7: staging/graphql: Push issues to sentry.Aug 31 2022, 4:56 PM
ardumont added a commit: R260:a3ebadbd9a17: staging/graphql/ingress: Fix double if instruction.
ardumont added a revision: D8423: service-urls: Update public with graphql instance.Sep 8 2022, 3:22 PM
ardumont updated the task description. (Show Details)Sep 8 2022, 3:23 PM
ardumont added a commit: rDDOC3108fa1e33fa: service-urls: Update public with graphql instance.Sep 8 2022, 3:54 PM
ardumont updated the task description. (Show Details)Sep 8 2022, 4:44 PM
ardumont added a revision: D8429: Add static check on the staging graphql instance.Sep 8 2022, 5:24 PM
ardumont updated the task description. (Show Details)Sep 8 2022, 5:27 PM
ardumont added a commit: rSPSITEdc47d24839ee: Add static check on the staging graphql instance.Sep 8 2022, 5:32 PM
ardumont added a revision: D8432: Add static check on the staging graphql instance 2/2.Sep 8 2022, 6:07 PM
ardumont added a commit: rSPSITEfa7e071f4f7f: Add static check on the staging graphql instance 2/2.Sep 8 2022, 6:07 PM
ardumont updated the task description. (Show Details)Sep 8 2022, 6:21 PM
ardumont moved this task from code-review/await-feedback/pause to deployed/landed/monitoring on the System administration board.
ardumont closed this task as Resolved.Sep 9 2022, 9:41 AM
ardumont claimed this task.
ardumont moved this task from deployed/landed/monitoring to done on the System administration board.
ardumont added a parent task: T4523: Dynamic infrastructure.Sep 9 2022, 5:14 PM
vsellier added a revision: D8450: icinga: fix a typo on the graphql host declaration.Sep 12 2022, 4:34 PM
vsellier added a commit: rSPSITEaf3bd6139e3e: icinga: fix a typo on the graphql host declaration.Sep 12 2022, 4:47 PM
gitlab-migration changed the task status from Resolved to Migrated.Oct 19 2022, 6:06 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

gitlab-migration changed the status of subtask T4413: Deploy argocd on admin vlan from Resolved to Migrated.Oct 19 2022, 6:07 PM
About · Developer information · Legal