Page MenuHomeSoftware Heritage
Feed Advanced Search

Oct 4 2022

vsellier triaged T4603: move graphql to a sub url instead of the standalone vhost as Normal priority.
Oct 4 2022, 6:20 PM · System administration, GraphQL API
vsellier closed T4132: Add the graphql service in the docker environment, a subtask of T4131: Graphql service in staging, as Resolved.
Oct 4 2022, 6:16 PM · System administration, GraphQL API
vsellier closed T4132: Add the graphql service in the docker environment as Resolved.
Oct 4 2022, 6:16 PM · System administration, GraphQL API
vsellier updated the task description for T4132: Add the graphql service in the docker environment.
Oct 4 2022, 6:16 PM · System administration, GraphQL API
vsellier closed T4497: [sentry] Out of disk space as Resolved.

Closing as there is no alerts since almost one month

Oct 4 2022, 6:15 PM · Sentry, System administration
vsellier committed rSKCONFee77cc15f039: argocd: ignore argocd-cm and argocd-rabc-cm changes (authored by vsellier).
argocd: ignore argocd-cm and argocd-rabc-cm changes
Oct 4 2022, 4:17 PM
vsellier closed T4534: Evaluate MetalLB as inbound loadbalancer, a subtask of T4523: Dynamic infrastructure, as Resolved.
Oct 4 2022, 3:03 PM · meta-task, System administration
vsellier closed T4534: Evaluate MetalLB as inbound loadbalancer as Resolved.

regarding the last tests, we can start using it to battle proof its usage.
I found in several documentations where it's the tool recommended to manage load balancing on on-premise kubernetes deployments, for example: https://kubernetes.github.io/ingress-nginx/deploy/baremetal/#a-pure-software-solution-metallb

Oct 4 2022, 3:03 PM · System administration
vsellier added a comment to T4385: Federate prometheus instances through thanos.

thanos exposed on the production cluster with this commit: rSPRE8fade05553ed4a01e54e1b8481150c0e055e3f34

Oct 4 2022, 2:32 PM · meta-task, System administration, Roadmap 2022
vsellier committed rSPRE8fade05553ed: Export the grpc port of thanos through an ingress configuration (authored by vsellier).
Export the grpc port of thanos through an ingress configuration
Oct 4 2022, 2:20 PM
vsellier committed R260:e79a98efcf72: cassandra-replay: increase revision replayer count (authored by vsellier).
cassandra-replay: increase revision replayer count
Oct 4 2022, 9:32 AM

Oct 3 2022

vsellier created P1468 (An Untitled Masterwork).
Oct 3 2022, 4:03 PM
vsellier committed rSPREfa50a5ff1cb2: k8s-archive-production: active thanos sidecard and services (authored by vsellier).
k8s-archive-production: active thanos sidecard and services
Oct 3 2022, 3:43 PM
vsellier committed R260:b4067e7cf048: cassandra-replay: reduce load on production cluster during metallb tests (authored by vsellier).
cassandra-replay: reduce load on production cluster during metallb tests
Oct 3 2022, 3:05 PM
vsellier requested review of D8599: k8s-archive-production: Add an internal ingress to expose reaper webui.
Oct 3 2022, 1:55 PM
vsellier added a revision to T4458: Test reaper to automate the cassandra repair actions: D8599: k8s-archive-production: Add an internal ingress to expose reaper webui.
Oct 3 2022, 1:55 PM · System administration
vsellier committed rSKCONF110115991c54: reaper: Add an ingress to internally expose the webui (authored by vsellier).
reaper: Add an ingress to internally expose the webui
Oct 3 2022, 1:49 PM

Oct 1 2022

vsellier committed R260:de634df57950: cassandra-replay: redispatch replayers after snapshot catchup (authored by vsellier).
cassandra-replay: redispatch replayers after snapshot catchup
Oct 1 2022, 12:06 PM

Sep 30 2022

vsellier committed rSKCONF39fd005dd4af: archive-production: configure metallb to allow several service on same ip (authored by vsellier).
archive-production: configure metallb to allow several service on same ip
Sep 30 2022, 8:01 PM
vsellier committed rSPRE3147328ddfa6: k8s-archive-production: configure thanos sidecar to push on azure (authored by vsellier).
k8s-archive-production: configure thanos sidecar to push on azure
Sep 30 2022, 5:55 PM
vsellier closed T4461: Move argocd to a private admin url as Resolved.
Sep 30 2022, 5:27 PM · System administration
vsellier committed rSPSITE4b2879647a52: argocd: fix the missing dot in the dns declaration (authored by vsellier).
argocd: fix the missing dot in the dns declaration
Sep 30 2022, 5:22 PM
vsellier updated the task description for T4385: Federate prometheus instances through thanos.
Sep 30 2022, 3:50 PM · meta-task, System administration, Roadmap 2022
vsellier closed D8577: Disable ping on hosts/ips managed by metallb.
Sep 30 2022, 9:28 AM
vsellier committed rSPSITEbee3d6dc026c: Disable ping on hosts/ips managed by metallb (authored by vsellier).
Disable ping on hosts/ips managed by metallb
Sep 30 2022, 9:28 AM

Sep 29 2022

vsellier requested review of D8577: Disable ping on hosts/ips managed by metallb.
Sep 29 2022, 10:42 AM
vsellier added a revision to T4534: Evaluate MetalLB as inbound loadbalancer: D8577: Disable ping on hosts/ips managed by metallb.
Sep 29 2022, 10:42 AM · System administration

Sep 28 2022

vsellier committed rSKCONF01abd45d6427: argocd: Force the redirect to https for the internal ingress (authored by vsellier).
argocd: Force the redirect to https for the internal ingress
Sep 28 2022, 7:23 PM
vsellier committed rSKCONF246fb2ffa395: argocd: Delete public ingress (authored by vsellier).
argocd: Delete public ingress
Sep 28 2022, 7:23 PM
vsellier closed D8567: argocd: Remove public site.
Sep 28 2022, 7:08 PM
vsellier committed rSPSITEdd8df9301d75: argocd: Remove public site (authored by vsellier).
argocd: Remove public site
Sep 28 2022, 7:08 PM
vsellier updated the test plan for D8567: argocd: Remove public site.
Sep 28 2022, 5:48 PM
vsellier updated the diff for D8567: argocd: Remove public site.
  • Add monitoring of the internal service
Sep 28 2022, 5:47 PM
vsellier requested review of D8567: argocd: Remove public site.
Sep 28 2022, 5:29 PM
vsellier added a revision to T4461: Move argocd to a private admin url: D8567: argocd: Remove public site.
Sep 28 2022, 5:29 PM · System administration
vsellier added a comment to T4461: Move argocd to a private admin url.

Argo is now reachable on the internal network at https://argocd.internal.admin.swh.network/ but it uses an self-signed certificated

Sep 28 2022, 12:28 PM · System administration
vsellier committed rSKCONF3bf578f9b891: Use the official kubernetes nginx ingress controller (authored by vsellier).
Use the official kubernetes nginx ingress controller
Sep 28 2022, 12:01 PM
vsellier committed rSKCONF4b7b571bba07: argocd: don't try to override the default nginx ingress service (authored by vsellier).
argocd: don't try to override the default nginx ingress service
Sep 28 2022, 10:51 AM
vsellier closed D8559: argocd: Prepare the configuration to migrate to the internal admin network.
Sep 28 2022, 10:35 AM
vsellier committed rSPSITE73ae99dafa58: argocd: Prepare the configuration to migrate to the internal admin network (authored by vsellier).
argocd: Prepare the configuration to migrate to the internal admin network
Sep 28 2022, 10:35 AM
vsellier updated the diff for D8559: argocd: Prepare the configuration to migrate to the internal admin network.

rebase

Sep 28 2022, 10:29 AM
vsellier committed rSPRE6b8883e1f458: argocd: disable default rancher ingress controller (authored by vsellier).
argocd: disable default rancher ingress controller
Sep 28 2022, 10:27 AM
vsellier requested review of D8559: argocd: Prepare the configuration to migrate to the internal admin network.
Sep 28 2022, 10:16 AM
vsellier added a revision to T4461: Move argocd to a private admin url: D8559: argocd: Prepare the configuration to migrate to the internal admin network.
Sep 28 2022, 10:16 AM · System administration
vsellier added a comment to T4461: Move argocd to a private admin url.

hum I completely forgot we'll have to think how to manage the ssl termination.
As is, there is not cert manager deployed on the cluster and I'm not sure we want to do it.

Sep 28 2022, 10:09 AM · System administration
vsellier committed rSKCONF311693349c85: argocd: migrate to an internal domain (authored by vsellier).
argocd: migrate to an internal domain
Sep 28 2022, 9:56 AM
vsellier changed the status of T4461: Move argocd to a private admin url from Open to Work in Progress.

As metallb is configured, the internal domain can used the VIP

Sep 28 2022, 9:56 AM · System administration
vsellier added a comment to D8551: first add forge now requests added to changelog.

Sure there is (and it must have) nothing secret logged in the task.

Sep 28 2022, 9:29 AM

Sep 27 2022

vsellier committed rSKCONF8862c36c5e25: metallb: ignore fields updated during runtime (authored by vsellier).
metallb: ignore fields updated during runtime
Sep 27 2022, 11:40 PM
vsellier added a comment to T4534: Evaluate MetalLB as inbound loadbalancer.

A new test with a node completely down, it seems it recover after ~5mn which looks related to some cache expiracy somewhere

Tue Sep 27 16:31:59 UTC 2022
60 bytes from 2e:81:20:19:02:4a (192.168.100.119): index=2926 time=1.679 msec
Tue Sep 27 16:32:01 UTC 2022
Timeout
Timeout
Tue Sep 27 16:32:03 UTC 2022
...
Tue Sep 27 16:37:56 UTC 2022
Timeout
Timeout
Tue Sep 27 16:37:58 UTC 2022
60 bytes from 2e:84:a0:44:9e:c9 (192.168.100.119): index=2927 time=814.409 msec
60 bytes from 2e:84:a0:44:9e:c9 (192.168.100.119): index=2928 time=864.574 usec
60 bytes from 2e:84:a0:44:9e:c9 (192.168.100.119): index=2929 time=973.083 msec
60 bytes from 2e:84:a0:44:9e:c9 (192.168.100.119): index=2930 time=32.151 msec
Sep 27 2022, 6:42 PM · System administration
vsellier added a comment to T4534: Evaluate MetalLB as inbound loadbalancer.

If a node is drained out of the cluster, the rebalancing occurs in ~10s which it's what it's announced in the documentation

Tue Sep 27 16:17:21 UTC 2022
60 bytes from 2e:84:a0:44:9e:c9 (192.168.100.119): index=1985 time=1.710 msec
60 bytes from 2e:84:a0:44:9e:c9 (192.168.100.119): index=1986 time=1.376 msec
Tue Sep 27 16:17:23 UTC 2022
Timeout
Tue Sep 27 16:17:25 UTC 2022
Timeout
Timeout
Tue Sep 27 16:17:27 UTC 2022
Timeout
Timeout
Tue Sep 27 16:17:29 UTC 2022
Timeout
Timeout
Tue Sep 27 16:17:31 UTC 2022
Timeout
Timeout
Tue Sep 27 16:17:33 UTC 2022
Timeout
Timeout
60 bytes from 2e:81:20:19:02:4a (192.168.100.119): index=1987 time=669.150 msec
Tue Sep 27 16:17:35 UTC 2022
60 bytes from 2e:81:20:19:02:4a (192.168.100.119): index=1988 time=959.452 usec
Sep 27 2022, 6:19 PM · System administration
vsellier added a comment to T4534: Evaluate MetalLB as inbound loadbalancer.

With the ingress controller correctly configured and an ingress declared, everything seems to work correctly:

vsellier@pergamon ~ % cat test-ingress.txt
GET /graphql/ HTTP/1.0
Host: archive.softwareheritage.org
Sep 27 2022, 6:04 PM · System administration
vsellier committed rSKCONF27bee789d015: production: Ensure the ingress class created by nginx is the default (authored by vsellier).
production: Ensure the ingress class created by nginx is the default
Sep 27 2022, 5:31 PM
vsellier committed R260:581d22afb81d: production: enabled graphql ingress (authored by vsellier).
production: enabled graphql ingress
Sep 27 2022, 4:40 PM
vsellier committed R260:f4cb0216d314: production: Add missing configuration entry to deploy graphql (authored by vsellier).
production: Add missing configuration entry to deploy graphql
Sep 27 2022, 4:37 PM
vsellier committed R260:1f07eaf3cece: production add mandatory values for graphql configuration (authored by vsellier).
production add mandatory values for graphql configuration
Sep 27 2022, 4:31 PM
vsellier committed R260:5f413a4e1350: production: test ingress / metallb by deploying graphql (authored by vsellier).
production: test ingress / metallb by deploying graphql
Sep 27 2022, 4:23 PM
vsellier committed rSKCONF9d4c4220af86: production: fix nginx ingress chart version (authored by vsellier).
production: fix nginx ingress chart version
Sep 27 2022, 4:11 PM
vsellier committed rSKCONF6c40e8fe60e3: production: fix nginx ingress chart version (authored by vsellier).
production: fix nginx ingress chart version
Sep 27 2022, 3:59 PM
vsellier committed rSKCONF28387e6c1fd9: production-cluster: Deploy the nginx ingress controller (authored by vsellier).
production-cluster: Deploy the nginx ingress controller
Sep 27 2022, 3:51 PM
vsellier committed rSPREacbc844f9b54: Don't use the default rancher ingress manager (authored by vsellier).
Don't use the default rancher ingress manager
Sep 27 2022, 3:46 PM
vsellier committed rSPRE833b5359f64f: temporary increase prometheus retention (authored by vsellier).
temporary increase prometheus retention
Sep 27 2022, 3:22 PM
vsellier committed rSPREea34a207f927: Update storage configuration to match the reality (authored by vsellier).
Update storage configuration to match the reality
Sep 27 2022, 3:22 PM
vsellier committed R260:3a2dd951b885: Upgrade storage replayers to last available version (authored by vsellier).
Upgrade storage replayers to last available version
Sep 27 2022, 11:48 AM
vsellier committed R259:ce042e30d7eb: Upgrade storage replayers to use the last swh-model optimisations (authored by vsellier).
Upgrade storage replayers to use the last swh-model optimisations
Sep 27 2022, 11:41 AM
vsellier committed rSKCONF6ccb84be453b: Declare metallb on production cluster (authored by vsellier).
Declare metallb on production cluster
Sep 27 2022, 10:43 AM
vsellier accepted D8534: rancher-node-production-worker*: Update nameserver entries with ns0.

Thanks!

Sep 27 2022, 8:37 AM

Sep 26 2022

vsellier closed T4541: Deploy git loaders dedicated to load add forge now requests, a subtask of T4523: Dynamic infrastructure, as Resolved.
Sep 26 2022, 1:54 PM · meta-task, System administration
vsellier closed T4541: Deploy git loaders dedicated to load add forge now requests as Resolved.
Sep 26 2022, 1:54 PM · Add Forge Now , System administration
vsellier added a comment to D8532: Set rancher-node-production worker to high load profile.

Sure, they should also declare ns0 for more redundancy.

Sep 26 2022, 12:26 PM

Sep 23 2022

vsellier committed rSKCONF25a3f0a22c4c: metallb: deploy metallb on the argocd cluster (authored by vsellier).
metallb: deploy metallb on the argocd cluster
Sep 23 2022, 7:38 PM
vsellier changed the status of T4534: Evaluate MetalLB as inbound loadbalancer, a subtask of T4523: Dynamic infrastructure, from Open to Work in Progress.
Sep 23 2022, 7:38 PM · meta-task, System administration
vsellier changed the status of T4534: Evaluate MetalLB as inbound loadbalancer from Open to Work in Progress.
Sep 23 2022, 7:38 PM · System administration
vsellier added a reverting change for rSKCONF0c2373f0112d: Revert "change the argocd ingress name to something more explicit": rSKCONF8797d2440cdd: Revert "Revert "change the argocd ingress name to something more explicit"".
Sep 23 2022, 6:36 PM
vsellier committed rSKCONF8797d2440cdd: Revert "Revert "change the argocd ingress name to something more explicit"" (authored by vsellier).
Revert "Revert "change the argocd ingress name to something more explicit""
Sep 23 2022, 6:36 PM
vsellier committed rSKCONF36a731d61b10: Revert "argocd: upgrade to v2.4.12" aka wrong file committed (authored by vsellier).
Revert "argocd: upgrade to v2.4.12" aka wrong file committed
Sep 23 2022, 6:36 PM
vsellier added a reverting change for rSKCONF487aaf959c17: argocd: upgrade to v2.4.12: rSKCONF36a731d61b10: Revert "argocd: upgrade to v2.4.12" aka wrong file committed.
Sep 23 2022, 6:36 PM
vsellier committed rSKCONF55cf9d0b2ac9: argocd: upgrade to v2.4.12 (authored by vsellier).
argocd: upgrade to v2.4.12
Sep 23 2022, 6:25 PM
vsellier added a reverting change for rSKCONF6eade03466d9: change the argocd ingress name to something more explicit: rSKCONF0c2373f0112d: Revert "change the argocd ingress name to something more explicit".
Sep 23 2022, 6:25 PM
vsellier committed rSKCONF0c2373f0112d: Revert "change the argocd ingress name to something more explicit" (authored by vsellier).
Revert "change the argocd ingress name to something more explicit"
Sep 23 2022, 6:25 PM
vsellier committed rSKCONF487aaf959c17: argocd: upgrade to v2.4.12 (authored by vsellier).
argocd: upgrade to v2.4.12
Sep 23 2022, 6:21 PM
vsellier committed rSKCONF6eade03466d9: change the argocd ingress name to something more explicit (authored by vsellier).
change the argocd ingress name to something more explicit
Sep 23 2022, 6:18 PM
vsellier committed rSKCONFb29962847b49: Declare the argocd installation to selft manage the argocd updates (authored by vsellier).
Declare the argocd installation to selft manage the argocd updates
Sep 23 2022, 6:16 PM
vsellier committed R260:8335b48876cf: cassandra-replay: redispatch replayers after origin-visit-status catchup (authored by vsellier).
cassandra-replay: redispatch replayers after origin-visit-status catchup
Sep 23 2022, 3:03 PM
vsellier accepted D8524: backend: Prevent query exception when lister ids is empty.

awesome thanks.

Sep 23 2022, 10:20 AM

Sep 22 2022

vsellier closed T4544: cgit loader: the repository detail is not necessary the summary tab as Resolved.
Sep 22 2022, 11:26 PM · Lister
vsellier committed rDLS9b3e565cf7de: cgit: Ensure the clone url is searched on the right tab (authored by vsellier).
cgit: Ensure the clone url is searched on the right tab
Sep 22 2022, 6:05 PM
vsellier created P1461 T4504 - staging.
Sep 22 2022, 5:59 PM
vsellier created P1460 (An Untitled Masterwork).
Sep 22 2022, 2:49 PM

Sep 21 2022

vsellier created P1458 T4505 production.
Sep 21 2022, 8:50 AM
vsellier created P1457 T4505 staging.
Sep 21 2022, 8:43 AM
vsellier closed D8508: cgit: Ensure the clone url is searched on the right tab.

closed by 9b3e565cf7de534f3adc4e52da3b9cc9321ee431

Sep 21 2022, 8:39 AM
vsellier committed R260:8beee5bc6b07: give the time to the loaders to finish it's task before killing them (authored by vsellier).
give the time to the loaders to finish it's task before killing them
Sep 21 2022, 7:48 AM
vsellier committed R260:7866eda1ecf2: ensure the return of init container is 0 even if optional credentials are not… (authored by vsellier).
ensure the return of init container is 0 even if optional credentials are not…
Sep 21 2022, 7:48 AM
vsellier committed R260:266fb3087770: ensure the return of init container is 0 even if optional credentials are not… (authored by vsellier).
ensure the return of init container is 0 even if optional credentials are not…
Sep 21 2022, 7:43 AM
vsellier committed R260:58d35361e7a4: give the time to the loader to finish it's task before killing it (authored by vsellier).
give the time to the loader to finish it's task before killing it
Sep 21 2022, 7:43 AM
vsellier closed D8502: add-forge-now: Declare the production application to deploy the add forge now workers.
Sep 21 2022, 1:15 AM
vsellier committed rSKCONFba29d4d3c6b2: add-forge-now: Declare the production application to deploy the add forge now… (authored by vsellier).
add-forge-now: Declare the production application to deploy the add forge now…
Sep 21 2022, 1:15 AM

Sep 20 2022

vsellier updated the diff for D8508: cgit: Ensure the clone url is searched on the right tab.

rebase

Sep 20 2022, 5:03 PM
vsellier updated the diff for D8508: cgit: Ensure the clone url is searched on the right tab.

retrieve the summary tab url from the page instead of using an hardcoded url

Sep 20 2022, 4:34 PM