Page MenuHomeSoftware Heritage
Feed Advanced Search

Jan 24 2023

vsellier committed R261:d9ce30ab3530: Update the new physical server template (authored by vsellier).
Update the new physical server template
Jan 24 2023, 4:27 PM

Jan 20 2023

vsellier committed R261:83596f9b35ff: new-server template: fix the task list (authored by vsellier).
new-server template: fix the task list
Jan 20 2023, 4:57 PM
vsellier committed R261:f76799941094: replace /todo by [ ] as they are not created as tasks (authored by vsellier).
replace /todo by [ ] as they are not created as tasks
Jan 20 2023, 4:57 PM
vsellier committed R261:d3d9c2724b24: Add a template for new servers installation (authored by vsellier).
Add a template for new servers installation
Jan 20 2023, 4:57 PM

Jan 7 2023

vsellier committed rSPSITE807020080264: cassandra: Increase the load profile (authored by vsellier).
cassandra: Increase the load profile
Jan 7 2023, 12:15 AM
vsellier committed rSPSITE270090a3ff95: archive-production-rke2: Declare dns entries and thanos configurations (authored by vsellier).
archive-production-rke2: Declare dns entries and thanos configurations
Jan 7 2023, 12:15 AM
vsellier committed rSPSITEf0c65d45565c: cassandra: Generate the jvm11-server.options based on configuration (authored by vsellier).
cassandra: Generate the jvm11-server.options based on configuration
Jan 7 2023, 12:15 AM
vsellier committed rSPSITE03948a558a6d: cassandra: adapt the settings to bne able to repair the directory_entry table (authored by vsellier).
cassandra: adapt the settings to bne able to repair the directory_entry table
Jan 7 2023, 12:15 AM
vsellier committed rSPSITEcb96aba3710a: rancher: configure dns and mountpoints for the new rke2 cluster nodes (authored by vsellier).
rancher: configure dns and mountpoints for the new rke2 cluster nodes
Jan 7 2023, 12:15 AM
vsellier committed rSPSITEab4ceaf22e1b: rancher: Configure /var/lib/rancher as a zfs dataset (authored by vsellier).
rancher: Configure /var/lib/rancher as a zfs dataset
Jan 7 2023, 12:15 AM
vsellier committed rSPSITEcd29683fc0a1: thanos: configure thanos query and thanos storage for archive-production (authored by vsellier).
thanos: configure thanos query and thanos storage for archive-production
Jan 7 2023, 12:15 AM
vsellier committed rSPSITE4c15c5c28ced: thanos: configure thanos query and thanos storage for gitlab-production (authored by vsellier).
thanos: configure thanos query and thanos storage for gitlab-production
Jan 7 2023, 12:15 AM
vsellier committed rSPSITEd7c8eb5ae096: thanos: configure thanos query and thanos storage for rancher cluster (authored by vsellier).
thanos: configure thanos query and thanos storage for rancher cluster
Jan 7 2023, 12:15 AM
vsellier committed rSPSITE8ad1e01d69ff: thanos: configure thanos query and thanos storage for gitlab-staging (authored by vsellier).
thanos: configure thanos query and thanos storage for gitlab-staging
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE75815edf3809: thanos: Activate thanos query for archive-staging (authored by vsellier).
thanos: Activate thanos query for archive-staging
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE28ab4a9e7da3: staging: Use the virtual IP of the kubernetes cluster as an entry point (authored by vsellier).
staging: Use the virtual IP of the kubernetes cluster as an entry point
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE48edf604c15c: cassandra: increase heap memory to 16G instead of the default 8G (authored by vsellier).
cassandra: increase heap memory to 16G instead of the default 8G
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE2d5c347e514a: monitoring: fix the minio http check expected string (authored by vsellier).
monitoring: fix the minio http check expected string
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE6dbdab42b6fd: archive-staging: configure the nodes to mount /tmp as a tmpfs (authored by vsellier).
archive-staging: configure the nodes to mount /tmp as a tmpfs
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE85c3321ea2ce: monitoring: fix the minio http check url (authored by vsellier).
monitoring: fix the minio http check url
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE13dbc1aaef92: dali: Reduce daily retention time (authored by vsellier).
dali: Reduce daily retention time
Jan 7 2023, 12:14 AM
vsellier committed rSPSITEc091f350231a: admin/minio: export the s3 api to internet through the admin reverse proxy (authored by vsellier).
admin/minio: export the s3 api to internet through the admin reverse proxy
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE03ca5092bcaf: admin/minio: remove not used dns entry (authored by vsellier).
admin/minio: remove not used dns entry
Jan 7 2023, 12:14 AM
vsellier committed rSPSITEfb239a218f72: minio: declare the dns entries (authored by vsellier).
minio: declare the dns entries
Jan 7 2023, 12:14 AM
vsellier committed rSPSITEb5651687c653: admin: Connect the admin cluster's thanos to the main stack (authored by vsellier).
admin: Connect the admin cluster's thanos to the main stack
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE18cac842ef67: argocd: migrate to the admin k8s cluster (authored by vsellier).
argocd: migrate to the admin k8s cluster
Jan 7 2023, 12:14 AM
vsellier committed rSPSITEfaf90e06e0f8: argocd: Update monitoring probes (authored by vsellier).
argocd: Update monitoring probes
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE58774ac43804: Support several origins for the savecodenow end to end checks (authored by vsellier).
Support several origins for the savecodenow end to end checks
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE7279b8731576: Ensure the rancher node regexp also apply for admin hosts (authored by vsellier).
Ensure the rancher node regexp also apply for admin hosts
Jan 7 2023, 12:14 AM
vsellier committed rSPSITE85d393668cf4: offboarding of grouss (authored by vsellier).
offboarding of grouss
Jan 7 2023, 12:14 AM
vsellier committed rSPSITEd353c33ff4dc: gitlab: fix the monitoring probes configuration (authored by vsellier).
gitlab: fix the monitoring probes configuration
Jan 7 2023, 12:14 AM

Dec 16 2022

vsellier committed R261:7b8ab17be15d: Refresh the real template and remove the wrong one (authored by vsellier).
Refresh the real template and remove the wrong one
Dec 16 2022, 6:49 PM
vsellier committed R261:cf18ad4b2a7f: fix template formating (authored by vsellier).
fix template formating
Dec 16 2022, 6:48 PM
vsellier committed R261:cc86b4020e2e: make the offboarding issues confidentials (authored by vsellier).
make the offboarding issues confidentials
Dec 16 2022, 6:48 PM
vsellier committed R261:1700d3b1709d: Test the gitlab issues templating (authored by vsellier).
Test the gitlab issues templating
Dec 16 2022, 6:48 PM
vsellier committed R261:3b47ea5796e2: Add notification attempt on the templates (authored by vsellier).
Add notification attempt on the templates
Dec 16 2022, 6:48 PM

Dec 9 2022

vsellier accepted D8929: sysadm: Add a "how to process add-forge-now requests".
  • Is it possible to also list this query I often use to check the status of the ingestion:

for example:
select last_visit_status, count(ovs.url) from origin_visit_stats ovs join listed_origins lo on lo.url = ovs.url and lo.visit_type = ovs.visit_type where lister_id='8d1ebad5-a2dc-482c-9b9b-188665274377' group by last_visit_status;

Dec 9 2022, 5:37 PM

Nov 22 2022

vsellier accepted D8871: sysadm/upgrade-swh-services: Update the build deps step.
Nov 22 2022, 4:55 PM
vsellier added a comment to D8871: sysadm/upgrade-swh-services: Update the build deps step.

Just an inline remark but otherwise it's ok.

Nov 22 2022, 4:54 PM

Nov 18 2022

vsellier accepted D8854: sysadm/data-silos/kafka: Bootstrap topics management page.
Nov 18 2022, 11:26 AM
vsellier requested changes to D8854: sysadm/data-silos/kafka: Bootstrap topics management page.
Nov 18 2022, 11:08 AM

Nov 7 2022

vsellier closed D8811: sysadmin: document the argocd user management.
Nov 7 2022, 4:53 PM
vsellier committed rDDOC5afff2c0924f: sysadmin: document the argocd user management (authored by vsellier).
sysadmin: document the argocd user management
Nov 7 2022, 4:53 PM
vsellier updated the diff for D8811: sysadmin: document the argocd user management.

Update according the review suggestions

Nov 7 2022, 2:28 PM

Nov 4 2022

vsellier requested review of D8811: sysadmin: document the argocd user management.
Nov 4 2022, 6:57 PM
vsellier closed D8809: Document the gitlab installation and procedures.
Nov 4 2022, 12:00 PM
vsellier committed rDDOCfecc22a95a54: Document the gitlab installation and procedures (authored by vsellier).
Document the gitlab installation and procedures
Nov 4 2022, 12:00 PM

Nov 3 2022

vsellier requested review of D8809: Document the gitlab installation and procedures.
Nov 3 2022, 6:12 PM
vsellier added a reverting change for rDSNIPa1d262f81b60: sysadmin: generate gitlab infra diagram in svg: rDSNIP3045f3c9cca4: Revert "sysadmin: generate gitlab infra diagram in svg".
Nov 3 2022, 4:01 PM
vsellier committed rDSNIP3045f3c9cca4: Revert "sysadmin: generate gitlab infra diagram in svg" (authored by vsellier).
Revert "sysadmin: generate gitlab infra diagram in svg"
Nov 3 2022, 4:01 PM
vsellier committed rDSNIPa1d262f81b60: sysadmin: generate gitlab infra diagram in svg (authored by vsellier).
sysadmin: generate gitlab infra diagram in svg
Nov 3 2022, 3:53 PM
vsellier committed rDSNIPb1502379cc79: Add gitlab infrastructure diagram (authored by vsellier).
Add gitlab infrastructure diagram
Nov 3 2022, 3:42 PM

Nov 2 2022

vsellier abandoned D8645: azure: create the production environment for gitlab.

migrated to gitlab MR https://gitlab.softwareheritage.org/infra/swh-sysadmin-provisioning/-/merge_requests/88

Nov 2 2022, 5:16 PM
vsellier abandoned D8617: thanos: Declare archive-production thanos for live data querying.

migrated to https://gitlab.softwareheritage.org/infra/puppet/puppet-swh-site/-/merge_requests/533

Nov 2 2022, 3:16 PM

Oct 24 2022

vsellier closed T4642: swh.journal.indexed.origin_extrinsic_metadata topic has a single partition as Invalid.

Task moved on gitlab: https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4653

Oct 24 2022, 10:49 AM · System administration, Journal

Oct 21 2022

vsellier added a comment to T4587: Add forge now - process https://gitweb.gentoo.org/.

task will be handled in gitlab: https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4648

Oct 21 2022, 5:53 PM · Unknown Object (Project)

Oct 20 2022

vsellier committed R261:40fc6712eec3: fix the template directory name (authored by vsellier).
fix the template directory name
Oct 20 2022, 3:35 PM
vsellier committed R261:042fab806d7a: Test the gitlab issues templating (authored by vsellier).
Test the gitlab issues templating
Oct 20 2022, 3:34 PM
vsellier placed T4646: CVSProtocolError: Error from CVS server: b"E cvs checkout: Skipping `$Log$' keyword due to excessive comment leader... up for grabs.
Oct 20 2022, 3:18 PM · CVS loader

Oct 19 2022

vsellier committed rSPSITE92d9c1ec0f91: gitlab: Add icinga checks to verify the gitlab production availability (authored by vsellier).
gitlab: Add icinga checks to verify the gitlab production availability
Oct 19 2022, 11:17 AM

Oct 17 2022

vsellier accepted D8687: Add migration node.

LGTM

Oct 17 2022, 3:53 PM

Oct 14 2022

vsellier committed rSPRE692729ef7b3b: azure-aks: Specify the kyybernetes version to use (authored by vsellier).
azure-aks: Specify the kyybernetes version to use
Oct 14 2022, 8:30 PM
vsellier committed rSPREfe8c7cafdb11: azure: create the production environment for gitlab (authored by vsellier).
azure: create the production environment for gitlab
Oct 14 2022, 8:30 PM
vsellier committed rSKCONFe05d5d88ddf7: gitlabs: fix prometheus metric collection (authored by vsellier).
gitlabs: fix prometheus metric collection
Oct 14 2022, 8:24 PM
vsellier committed rSKCONF3427d852f1bb: gitlab-production: declare necessary namespaces (authored by vsellier).
gitlab-production: declare necessary namespaces
Oct 14 2022, 2:14 PM
vsellier committed rSKCONFfe81214e725d: gitlab-production: Deploy the monitoring stack (authored by vsellier).
gitlab-production: Deploy the monitoring stack
Oct 14 2022, 2:14 PM
vsellier committed rSKCONF4019500a616a: gitlab-production: Add an ingress controller to expose service not related to… (authored by vsellier).
gitlab-production: Add an ingress controller to expose service not related to…
Oct 14 2022, 12:15 PM
vsellier committed rSKCONF3ba2a79fb632: gitlab-staging: Add the cluster configuration application (authored by vsellier).
gitlab-staging: Add the cluster configuration application
Oct 14 2022, 12:11 PM
vsellier committed rSKCONFea3dd4db2c71: gitlab-staging: configure prometheus thanos sidecar (authored by vsellier).
gitlab-staging: configure prometheus thanos sidecar
Oct 14 2022, 12:02 PM
vsellier committed rSKCONFfbb1633d201b: gitlab-staging: Deploy an ingress controller to manager the services other… (authored by vsellier).
gitlab-staging: Deploy an ingress controller to manager the services other…
Oct 14 2022, 12:02 PM
vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

thanks, I will try to restart the loading with the patch to see how it behaves

Oct 14 2022, 8:57 AM · System administration, Archive coverage

Oct 13 2022

vsellier committed rSKCONFaf24ef54105d: gitlab-staging: Declare prometheus external labels matching the environment (authored by vsellier).
gitlab-staging: Declare prometheus external labels matching the environment
Oct 13 2022, 2:15 PM
vsellier committed rSKCONFde5f4188cf51: gitlab-staging: apply default labels to prometheus metrics (authored by vsellier).
gitlab-staging: apply default labels to prometheus metrics
Oct 13 2022, 12:12 PM
vsellier committed rSKCONF63eea02234d1: gitlab-staging: Configure prometheus to discover custom .*Monitor objects (authored by vsellier).
gitlab-staging: Configure prometheus to discover custom .*Monitor objects
Oct 13 2022, 11:44 AM
vsellier committed rSKCONFc0ee728309ae: gitlab-staging: upgrade prometheus operatore to v41.3.2 (authored by vsellier).
gitlab-staging: upgrade prometheus operatore to v41.3.2
Oct 13 2022, 11:40 AM
vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

The loading finally failed:

INFO:swh.loader.cvs.loader.CvsLoader:Load origin 'ssh://anoncvs@anoncvs.NetBSD.org:/cvsroot/src' with type 'cvs'
DEBUG:swh.loader.cvs.loader.CvsLoader:lister_not provided, skipping extrinsic origin metadata
DEBUG:swh.loader.cvs.loader.CvsLoader:prepare; origin_url=ssh://anoncvs@anoncvs.NetBSD.org:/cvsroot/src scheme=ssh path=/cvsroot/src
Warning: Permanently added 'anoncvs.netbsd.org,199.233.217.198' (RSA) to the list of known hosts.
DEBUG:swh.loader.cvs.loader.CvsLoader:Fetching CVS rlog from anoncvs.netbsd.org:/cvsroot/src
Oct 13 2022, 9:21 AM · System administration, Archive coverage

Oct 12 2022

vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

I've launched the loader on the /src subdirectory, if I follow the cvs loader documentation: https://forge.softwareheritage.org/source/swh-loader-cvs/browse/master/docs/ ,
it should also be launched on the other subdirectories:

Oct 12 2022, 6:44 PM · System administration, Archive coverage
vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

The loader got killed after it starts to consume a lot of memory...

Warning: Permanently added 'anoncvs.netbsd.org,199.233.217.198' (RSA) to the list of known hosts.
DEBUG:swh.loader.cvs.loader.CvsLoader:Fetching CVS rlog from anoncvs.netbsd.org:/cvsroot/src
Killed
swh@loader-cvs-manual:~$
Oct 12 2022, 6:20 PM · System administration, Archive coverage
vsellier moved T4626: staging: ingest openbsd.org cvs forge from Backlog to in-progress on the System administration board.
Oct 12 2022, 11:45 AM · System administration, Archive coverage
vsellier moved T4625: staging: ingest netbsd.org cvs forge from Backlog to in-progress on the System administration board.
Oct 12 2022, 11:45 AM · System administration, Archive coverage
vsellier added a project to T4625: staging: ingest netbsd.org cvs forge: System administration.
Oct 12 2022, 11:45 AM · System administration, Archive coverage
vsellier added a comment to T4626: staging: ingest openbsd.org cvs forge.

Manual loading in progress:

  • pod created with P1494
  • manual loading:
Oct 12 2022, 11:45 AM · System administration, Archive coverage
vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

The first try with a oneshot task failed, the containe was killed:

swhscheduler@scheduler0:~$ swh scheduler --url http://scheduler0.internal.staging.swh.network:5008/ task add load-cvs -p oneshot ssh://anoncvs@anoncvs.NetBSD.org:/cvsroot/src
Created 1 tasks
Oct 12 2022, 11:43 AM · System administration, Archive coverage
vsellier created P1494 CVS loader container.
Oct 12 2022, 11:38 AM
vsellier changed the status of T4626: staging: ingest openbsd.org cvs forge from Open to Work in Progress.
Oct 12 2022, 11:36 AM · System administration, Archive coverage
vsellier changed the status of T4625: staging: ingest netbsd.org cvs forge from Open to Work in Progress.
Oct 12 2022, 11:35 AM · System administration, Archive coverage
vsellier updated the task description for T4621: staging: Autoscaling of the indexers not properly configured.
Oct 12 2022, 10:15 AM · System administration
vsellier renamed T4621: staging: Autoscaling of the indexers not properly configured from staging: Autoscaling of the indexer not properly well configurred to staging: Autoscaling of the indexers not properly configured.
Oct 12 2022, 10:14 AM · System administration
vsellier triaged T4621: staging: Autoscaling of the indexers not properly configured as Normal priority.
Oct 12 2022, 10:11 AM · System administration

Oct 11 2022

vsellier accepted D8653: production: Migrate large workers consumption in production cluster.
Oct 11 2022, 4:11 PM
vsellier accepted D8659: Add new production rancher worker with dedicated swh/loader role.
Oct 11 2022, 4:09 PM
vsellier closed D8654: gitlab-production: declare the internal ingress ip.
Oct 11 2022, 4:04 PM
vsellier committed rSPSITEb39dafeec18b: gitlab-production: declare the internal ingress ip (authored by vsellier).
gitlab-production: declare the internal ingress ip
Oct 11 2022, 4:04 PM
vsellier added a comment to T4063: Deploy gitlab instance for production.

DNS entry gitlab.softwareheritage.org declared in the dns

Oct 11 2022, 10:17 AM · System administration, GitLab migration
vsellier requested review of D8654: gitlab-production: declare the internal ingress ip.
Oct 11 2022, 9:31 AM
vsellier added a revision to T4063: Deploy gitlab instance for production: D8654: gitlab-production: declare the internal ingress ip.
Oct 11 2022, 9:31 AM · System administration, GitLab migration

Oct 10 2022

vsellier committed rSKCONFcdc17a029b7b: gitlab-production: Declare application (authored by vsellier).
gitlab-production: Declare application
Oct 10 2022, 8:15 PM
vsellier added a comment to T4063: Deploy gitlab instance for production.
  • gitlab production instance created:
name: euwest-gitlab-production
internal_ip: 192.168.200.4
public_ip: 20.50.42.230
  • private link created: the address of the services exposed in the VPN network will be 192.168.200.5 (prometheus/...)
Oct 10 2022, 3:03 PM · System administration, GitLab migration
vsellier requested review of D8645: azure: create the production environment for gitlab.
Oct 10 2022, 2:01 PM
vsellier added a revision to T4063: Deploy gitlab instance for production: D8645: azure: create the production environment for gitlab.
Oct 10 2022, 2:01 PM · System administration, GitLab migration