Page MenuHomeSoftware Heritage
Feed Advanced Search

Feb 4 2022

vsellier committed rSPSITEc7f2d377d52b: Avoid saam to declare global mountpoints not matching its configuration (authored by vsellier).
Avoid saam to declare global mountpoints not matching its configuration
Feb 4 2022, 9:47 AM
vsellier closed D7081: Avoid saam to declare global mountpoints not matching its configuration.
Feb 4 2022, 9:47 AM
vsellier requested review of D7081: Avoid saam to declare global mountpoints not matching its configuration.
Feb 4 2022, 9:41 AM
vsellier closed D7079: varnish: export the metrics to prometheus.
Feb 4 2022, 9:10 AM
vsellier committed rSPSITE29566b79693e: varnish: export the metrics to prometheus (authored by vsellier).
varnish: export the metrics to prometheus
Feb 4 2022, 9:09 AM

Feb 3 2022

vsellier requested review of D7079: varnish: export the metrics to prometheus.
Feb 3 2022, 7:04 PM
vsellier added a revision to T2733: Explore / install a varnish prometheus probe: D7079: varnish: export the metrics to prometheus.
Feb 3 2022, 7:04 PM · Metrics/monitoring, System administration
vsellier changed the status of T2733: Explore / install a varnish prometheus probe from Open to Work in Progress.
Feb 3 2022, 7:01 PM · Metrics/monitoring, System administration
vsellier closed D7075: nginx: add the configuration to retrieve the metrics in prometheus.
Feb 3 2022, 4:14 PM
vsellier committed rSPSITE109591ae1f3a: nginx: add the configuration to retrieve the metrics in prometheus (authored by vsellier).
nginx: add the configuration to retrieve the metrics in prometheus
Feb 3 2022, 4:14 PM
vsellier updated the diff for D7075: nginx: add the configuration to retrieve the metrics in prometheus.

use -- for all the options of the exporter configuration

Feb 3 2022, 4:10 PM
vsellier updated the diff for D7075: nginx: add the configuration to retrieve the metrics in prometheus.

minor update on documentation

Feb 3 2022, 4:05 PM
vsellier updated the summary of D7075: nginx: add the configuration to retrieve the metrics in prometheus.
Feb 3 2022, 4:02 PM
vsellier requested review of D7075: nginx: add the configuration to retrieve the metrics in prometheus.
Feb 3 2022, 3:55 PM
vsellier closed T3899: Clean nfs mountpoint on workers as Resolved.
Feb 3 2022, 12:02 PM · System administration
vsellier added a comment to T3899: Clean nfs mountpoint on workers.
  • D7068 deployed and applied on the workers:
root@pergamon:/etc/clustershell# clush -b -w @workers -w worker17 -w worker18 "set -e; puppet agent --test"
clush:  0/31
clush: in progress(31): worker[01-18],worker[01-13].euwest.azure
---------------
worker01.euwest.azure
---------------
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for worker01.euwest.azure.internal.softwareheritage.org
Info: Applying configuration version '1643885189'
Notice: Applied catalog in 11.65 seconds
...
------------
worker18
---------------
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for worker18.softwareheritage.org
Info: Applying configuration version '1643885204'
Notice: /Stage[main]/Profile::Mountpoints/Mount[/srv/storage/space]/options: options changed 'rw,soft,intr,rsize=8192,wsize=8192,noauto,x-systemd.automount,x-systemd.device-timeout=10' to 'ro,soft,intr,rsize=8192,wsize=8192,noauto,x-systemd.automount,x-systemd.device-timeout=10'
Info: Computing checksum on file /etc/fstab
Info: /Stage[main]/Profile::Mountpoints/Mount[/srv/storage/space]: Scheduling refresh of Mount[/srv/storage/space]
Info: Mount[/srv/storage/space](provider=parsed): Remounting
Notice: /Stage[main]/Profile::Mountpoints/Mount[/srv/storage/space]: Triggered 'refresh' from 1 event
Info: /Stage[main]/Profile::Mountpoints/Mount[/srv/storage/space]: Scheduling refresh of Mount[/srv/storage/space]
Notice: Applied catalog in 19.67 seconds
clush: worker[01-18] (18): exited with exit code 2
Feb 3 2022, 12:01 PM · System administration
vsellier closed D7068: mountpoints: remove useless default mountpoints.
Feb 3 2022, 11:43 AM
vsellier committed rSPSITE37739f1543ea: mountpoints: remove useless default mountpoints (authored by vsellier).
mountpoints: remove useless default mountpoints
Feb 3 2022, 11:43 AM
vsellier updated the diff for D7068: mountpoints: remove useless default mountpoints.

completely remove the mountpoint to remove as the mount class
is not doing the cleanup when it's declared as absent.

Feb 3 2022, 11:42 AM
vsellier requested review of D7068: mountpoints: remove useless default mountpoints.
Feb 3 2022, 11:32 AM
vsellier added a revision to T3899: Clean nfs mountpoint on workers: D7068: mountpoints: remove useless default mountpoints.
Feb 3 2022, 11:32 AM · System administration

Feb 2 2022

vsellier created P1271 scn timeouts.
Feb 2 2022, 11:04 AM

Feb 1 2022

vsellier committed rSPSITEe30059ef882c: icinga: Fix hedgedoc and read-only objstorage http checks (authored by vsellier).
icinga: Fix hedgedoc and read-only objstorage http checks
Feb 1 2022, 5:55 PM
vsellier accepted D7058: pergamon: Drop no longer relevant rewrite_rule about sentry.s.o.
Feb 1 2022, 5:45 PM
vsellier committed rSPSITE76ddf4d7db67: icinga: fix the webapp and r/o objstorage check url (authored by vsellier).
icinga: fix the webapp and r/o objstorage check url
Feb 1 2022, 5:07 PM
vsellier accepted D7045: Migrate sentry node to admin vlan.
Feb 1 2022, 3:31 PM
vsellier requested changes to D7045: Migrate sentry node to admin vlan.
Feb 1 2022, 2:37 PM
vsellier accepted D7055: storage01.euwest.azure: Only keep the gunicorn-swh-storage service.
Feb 1 2022, 2:31 PM

Jan 31 2022

vsellier added a comment to T3903: Clean up unused azure vms or services.

There are also:

Jan 31 2022, 6:48 PM · System administration
vsellier triaged T3902: Test linux gardener pro/cons as Normal priority.
Jan 31 2022, 6:04 PM · System administration
vsellier accepted D7049: weekly-planning: Use curl instead of httpie.
Jan 31 2022, 12:37 PM
vsellier moved T3899: Clean nfs mountpoint on workers from Backlog to in-progress on the System administration board.
Jan 31 2022, 12:33 PM · System administration
vsellier added a project to T3899: Clean nfs mountpoint on workers: System administration.
Jan 31 2022, 12:33 PM · System administration
vsellier changed the status of T3899: Clean nfs mountpoint on workers from Open to Work in Progress.
Jan 31 2022, 12:32 PM · System administration

Jan 28 2022

vsellier added a comment to D7021: Add graph dataset reading classes (orc+edges).

a few minor remarks

Jan 28 2022, 6:34 PM
vsellier planned changes to D6926: First iteration of prometheus export of the e2e metrics.
Jan 28 2022, 5:51 PM
vsellier requested review of D6926: First iteration of prometheus export of the e2e metrics.
Jan 28 2022, 5:00 PM
vsellier accepted D7048: base_check: Add typing annotation to constructor.

Thanks for making mypy happy ;)

Jan 28 2022, 4:46 PM

Jan 27 2022

vsellier accepted D7046: vagrant: Fix dns search domains per deployment.

LGTM, it's not really ideal but it avoid to break the current subnet spliting we are using

Jan 27 2022, 4:09 PM
vsellier added inline comments to D7045: Migrate sentry node to admin vlan.
Jan 27 2022, 11:38 AM
vsellier changed the status of T3889: Admin database backup from Open to Work in Progress.
Jan 27 2022, 10:50 AM · System administration
vsellier added a comment to T3889: Admin database backup.

Unfornunately I didn't find any tool supporting natively the backup of postgresql through zfs snapshots.
We have a couple of alternatives:

  • Use borg to to implement a clean backup of the postgresql data directory based on hooks [1]
  • Use a tool like pgbackrest[2], the backup type and storage target need to be choosen (S3, storage server, ...)
  • Implement a backup based on zfs snapshots, which should not be too complicated but we need to manage all the plumbing to deal with full/incremental backups and the history cleaning
Jan 27 2022, 10:50 AM · System administration
vsellier added a comment to P1269 quick and dirty postgresql backup script based on zfs snapshot.

related to T3889

Jan 27 2022, 10:26 AM
vsellier created P1269 quick and dirty postgresql backup script based on zfs snapshot.
Jan 27 2022, 10:25 AM

Jan 25 2022

vsellier updated the task description for T3889: Admin database backup.
Jan 25 2022, 5:28 PM · System administration
vsellier created T3889: Admin database backup.
Jan 25 2022, 5:27 PM · System administration
vsellier closed D6979: Document the mirror credentials management.
Jan 25 2022, 5:05 PM
vsellier committed rDDOC4ac8648fb0a7: Document the mirror credentials management (authored by vsellier).
Document the mirror credentials management
Jan 25 2022, 5:05 PM
vsellier updated the diff for D6979: Document the mirror credentials management.
  • rebase
  • update according the feedback
Jan 25 2022, 4:57 PM
vsellier added inline comments to D6979: Document the mirror credentials management.
Jan 25 2022, 4:36 PM
vsellier accepted D7034: common/origin_save: Fix extraction of elasticsearch document fields.

LGTM

Jan 25 2022, 4:34 PM
vsellier accepted D7035: Automate weekly-planning script.

thanks, just a small inlined remark regarding the httpie package installation

Jan 25 2022, 4:28 PM

Jan 24 2022

vsellier closed D7018: netbox: migrate the vm to the admin vlan.
Jan 24 2022, 3:05 PM
vsellier committed rSPSITEd6d9ca9a373c: netbox: migrate the vm to the admin vlan (authored by vsellier).
netbox: migrate the vm to the admin vlan
Jan 24 2022, 3:05 PM
vsellier updated the diff for D7018: netbox: migrate the vm to the admin vlan.

rebase

Jan 24 2022, 3:04 PM
vsellier accepted D7023: Add missing maven lister task subscription to lister config.
Jan 24 2022, 12:00 PM

Jan 21 2022

vsellier closed D7020: Configure the kafka clusters environment for the consumer lag exporter.
Jan 21 2022, 7:07 PM
vsellier committed rSPSITEe2fd42a3f28c: Configure the kafka clusters environment for the consumer lag exporter (authored by vsellier).
Configure the kafka clusters environment for the consumer lag exporter
Jan 21 2022, 7:07 PM
vsellier requested review of D7020: Configure the kafka clusters environment for the consumer lag exporter.
Jan 21 2022, 7:00 PM
vsellier updated the diff for D7018: netbox: migrate the vm to the admin vlan.

rebase

Jan 21 2022, 5:45 PM
vsellier closed D6986: kafka: add a script to create the kafka credentials.
Jan 21 2022, 5:43 PM
vsellier committed rSPSITE87f392051fb0: vagrant: declare the getty ip (authored by vsellier).
vagrant: declare the getty ip
Jan 21 2022, 5:43 PM
vsellier committed rSPSITE847db4bbe80f: kafka: add a script to create the kafka credentials (authored by vsellier).
kafka: add a script to create the kafka credentials
Jan 21 2022, 5:43 PM
vsellier updated the task description for T3873: Migrate inventory to the admin vlan.
Jan 21 2022, 5:35 PM · System administration
vsellier requested review of D7018: netbox: migrate the vm to the admin vlan.
Jan 21 2022, 5:32 PM
vsellier added a revision to T3873: Migrate inventory to the admin vlan: D7018: netbox: migrate the vm to the admin vlan.
Jan 21 2022, 5:32 PM · System administration
vsellier committed rSENV7c0fa9643d40: Prepare the bojiman migration to the admin vlan (authored by vsellier).
Prepare the bojiman migration to the admin vlan
Jan 21 2022, 5:18 PM
vsellier updated the task description for T3873: Migrate inventory to the admin vlan.
Jan 21 2022, 4:51 PM · System administration
vsellier committed rSENVa00824d0f4f5: vagrant: update the facts at each run (authored by vsellier).
vagrant: update the facts at each run
Jan 21 2022, 4:18 PM
vsellier updated the task description for T3873: Migrate inventory to the admin vlan.
Jan 21 2022, 3:46 PM · System administration
vsellier updated the task description for T3873: Migrate inventory to the admin vlan.
Jan 21 2022, 3:26 PM · System administration
vsellier updated the task description for T3873: Migrate inventory to the admin vlan.
Jan 21 2022, 3:14 PM · System administration
vsellier changed the status of T3873: Migrate inventory to the admin vlan from Open to Work in Progress.
Jan 21 2022, 3:13 PM · System administration
vsellier updated the diff for D6986: kafka: add a script to create the kafka credentials.

fix the typo on the getty hostname

Jan 21 2022, 12:26 PM
vsellier updated the diff for D6979: Document the mirror credentials management.
  • Rebase
  • Update the kafka credentials part to use the new user management script
  • Add a section to describe the username pattern
  • Add a section to describe how to create the read-only storage credentials
Jan 21 2022, 11:52 AM
vsellier closed D6993: fix the admin layout.
Jan 21 2022, 10:16 AM
vsellier committed rDDOC50f23abeb1e2: fix the admin layout (authored by vsellier).
fix the admin layout
Jan 21 2022, 10:16 AM
vsellier updated the diff for D6993: fix the admin layout.

rebase

Jan 21 2022, 10:16 AM
vsellier requested review of D6993: fix the admin layout.
Jan 21 2022, 10:13 AM
vsellier updated the diff for D6986: kafka: add a script to create the kafka credentials.

add the puppet header on the script

Jan 21 2022, 9:41 AM

Jan 20 2022

vsellier updated the test plan for D6986: kafka: add a script to create the kafka credentials.
Jan 20 2022, 7:32 PM
vsellier updated the diff for D6986: kafka: add a script to create the kafka credentials.
  • Install the scripts for all the environments in getty, the journal orchestrator;
  • as the cluster configurations are now global, it impacts the consumer group exporter. It make sense to move it from storage1 to getty to also centralize this part (FW rules will need to be adapted accordingly);
Jan 20 2022, 7:32 PM
vsellier committed rSENV52b1bc7a1ac7: vagrant: Declare the getty node (authored by vsellier).
vagrant: Declare the getty node
Jan 20 2022, 7:04 PM
vsellier committed rSENVc0a1faa68152: vagrant: Declare the default vm type to debian (authored by vsellier).
vagrant: Declare the default vm type to debian
Jan 20 2022, 6:07 PM
vsellier committed rSPSITE58fe00948b49: staging: remove duplicate properties (authored by vsellier).
staging: remove duplicate properties
Jan 20 2022, 11:36 AM
vsellier committed rSPSITEa5ae23ec1c37: staging: Use the right elasticsearch cluster to display the scn task logs (authored by vsellier).
staging: Use the right elasticsearch cluster to display the scn task logs
Jan 20 2022, 11:36 AM
vsellier committed rSENV2530f6f3b1a5: remove the fact of the decommissioned webapp0 production server (authored by vsellier).
remove the fact of the decommissioned webapp0 production server
Jan 20 2022, 11:36 AM
vsellier added a comment to D6986: kafka: add a script to create the kafka credentials.

Yes it could, It will be cleaner, I will update in this way.
I took a shortcut estimating we should know which broker is up or not, to avoid to have to build the connection string :)

Jan 20 2022, 11:13 AM
vsellier updated the diff for D6986: kafka: add a script to create the kafka credentials.

rebase

Jan 20 2022, 10:59 AM
vsellier requested review of D6986: kafka: add a script to create the kafka credentials.
Jan 20 2022, 10:54 AM

Jan 19 2022

vsellier closed T3857: unexpected long time between a task is received and the loading start for few tasks as Resolved.

The ack late propery is deployed in staging and production.
The number of scn in the 'scheduled' status should decrease.

Jan 19 2022, 4:21 PM · System administration, Scheduling utilities
vsellier requested review of D6979: Document the mirror credentials management.
Jan 19 2022, 3:33 PM
vsellier added a revision to T3829: Document mirror - how to create and deploy a mirror from scratch: D6979: Document the mirror credentials management.
Jan 19 2022, 3:23 PM · Mirror
vsellier closed D6974: Reactivate the ack late configuration for the high priority tasks.
Jan 19 2022, 2:31 PM
vsellier committed rSPSITE768ce43e60f9: Reactivate the ack late configuration for the high priority tasks (authored by vsellier).
Reactivate the ack late configuration for the high priority tasks
Jan 19 2022, 2:31 PM
vsellier updated the diff for D6974: Reactivate the ack late configuration for the high priority tasks.

rebase

Jan 19 2022, 2:31 PM
vsellier closed D6966: Move mirror documentation from devel to sysadm.
Jan 19 2022, 2:23 PM
vsellier committed rDDOC09ef5e2c6aa7: Move mirror documentation from devel to sysadm (authored by vsellier).
Move mirror documentation from devel to sysadm
Jan 19 2022, 2:23 PM
vsellier updated the diff for D6966: Move mirror documentation from devel to sysadm.

fix according the review

Jan 19 2022, 2:10 PM
vsellier updated the diff for D6966: Move mirror documentation from devel to sysadm.
  • rebase
Jan 19 2022, 12:29 PM