• rebase
• update tests according to the review feedbacks

It seems redis has a Hyperloglog functionnality[1] that can match with the requirements (bloom filter / limited deviation / small memory footprint / efficiency).

rebase

rebase

Test with "several" database upserts as it's more realistic

split the long entrypoint's command line

rebase

With the longer warning threshold, the monitoring is now green.

Backfill launched from storage1 with this script : P927 (10 ranges in //) and finished in ~15mn

All staging worker stopped:

root@pergamon:~# sudo clush -b -w @staging-workers 'puppet agent --disable "Deploy new storage version"; cd /etc/systemd/system/multi-user.target.wants; for unit in swh-worker@*; do systemctl disable $unit; done; systemctl stop swh-worker@*'

it seems it's the scheduler running that is taking time to scheduler the deposit task :
08:37:53 -> task is created
08:43:05 -> the runner is scheduling the task
08:43:24 -> the worker acknowledge the task

The interface on VLAN1330 and VLAN440 were already configured

Remove an wrong file removal

The package python3-swh.icingaplugins:v0.4.3 is released and deployed on pergamon

The shard reallocation is done :

❯ curl -s http://192.168.100.63:9200/_cat/allocation\?v\&s\=node
shards disk.indices disk.used disk.avail disk.total disk.percent host           ip             node
  2935        2.9tb       3tb      3.7tb      6.7tb           44 192.168.100.61 192.168.100.61 esnode1
  2936        2.9tb       3tb      3.7tb      6.7tb           44 192.168.100.62 192.168.100.62 esnode2
  2935        2.9tb     2.9tb      3.8tb      6.7tb           43 192.168.100.63 192.168.100.63 esnode3

Rework the sql query to use the "using" keyword to join

esnode3 configured with the same procedure as esnode2 (check the previous comments)