- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed Advanced Search
Advanced Search
Advanced Search
Aug 25 2022
Aug 25 2022
vsellier committed R260:3b295de813f3: Bootstrap a swh chart only with the cassandra replayer stack (authored by vsellier).
Bootstrap a swh chart only with the cassandra replayer stack
Move the checksum computation inside the if
Aug 24 2022
Aug 24 2022
Everything is almost ready to start the ingestion tests:
- The deployment of the replayers is implemented
- The monitoring can also be easily deployed in a kubernetes cluster having the prometheus operator installed [1]
- the grafana dashboards are also available as templates [1]
vsellier committed rSKCONF1fef5d96e81b: Add grafana dashboards dedicated to cassandra and replayers statistics (authored by vsellier).
Add grafana dashboards dedicated to cassandra and replayers statistics
fix statsd_exporter image version
vsellier committed R260:8d11190a0259: Deploy a prometheus statsd exporter and declare the monitoring (authored by vsellier).
Deploy a prometheus statsd exporter and declare the monitoring
vsellier committed R260:767f83f70b95: Add more cassandra servers as seed to dispatch the connections accross all the… (authored by vsellier).
Add more cassandra servers as seed to dispatch the connections accross all the…
Aug 23 2022
Aug 23 2022
vsellier committed rSKCONF99b55cad41a6: Declare the monitoring of external cassandra nodes (authored by vsellier).
Declare the monitoring of external cassandra nodes
Aug 22 2022
Aug 22 2022
remove useless name overriding
fix affinity criteria quoting
affine the resources requests
vsellier committed R260:b95e7a1f37f3: Allow to specify the replayers's pods affinity (authored by vsellier).
Allow to specify the replayers's pods affinity
Aug 20 2022
Aug 20 2022
reduce the deployment history
vsellier committed R260:a400f447256d: Add snapshots and increase the directory parallelism (authored by vsellier).
Add snapshots and increase the directory parallelism
vsellier committed R260:b0983dbf143c: Adjust memory limits according the real values (authored by vsellier).
Adjust memory limits according the real values
fix memory spec
ingest more object type
vsellier committed R260:96f4eeefd246: Fix syntax issues with topics containing a _ (authored by vsellier).
Fix syntax issues with topics containing a _
vsellier committed R260:ebed5df4cf6f: Configure autoscaling and add other origin_* topics (authored by vsellier).
Configure autoscaling and add other origin_* topics
Aug 19 2022
Aug 19 2022
fix cpu reservation unit
Declare a cpu and memory request
Use internal kafka addresses
vsellier committed rSKCONFe19517bb86ea: Depploy keda on cassandra-production cluster (authored by vsellier).
Depploy keda on cassandra-production cluster
vsellier committed rSKCONF4434c969017a: declare the cassandra production replaying stack (authored by vsellier).
declare the cassandra production replaying stack
Remove debug logs
vsellier committed rSPSITE42405baf257b: fix kafka credential creation script (authored by vsellier).
fix kafka credential creation script
vsellier committed R260:a432372b70d2: bump swh-storage-replayer image versions (authored by vsellier).
bump swh-storage-replayer image versions
vsellier committed R259:a3571bce9a00: use apt clean instead of removing all the dpkg content (authored by vsellier).
use apt clean instead of removing all the dpkg content
vsellier committed R260:fb1c89846639: fix the way to quote the server in the init command (authored by vsellier).
fix the way to quote the server in the init command
fix init database command
Add an init cassandra option
remove port from cassandra host
Improve indentation v4
Improve indentation v3
Improve indentation
improve indentation
fix the configmap name
vsellier committed R260:eb40b1cbbc6c: add more default values for cassandra replayer (authored by vsellier).
add more default values for cassandra replayer
vsellier committed rSKCONF6a8eab9de494: Configure the cassandra-production cluster (authored by vsellier).
Configure the cassandra-production cluster
vsellier committed rSKCONF438c33c127f1: Declare the swh namespace for the production-cassandra cluster (authored by vsellier).
Declare the swh namespace for the production-cassandra cluster
specify the journal client user
prepare production deployment
vsellier committed rSPSITE5d845aa0805a: cassandra: extend directory permission to allow monitoring to check the disks (authored by vsellier).
cassandra: extend directory permission to allow monitoring to check the disks
Aug 18 2022
Aug 18 2022
all server reconfigured and cassandra started on them:
/opt/cassandra/bin/nodetool status Datacenter: sesi_rocquencourt ============================= Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns (effective) Host ID Rack UN 192.168.100.184 88.65 KiB 16 34.3% e0c24d24-6f68-4a26-8561-94e67b58211a rack1 UN 192.168.100.181 84.71 KiB 16 31.3% 1d9b9e7d-b376-4afe-8f67-482e8412f21b rack1 UN 192.168.100.186 69.07 KiB 16 34.2% 0dd3426d-9159-47bd-9b4e-065ff0fbb889 rack1 UN 192.168.100.183 69.08 KiB 16 37.1% 78281a92-7fa0-43bd-bc33-c5b419ee8715 rack1 UN 192.168.100.185 69.07 KiB 16 32.2% abf9b69e-3cec-4ac3-a195-a54481e4d9da rack1 UN 192.168.100.182 74.05 KiB 16 30.9% eca5ea5d-8bd5-4301-9a5e-ffa01aa1b7e5 rack1
vsellier closed T4446: Prepare the disks and configure zfs, a subtask of T4373: [cassandra] Test the new hardware, as Resolved.
Recreating the zpool correctly:
# mixedused ls /dev/disk/by-id/nvme-MO003200KXAVU* | grep -v part | xargs -t zpool create -o ashift=12 -O mountpoint=none mixeduse zfs create -o mountpoint=/srv/cassandra/instance1/data mixeduse/cassandra-instance1-data
Testing the performances of the different configuration (on a zfs pool with only one disk):
- disk block: 512k / zpool ashift:9
zpool create -o ashift=9 -O mountpoint=none mixeduse /dev/disk/by-id/nvme-MO003200KXAVU_SJA4N7938I0405A0U
zfs create -o mountpoint=/srv/cassandra/instance1/data -o atime=off -o relatime=on mixeduse/cassandra-data
cd /srv/cassandra/instance1/data
bonnie++ -d . -m cassandra04 -u nobody
Using uid:65534, gid:65534.
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 2.00 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Name:Size etc /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
cassandra04 515496M 293k 99 1.0g 99 703m 99 661k 99 1.4g 91 13717 463
Latency 48216us 7316us 8224us 23303us 7928us 1606us
Version 2.00 ------Sequential Create------ --------Random Create--------
cassandra04 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 16384 98 +++++ +++ 16384 8 +++++ +++ +++++ +++ 16384 99
Latency 2679us 1207us 4851ms 2850us 138us 301us
1.98,2.00,cassandra04,1,1659338044,515496M,,8192,5,293,99,1080974,99,720299,99,661,99,1488832,91,13717,463,16,,,,,28232,98,+++++,+++,2018,8,+++++,+++,+++++,+++,24821,99,48216us,7316us,8224us,23303us,7928us,1606us,2679us,1207us,4851ms,2850us,138us,301usThe nvme format command didn't succeed on the write intensive disk. It never exits and the disk become unresponsive after that.
vsellier changed the status of T4446: Prepare the disks and configure zfs from Open to Work in Progress.
cassandra: Add missing dependencies
vsellier committed rSPSITEa6e31c14466c: cassandra: Remove the no longer used cassandra.yaml template (authored by vsellier).
cassandra: Remove the no longer used cassandra.yaml template
vsellier committed rSPRE8791a2ec762d: cassandra: Cleanup extra mounts of the cassandra directories (authored by vsellier).
cassandra: Cleanup extra mounts of the cassandra directories
vsellier committed rSPSITE57d978d9e3de: cassandra: Refactor the former installation scripts (authored by vsellier).
cassandra: Refactor the former installation scripts
Aug 17 2022
Aug 17 2022
vsellier committed R260:8f12e7df4312: Bootstrap a swh chart only with the cassandra replayer to starting (authored by vsellier).
Bootstrap a swh chart only with the cassandra replayer to starting
add swh-storage-replayer application
use inline_yaml instead of to_yaml
thanks for the hint, I completely forgot about it
Aug 16 2022
Aug 16 2022
fix the private data repository url
fix indentation
fix indentation
vsellier committed rSKCONF6f65eb6f12f0: Configure the argocd applications configuration (authored by vsellier).
Configure the argocd applications configuration
Activate applications autosync
vsellier committed rSKCONF59bd7205efc8: declare argocd own applications and prepare the cassandra cluster ones (authored by vsellier).
declare argocd own applications and prepare the cassandra cluster ones
vsellier committed R261:15987cb50831: Declare the private repository that will contain the k8s clusters secrets (authored by vsellier).
Declare the private repository that will contain the k8s clusters secrets
- Rebase
- Override the jmx port value via the JVM_EXTRA_OPTS environment
- Inline the cassandra.yaml properties in hiera
Increase cassandra nodes memory
Aug 12 2022
Aug 12 2022
Setting -Dcassandra.jmx.local.port / -Dcassandra.jmx.remote.port / -Dcom.sun.management.jmxremote.rmi.port in the $JVM_EXTRA_OPTS env var should override what this file configured
I specified why on the first line of the file, but I realize it not very visible.
vsellier renamed T4435: scrubber log verbosity puts a risk on ELK from scrubber log verbosity put a risk on ELK to scrubber log verbosity puts a risk on ELK .
vsellier changed the status of T4391: [cassandra] deploy the replaying stack, a subtask of T4373: [cassandra] Test the new hardware, from Open to Work in Progress.
vsellier changed the status of T4391: [cassandra] deploy the replaying stack from Open to Work in Progress.
The puppet code is ready for review. It was updated to support multi instances deployment in anticipation of T4375.
Aug 11 2022
Aug 11 2022
vsellier committed rSPSITEfbbc0eb13e49: vagrant: fix the internal_network declaration (authored by vsellier).
vagrant: fix the internal_network declaration
vsellier committed rSENVaef11d699700: vagrant: Fix the initial environment synchronization (authored by vsellier).
vagrant: Fix the initial environment synchronization
Aug 10 2022
Aug 10 2022
vsellier committed rSPRE5fc8dc9b2626: cassandra: expose the log directory to the pods (authored by vsellier).
cassandra: expose the log directory to the pods
For the record, the issues related to the commitlog_directory configuration:
After spending some time to successfully start a cassandra cluster of 2 nodes with a declarative configuration), these are the observations:
- A service can't be used to expose the cassandra ports to the clustrer, the pod address must be used. It's because cassandra use the dns name provided as listen address
- It should work by setting the listen address to 0.0.0.0 but it's stongly recommanded to not use this in the documentation
Setting listen_address to 0.0.0.0 is always wrong.
- Using internal pod address will avoid multi DC deployment for the future
vsellier committed rSKCONF8e37b0f5df0c: cassandra: poc working manual declarative configuration (authored by vsellier).
cassandra: poc working manual declarative configuration
Aug 9 2022
Aug 9 2022
vsellier committed rSPRE73d451f1b506: cassandra: Remove ballooning for the control-plane nodes (authored by vsellier).
cassandra: Remove ballooning for the control-plane nodes
vsellier committed R260:6908e14f5cf3: workers: try to stop argocd to override the keda status (authored by vsellier).
workers: try to stop argocd to override the keda status
Aug 8 2022
Aug 8 2022
VPN certificate revoked in opensense
vsellier changed the status of T4414: Outboard sysadm intern Amadou Thiam from Open to Work in Progress.
vsellier accepted D8212: docker: add a generic 'swh' command in storage and worker entrypoint scripts.
looks good, thanks.
I'm not sure if it's not better to keep the KAFKA_BROKER_ID closed to the instance declaration as it needs to be unique in a cluster and can't be shared between the instances.
But otherwise, thank, it's simpler like that.