In order to prepare the final production deployment, we would like to test the cass operator from datastax [1]
The main features will simplify the operation of the cluster:
- Proper token ring initialization, with only one node bootstrapping at a time
- Seed node management - one per rack, or three per datacenter, whichever is more
- Server configuration integrated into the CassandraDatacenter CRD
- Rolling reboot nodes by changing the CRD
- Store data in a rack-safe way - one replica per cloud AZ
- Scale up racks evenly with new nodes
- Scale down racks evenly by decommissioning existing nodes
- Replace dead/unrecoverable nodes
- Multi DC clusters (limited to one Kubernetes namespace)
We also want to test different cluster topologies. A subtask per topology will be created.
Per topology, the following scenarios will be tested:
- Configure and bootstrap the cluster, for some topologies, rack configuration can be needed
- Import data / measure performance
- Check recurring jobs (NodeSync / ...)
- Add a new node / check data rebalancing
- Remove a node / check data rebalancing
- Kill a cassandra instance check recovery / rebalancing
2 steps:
- Recover the killed instance
- Replace the killed instance
[1] https://github.com/k8ssandra/cass-operator