Page MenuHomeSoftware Heritage
Create Task
Maniphest T2033

Run Cassandra storage backend with production data
Closed, MigratedEdits Locked
Actions

Description

To make sure nothing unexpected happens

Related Objects
Search...

StatusAssignedTask
 Migratedgitlab-migrationT2213 Storage
 Migratedgitlab-migrationT2214 Scale-out graph and database storage in production
 Migratedgitlab-migrationT1892 Cassandra as a storage backend
 Migratedgitlab-migrationT1816 Make all clients of swh-storage use origin URL as identifier and use visit types instead of origin types
 Migratedgitlab-migrationT1731 Intrinsic identifiers for origins
 Migratedgitlab-migrationT1912 Support origin pagination without origin ids
 Migratedgitlab-migrationT2033 Run Cassandra storage backend with production data
 Migratedgitlab-migrationT2498 Re-create the Cassandra cluster using on-premise servers
 Migratedgitlab-migrationT4644 replayer crashes on invalid directory entry name (b'gitter/gitter.xml')
 Migratedgitlab-migrationT4668 replayer: Crashes because of directories with duplicated entry names in journal
 Migratedgitlab-migrationT2185 Make webapp0 use Cassandra as storage backend.
 Migratedgitlab-migrationT2186 Merge swh-storage-cassandra in swh-storage master
 Migratedgitlab-migrationT2602 Investigate how to upgrade the schema of the Cassandra storage
 Migratedgitlab-migrationT3314 Test swh.storage.cassandra with ScyllaDB
 Migratedgitlab-migrationT3357 Perform some tests of the cassandra storage on Grid5000
 Migratedgitlab-migrationT3394 cassandra - origin url hashing encoding issue
 Migratedgitlab-migrationT3395 cassandra - Timeouts during revision import
 Migratedgitlab-migrationT3396 cassandra - allow to configure the consistency level used by the queries
 Migratedgitlab-migrationT3464 Prepare a quote for the cassandra servers
 Migratedgitlab-migrationT3465 Test multidatacenter replication
 Migratedgitlab-migrationT3491 Origin visit ids restart from 1 even if there is previous visits
 Migratedgitlab-migrationT3493 [cassandra] Git loader performance are very bad
 Migratedgitlab-migrationT3517 [cassandra] decorate the method calls to have statsd metrics
Migratedgitlab-migrationT3573 [cassandra] directory and content read benchmarks
 Migratedgitlab-migrationT3577 Parallel loaders performances
Migratedgitlab-migrationT3683 cassandra - benchmark the vault
 Migratedgitlab-migrationT3585 Fix inconsistencies of the Cassandra backend with postgres
 Migratedgitlab-migrationT3582 cassandra: Use 'git ordering' for directory entries
 Migratedgitlab-migrationT3586 Figure out what to do with 'misordered' directories in Cassandra

Event Timeline

vlorentz triaged this task as Low priority.Oct 10 2019, 12:01 PM
vlorentz created this task.
vlorentz closed subtask T2034: Unbreak journal clients as Resolved.Nov 4 2019, 12:22 PM
vlorentz reopened subtask T2034: Unbreak journal clients as Open.Nov 14 2019, 3:41 PM
olasd mentioned this in rSENV4ce88b7b4a51: Add puppet-cassandra module.Dec 20 2019, 3:27 PM
olasd mentioned this in rSPSITE2eee9282b2a3: Add a cassandra node role.
olasd mentioned this in rSPREd24116dfe58b: Add definitions for cassandra servers.Dec 20 2019, 6:43 PM
olasd changed the task status from Open to Work in Progress.Dec 20 2019, 6:56 PM
olasd added a subscriber: olasd.

A cassandra cluster has been deployed on cassandra[01-06].euwest.azure.internal.softwareheritage.org.

The deployment, done via terraform and puppet, follows the best practices from https://docs.microsoft.com/en-us/azure/architecture/best-practices/cassandra

vlorentz removed a subtask: T2034: Unbreak journal clients.Jan 14 2020, 3:10 PM
vlorentz added a subtask: T2214: Scale-out graph and database storage in production.Jul 20 2020, 5:58 PM
vlorentz removed a subtask: T2214: Scale-out graph and database storage in production.
vlorentz added a parent task: T2214: Scale-out graph and database storage in production.
vlorentz renamed this task from Run Cassandra storage draft with production data to Run Cassandra storage backend with production data.Aug 20 2020, 8:36 AM
vlorentz added a subtask: T4644: replayer crashes on invalid directory entry name (b'gitter/gitter.xml').Oct 18 2022, 3:40 PM
vlorentz closed subtask T2498: Re-create the Cassandra cluster using on-premise servers as Resolved.
gitlab-migration changed the status of subtask T2498: Re-create the Cassandra cluster using on-premise servers from Resolved to Migrated.Oct 19 2022, 5:57 PM
vlorentz added a subtask: T4668: replayer: Crashes because of directories with duplicated entry names in journal.Nov 2 2022, 11:32 AM
gitlab-migration closed this task as Migrated.Jan 8 2023, 5:01 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

gitlab-migration closed subtask T4644: replayer crashes on invalid directory entry name (b'gitter/gitter.xml') as Migrated.Jan 8 2023, 10:25 PM
gitlab-migration closed subtask T4668: replayer: Crashes because of directories with duplicated entry names in journal as Migrated.
About · Developer information · Legal