Page MenuHomeSoftware Heritage
Create Task
Maniphest T2214

Scale-out graph and database storage in production
Closed, MigratedEdits Locked
Actions

Description

  • post-postgresql,
  • Cassandra (see T1892).

Revisions and Commits

rDWAPPS Web applications
D3675rDWAPPSb19fb3f661fe origin: Migrate use to storage.origin_list instead of origin_get_range

Related Objects
Search...

StatusAssignedTask
 Migratedgitlab-migrationT2213 Storage
 Migratedgitlab-migrationT2214 Scale-out graph and database storage in production
 Migratedgitlab-migrationT1892 Cassandra as a storage backend
 Migratedgitlab-migrationT1816 Make all clients of swh-storage use origin URL as identifier and use visit types instead of origin types
 Migratedgitlab-migrationT1731 Intrinsic identifiers for origins
 Migratedgitlab-migrationT1912 Support origin pagination without origin ids
 Migratedgitlab-migrationT2033 Run Cassandra storage backend with production data
 Migratedgitlab-migrationT2498 Re-create the Cassandra cluster using on-premise servers
 Migratedgitlab-migrationT4644 replayer crashes on invalid directory entry name (b'gitter/gitter.xml')
 Migratedgitlab-migrationT4668 replayer: Crashes because of directories with duplicated entry names in journal
 Migratedgitlab-migrationT2185 Make webapp0 use Cassandra as storage backend.
 Migratedgitlab-migrationT2186 Merge swh-storage-cassandra in swh-storage master
 Migratedgitlab-migrationT2602 Investigate how to upgrade the schema of the Cassandra storage
 Migratedgitlab-migrationT3314 Test swh.storage.cassandra with ScyllaDB
 Migratedgitlab-migrationT3357 Perform some tests of the cassandra storage on Grid5000
 Migratedgitlab-migrationT3394 cassandra - origin url hashing encoding issue
 Migratedgitlab-migrationT3395 cassandra - Timeouts during revision import
 Migratedgitlab-migrationT3396 cassandra - allow to configure the consistency level used by the queries
 Migratedgitlab-migrationT3464 Prepare a quote for the cassandra servers
 Migratedgitlab-migrationT3465 Test multidatacenter replication
 Migratedgitlab-migrationT3491 Origin visit ids restart from 1 even if there is previous visits
 Migratedgitlab-migrationT3493 [cassandra] Git loader performance are very bad
 Migratedgitlab-migrationT3517 [cassandra] decorate the method calls to have statsd metrics
Migratedgitlab-migrationT3573 [cassandra] directory and content read benchmarks
 Migratedgitlab-migrationT3577 Parallel loaders performances
Migratedgitlab-migrationT3683 cassandra - benchmark the vault
 Migratedgitlab-migrationT3585 Fix inconsistencies of the Cassandra backend with postgres
 Migratedgitlab-migrationT3582 cassandra: Use 'git ordering' for directory entries
 Migratedgitlab-migrationT3586 Figure out what to do with 'misordered' directories in Cassandra

Event Timeline

douardda created this task.Jan 20 2020, 2:24 PM
vlorentz renamed this task from Scale0out storage in production to Scale-out storage in production.Jan 22 2020, 4:18 PM
vlorentz triaged this task as Normal priority.Jan 22 2020, 4:19 PM
vlorentz added a project: Storage manager.Jan 22 2020, 4:24 PM
vlorentz changed the task status from Open to Work in Progress.Jan 22 2020, 4:46 PM
vlorentz claimed this task.
vlorentz moved this task from Backlog to Work in progress on the Roadmap 2020 board.
vlorentz added a parent task: T2033: Run Cassandra storage backend with production data.Jul 20 2020, 5:58 PM
vlorentz removed a parent task: T2033: Run Cassandra storage backend with production data.
vlorentz added a subtask: T2033: Run Cassandra storage backend with production data.
ardumont added a revision: D3675: origin: Migrate use to storage.origin_list instead of origin_get_range.Aug 2 2020, 11:17 AM
ardumont added a commit: rDWAPPSb19fb3f661fe: origin: Migrate use to storage.origin_list instead of origin_get_range.Aug 3 2020, 9:49 AM
dachary added a subscriber: dachary.Feb 14 2021, 4:58 PM
rdicosmo renamed this task from Scale-out storage in production to Scale-out storage (graph) in production.Mar 10 2021, 4:35 PM
rdicosmo renamed this task from Scale-out storage (graph) in production to Scale-out graph and database storage in production.
rdicosmo edited projects, added Roadmap 2021; removed Roadmap 2020.Mar 10 2021, 4:39 PM
rdicosmo added a subtask: T1892: Cassandra as a storage backend.
vlorentz moved this task from Backlog to Work in progress on the Roadmap 2021 board.Apr 23 2021, 4:47 PM
vlorentz closed subtask T2602: Investigate how to upgrade the schema of the Cassandra storage as Resolved.May 26 2021, 11:25 AM
vsellier created subtask T3357: Perform some tests of the cassandra storage on Grid5000.Jun 2 2021, 6:25 PM
vlorentz removed a subtask: T3357: Perform some tests of the cassandra storage on Grid5000.Jun 25 2021, 4:32 PM
bchauvet added projects: Roadmap 2022, meta-task.Mar 23 2022, 5:04 PM
bchauvet raised the priority of this task from Normal to High.Mar 25 2022, 5:30 PM
vlorentz moved this task from Backlog to Work in progress on the Roadmap 2022 board.Jun 2 2022, 9:57 AM
gitlab-migration changed the status of subtask T2602: Investigate how to upgrade the schema of the Cassandra storage from Resolved to Migrated.Jan 8 2023, 4:31 PM
gitlab-migration closed this task as Migrated.Jan 8 2023, 5:00 PM
gitlab-migration claimed this task.
gitlab-migration closed subtask T1892: Cassandra as a storage backend as Migrated.
gitlab-migration closed subtask T2033: Run Cassandra storage backend with production data as Migrated.
gitlab-migration added subscribers: vlorentz, gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal