Page Menu
Home
Software Heritage
Search
Configure Global Search
Log In
Create Task
Maniphest
T2213
Storage
Open, Normal
Public
Actions
Edit Task
Edit Related Tasks...
Create Subtask
Edit Parent Tasks
Edit Subtasks
Merge Duplicates In
Close As Duplicate
Edit Related Objects...
Edit Commits
Edit Mocks
Edit Revisions
Subscribe
Mute Notifications
Award Token
Flag For Later
Assigned To
None
Authored By
douardda
Jan 20 2020, 2:23 PM
2020-01-20 14:23:32 (UTC+1)
Tags
Roadmap 2020
(Backlog)
meta-task
(Backlog)
Subscribers
douardda
Related Objects
Search...
Task Graph
Mentions
Status
Assigned
Task
Open
None
T2213
Storage
Work in Progress
vlorentz
T2214
Scale-out graph and database storage in production
Open
vlorentz
T1892
Cassandra as a storage backend
Resolved
vlorentz
T1816
Make all clients of swh-storage use origin URL as identifier and use visit types instead of origin types
Resolved
vlorentz
T1731
Intrinsic identifiers for origins
Resolved
vlorentz
T1910
Redesign origin search using a dedicated component (swh-search)
Resolved
olasd
T2052
Publish swh-search on PyPI
Resolved
vsellier
T2182
Switch production swh-web to use swh-search instead of postgresql search.
Resolved
vsellier
T2497
Create an ElasticSearch cluster tuned for origin/metadata search
Resolved
vlorentz
T2590
Finish the indexer -> swh-search pipeline
Resolved
vlorentz
T2651
Make the indexer-storage publish its rows to Kafka
Resolved
vlorentz
T2652
Make the indexer-storage interface use attr classes instead of dicts
Resolved
vsellier
T2816
Enable the journal-writer for the swh-idx-storage in staging
Resolved
vsellier
T2817
Enable the swh-search environment in staging
Resolved
vlorentz
T2876
metadata indexation : ES' dynamic mapping creation fails for field values that are of varying types
Resolved
vsellier
T2904
Create a new production webapp using the frozen index on the staging ES
Resolved
vsellier
T2905
Deploy swh-search for production
Resolved
vlorentz
T2936
Update the swh-search journal client to only set "has_visit" on "full" status of the visit
Resolved
vsellier
T2944
Deploy swh-search v0.4.1
Resolved
ardumont
T3037
Reschedule origin-intrinsic-metadata tasks for all origins
Resolved
ardumont
T2780
Enable the journal-writer for the swh-idx-storage in production
Resolved
vsellier
T3040
[production] Enable swh-search's journal-client for indexed objects
Resolved
vsellier
T3041
[production] Provision enough space for the search ES cluster to ingest all intrinsic metadata
Resolved
vsellier
T3373
Metadata search is failing due to a boolean field in the mapping of the metadata fields
Resolved
vsellier
T3391
[swh-search] Deploy v0.9.0 on staging and execute a full origin and metadata reindexation
Resolved
vsellier
T3392
[staging] Properly recreate the origin_intrinsic_metadata topic
Resolved
vsellier
T3398
[swh-search] Deploy v0.9.0 on production and execute a full origin and metadata reindexation
Resolved
anlambert
T3047
Enable to search in origin metadata with swh-search in webapp
Resolved
vlorentz
T3058
Metadata search is failing with "failed to parse date field"
Resolved
vsellier
T3060
Deploy swh-search v0.6.0 in **staging**
Resolved
vsellier
T3076
[swh-search] Improve the index/mapping migration process
Resolved
vsellier
T3083
Deploy swh-search v0.7.0/v0.7.1
Resolved
vlorentz
T1912
Support origin pagination without origin ids
Work in Progress
vlorentz
T2033
Run Cassandra storage backend with production data
Open
None
T2498
Re-create the Cassandra cluster using on-premise servers
Resolved
vlorentz
T2185
Make webapp0 use Cassandra as storage backend.
Resolved
vlorentz
T2186
Merge swh-storage-cassandra in swh-storage master
Resolved
vlorentz
T2183
Switch webapp0 to use swh-search instead of postgresql search.
Resolved
ardumont
T2167
Deploy swh-search
Duplicate
olasd
T2174
Add debian package for swh-search
Resolved
vlorentz
T2184
Replay origins to ElasticSearch's "origin" index
Resolved
vlorentz
T2602
Investigate how to upgrade the schema of the Cassandra storage
Resolved
vlorentz
T3314
Test swh.storage.cassandra with ScyllaDB
Resolved
vsellier
T3357
Perform some tests of the cassandra storage on Grid5000
Resolved
vlorentz
T3394
cassandra - origin url hashing encoding issue
Resolved
vsellier
T3395
cassandra - Timeouts during revision import
Resolved
vsellier
T3396
cassandra - allow to configure the consistency level used by the queries
Resolved
vsellier
T3464
Prepare a quote for the cassandra servers
Resolved
vsellier
T3465
Test multidatacenter replication
Duplicate
vsellier
T3491
Origin visit ids restart from 1 even if there is previous visits
Resolved
vsellier
T3493
[cassandra] Git loader performance are very bad
Resolved
vsellier
T3517
[cassandra] decorate the method calls to have statsd metrics
Resolved
vsellier
T3573
[cassandra] directory and content read benchmarks
Resolved
vsellier
T3577
Parallel loaders performances
Resolved
vsellier
T3683
cassandra - benchmark the vault
Wontfix
vlorentz
T3585
Fix inconsistencies of the Cassandra backend with postgres
Wontfix
vlorentz
T3582
cassandra: Use 'git ordering' for directory entries
Resolved
vlorentz
T3586
Figure out what to do with 'misordered' directories in Cassandra
Open
None
T2215
Streaming support everywhere
Open
None
T2216
Packing object storage
Mentioned In
T3116: Roll out at least one operational mirror
Event Timeline
douardda
created this task.
Jan 20 2020, 2:23 PM
2020-01-20 14:23:32 (UTC+1)
vlorentz
triaged this task as
Normal
priority.
Jan 22 2020, 4:19 PM
2020-01-22 16:19:18 (UTC+1)
vlorentz
renamed this task from
Storage
to
[meta-task] Storage
.
Jan 22 2020, 4:21 PM
2020-01-22 16:21:17 (UTC+1)
vlorentz
renamed this task from
[meta-task] Storage
to
Storage
.
vlorentz
added a project:
meta-task
.
vlorentz
changed the status of subtask
T2214: Scale-out graph and database storage in production
from
Open
to
Work in Progress
.
Jan 22 2020, 4:46 PM
2020-01-22 16:46:37 (UTC+1)
rdicosmo
mentioned this in
T3116: Roll out at least one operational mirror
.
Mar 11 2021, 7:59 PM
2021-03-11 19:59:38 (UTC+1)