Page MenuHomeSoftware Heritage
Create Task
Maniphest T2213

Storage
Open, NormalPublic
Actions

Related Objects
Search...

StatusAssignedTask
 OpenNoneT2213 Storage
 Work in ProgressvlorentzT2214 Scale-out graph and database storage in production
 OpenvlorentzT1892 Cassandra as a storage backend
 ResolvedvlorentzT1816 Make all clients of swh-storage use origin URL as identifier and use visit types instead of origin types
 ResolvedvlorentzT1731 Intrinsic identifiers for origins
 ResolvedvlorentzT1910 Redesign origin search using a dedicated component (swh-search)
 ResolvedolasdT2052 Publish swh-search on PyPI
 ResolvedvsellierT2182 Switch production swh-web to use swh-search instead of postgresql search.
 ResolvedvsellierT2497 Create an ElasticSearch cluster tuned for origin/metadata search
 Work in ProgressvlorentzT2590 Finish the indexer -> swh-search pipeline
 ResolvedvlorentzT2651 Make the indexer-storage publish its rows to Kafka
 ResolvedvlorentzT2652 Make the indexer-storage interface use attr classes instead of dicts
 ResolvedvsellierT2816 Enable the journal-writer for the swh-idx-storage in staging
 ResolvedvsellierT2817 Enable the swh-search environment in staging
 ResolvedvlorentzT2876 metadata indexation : ES' dynamic mapping creation fails for field values that are of varying types
 ResolvedvsellierT2904 Create a new production webapp using the frozen index on the staging ES
 ResolvedvsellierT2905 Deploy swh-search for production
 ResolvedvlorentzT2936 Update the swh-search journal client to only set "has_visit" on "full" status of the visit
 ResolvedvsellierT2944 Deploy swh-search v0.4.1
 ResolvedardumontT3037 Reschedule origin-intrinsic-metadata tasks for all origins
 ResolvedardumontT2780 Enable the journal-writer for the swh-idx-storage in production
 OpenNoneT3040 [production] Enable swh-search's journal-client for indexed objects
 OpenvsellierT3041 [production] Provision enough space for the search ES cluster to ingest all intrinsic metadata
 ResolvedanlambertT3047 Enable to search in origin metadata with swh-search in webapp
 ResolvedvlorentzT3058 Metadata search is failing with "failed to parse date field"
 ResolvedvsellierT3060 Deploy swh-search v0.6.0 in **staging**
 ResolvedvsellierT3076 [swh-search] Improve the index/mapping migration process
 ResolvedvsellierT3083 Deploy swh-search v0.7.0/v0.7.1
 ResolvedvlorentzT1912 Support origin pagination without origin ids
 Work in ProgressvlorentzT2033 Run Cassandra storage backend with production data
 OpenNoneT2498 Re-create the Cassandra cluster using on-premise servers
 ResolvedvlorentzT2185 Make webapp0 use Cassandra as storage backend.
 ResolvedvlorentzT2186 Merge swh-storage-cassandra in swh-storage master
 ResolvedvlorentzT2183 Switch webapp0 to use swh-search instead of postgresql search.
 ResolvedardumontT2167 Deploy swh-search
 DuplicateolasdT2174 Add debian package for swh-search
 ResolvedvlorentzT2184 Replay origins to ElasticSearch's "origin" index
 OpenvlorentzT2602 Investigate how to upgrade the schema of the Cassandra storage
 OpenvlorentzT3314 Test swh.storage.cassandra with ScyllaDB
 OpenNoneT2215 Streaming support everywhere
 OpenNoneT2216 Packing object storage

Event Timeline

douardda created this task.Jan 20 2020, 2:23 PM
vlorentz triaged this task as Normal priority.Jan 22 2020, 4:19 PM
vlorentz renamed this task from Storage to [meta-task] Storage.Jan 22 2020, 4:21 PM
vlorentz renamed this task from [meta-task] Storage to Storage.
vlorentz added a project: meta-task.
vlorentz changed the status of subtask T2214: Scale-out graph and database storage in production from Open to Work in Progress.Jan 22 2020, 4:46 PM
rdicosmo mentioned this in T3116: Roll out first operational mirror.Mar 11 2021, 7:59 PM
About · Developer information · Legal