Page MenuHomeSoftware Heritage
Create Task
Maniphest T2182

Switch production swh-web to use swh-search instead of postgresql search.
Closed, MigratedEdits Locked
Actions

Revisions and Commits

rSPSITE puppet-swh-site
D5063rSPSITE6f18a7797f1c webapp: use swh-search as main search engine

Related Objects
Search...

StatusAssignedTask
 Migratedgitlab-migrationT1523 Search tools on metadata
 Migratedgitlab-migrationT1117 Origin search is *slow* when you look for very common words
 Migratedgitlab-migrationT1910 Redesign origin search using a dedicated component (swh-search)
 Migratedgitlab-migrationT2182 Switch production swh-web to use swh-search instead of postgresql search.
 Migratedgitlab-migrationT2183 Switch webapp0 to use swh-search instead of postgresql search.
 Migratedgitlab-migrationT2167 Deploy swh-search
 Migratedgitlab-migrationT2174 Add debian package for swh-search
 Migratedgitlab-migrationT2184 Replay origins to ElasticSearch's "origin" index
 Migratedgitlab-migrationT2497 Create an ElasticSearch cluster tuned for origin/metadata search
 Migratedgitlab-migrationT2590 Finish the indexer -> swh-search pipeline
 Migratedgitlab-migrationT2651 Make the indexer-storage publish its rows to Kafka
 Migratedgitlab-migrationT2652 Make the indexer-storage interface use attr classes instead of dicts
 Migratedgitlab-migrationT2816 Enable the journal-writer for the swh-idx-storage in staging
 Migratedgitlab-migrationT2817 Enable the swh-search environment in staging
 Migratedgitlab-migrationT2876 metadata indexation : ES' dynamic mapping creation fails for field values that are of varying types
 Migratedgitlab-migrationT2904 Create a new production webapp using the frozen index on the staging ES
 Migratedgitlab-migrationT2905 Deploy swh-search for production
 Migratedgitlab-migrationT2936 Update the swh-search journal client to only set "has_visit" on "full" status of the visit
 Migratedgitlab-migrationT2944 Deploy swh-search v0.4.1
 Migratedgitlab-migrationT3037 Reschedule origin-intrinsic-metadata tasks for all origins
 Migratedgitlab-migrationT2780 Enable the journal-writer for the swh-idx-storage in production
 Migratedgitlab-migrationT3040 [production] Enable swh-search's journal-client for indexed objects
 Migratedgitlab-migrationT3041 [production] Provision enough space for the search ES cluster to ingest all intrinsic metadata
 Migratedgitlab-migrationT3373 Metadata search is failing due to a boolean field in the mapping of the metadata fields
 Migratedgitlab-migrationT3391 [swh-search] Deploy v0.9.0 on staging and execute a full origin and metadata reindexation
 Migratedgitlab-migrationT3392 [staging] Properly recreate the origin_intrinsic_metadata topic
 Migratedgitlab-migrationT3398 [swh-search] Deploy v0.9.0 on production and execute a full origin and metadata reindexation
 Migratedgitlab-migrationT3047 Enable to search in origin metadata with swh-search in webapp
 Migratedgitlab-migrationT3058 Metadata search is failing with "failed to parse date field"
 Migratedgitlab-migrationT3060 Deploy swh-search v0.6.0 in **staging**
 Migratedgitlab-migrationT3076 [swh-search] Improve the index/mapping migration process
 Migratedgitlab-migrationT3083 Deploy swh-search v0.7.0/v0.7.1

Event Timeline

vlorentz triaged this task as Normal priority.Jan 15 2020, 3:01 PM
vlorentz created this task.
ardumont closed subtask T2183: Switch webapp0 to use swh-search instead of postgresql search. as Resolved.Feb 5 2020, 3:01 PM
vlorentz added a subtask: T2497: Create an ElasticSearch cluster tuned for origin/metadata search.Jul 20 2020, 5:56 PM
olasd added a subtask: T2905: Deploy swh-search for production.Jan 6 2021, 11:15 AM
vlorentz closed subtask T2497: Create an ElasticSearch cluster tuned for origin/metadata search as Resolved.Jan 7 2021, 2:13 PM
vsellier closed subtask T2905: Deploy swh-search for production as Resolved.Jan 13 2021, 9:23 AM
vsellier changed the task status from Open to Work in Progress.Feb 11 2021, 9:24 AM
vsellier claimed this task.
vsellier added a subscriber: vsellier.

The main webapp search can be switch from the sql search to the swh-search as all the tests performed on staging and https://webapp1.internal.softwareheritage.org are ok

vlorentz changed the status of subtask T2590: Finish the indexer -> swh-search pipeline from Open to Work in Progress.Feb 11 2021, 11:01 AM
vsellier added a revision: D5063: webapp: use swh-search as main search engine in production.Feb 11 2021, 11:27 AM
vsellier added a commit: rSPSITE6f18a7797f1c: webapp: use swh-search as main search engine.Feb 11 2021, 12:00 PM
vsellier closed this task as Resolved.Feb 11 2021, 12:10 PM

D5063 is applied, the main webapp is now using swh-search by default.

vsellier added a project: System administration.Feb 11 2021, 12:14 PM
vlorentz closed subtask T2590: Finish the indexer -> swh-search pipeline as Resolved.Sep 8 2021, 3:35 PM
gitlab-migration changed the task status from Resolved to Migrated.Oct 19 2022, 5:57 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

gitlab-migration changed the status of subtask T2497: Create an ElasticSearch cluster tuned for origin/metadata search from Resolved to Migrated.Oct 19 2022, 5:57 PM
gitlab-migration changed the status of subtask T2905: Deploy swh-search for production from Resolved to Migrated.Oct 19 2022, 6:00 PM
gitlab-migration changed the status of subtask T2183: Switch webapp0 to use swh-search instead of postgresql search. from Resolved to Migrated.Jan 8 2023, 10:00 PM
gitlab-migration changed the status of subtask T2590: Finish the indexer -> swh-search pipeline from Resolved to Migrated.
About · Developer information · Legal