Differential D4709

indexer_storage: Publish indexer computation to journal topics
ClosedPublic
Actions

Authored by ardumont on Dec 9 2020, 10:09 PM.

Details

Reviewers

vsellier

Group Reviewers

Reviewers

Maniphest Tasks

T2590: Finish the indexer -> swh-search pipeline
T2817: Enable the swh-search environment in staging

Commits

rDENVf5d483c14997: indexer_storage: Publish indexer computation to journal topics

Summary

This allows going straight from loading an origin up to indexing in
elasticsearch's index origin loaded.

Related to T2817
Related to T2590

Test Plan

$ doco -f docker-compose.yml -f docker-compose.search.yml -f docker-compose.override.yml up -d`
$ doco -f docker-compose.yml -f docker-compose.search.yml -f docker-compose.override.yml exec swh-loader swh loader run https://github.com/codemeta/codemeta
$ http://localhost:9200/_cat/indices?v

health status index  uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   origin 44iUHZqWST-QoE5eOA39oQ   1   1         13            5     65.1kb         65.1kb
# There is elasticsearch "document" in the "origin" index