equivalent of T2780 but for the staging
Description
Description
Revisions and Commits
Revisions and Commits
| rSPSITE puppet-swh-site | |||
| D4625 | rSPSITE2e1a65a3e33b staging: Fix object storage configuration for indexers | ||
| D4620 | rSPSITEa2a84c2efb3e staging: configure idx-storage to write to kafka | ||
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Migrated | gitlab-migration | T1523 Search tools on metadata | ||
| Migrated | gitlab-migration | T1117 Origin search is *slow* when you look for very common words | ||
| Migrated | gitlab-migration | T1910 Redesign origin search using a dedicated component (swh-search) | ||
| Migrated | gitlab-migration | T2052 Publish swh-search on PyPI | ||
| Migrated | gitlab-migration | T2167 Deploy swh-search | ||
| Migrated | gitlab-migration | T2174 Add debian package for swh-search | ||
| Migrated | gitlab-migration | T2182 Switch production swh-web to use swh-search instead of postgresql search. | ||
| Migrated | gitlab-migration | T2590 Finish the indexer -> swh-search pipeline | ||
| Migrated | gitlab-migration | T2816 Enable the journal-writer for the swh-idx-storage in staging |
Event Timeline
Comment Actions
The swh-indexer stack is deployed on staging and the initial loading is done.
The volumes are quite low :
% for topic in $(./kafka-topics.sh --bootstrap-server $SERVER --list | grep indexed)
do
echo -n "$topic : "
./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list $SERVER --topic $topic | awk -F: '{s+=$3}END{print s}'
done
swh.journal.indexed.content_metadata : 4
swh.journal.indexed.origin_intrinsic_metadata : 20
swh.journal.indexed.revision_intrinsic_metadata : 20IMO, This task can be closed, but the retention / partitioning still have to be defined