Change Details

This is the version that brings extrinsic-metadata indexing * [x] tag swh-indexer v2.2.0 * [x] T4406#88618: unstuck failing debian (buster) build [1] - [ ] D8161: puppet: prepare new manfest for each indexer as journal client service [2] staging: * [ ] run SQL migrations (to create `origin_extrinsic_metadata` table and related functions) * [ ] update and restart idx-storage server * [ ] update and restart indexer journal client * [ ] make sure the indexer journal client subscribed to the `raw_extrinsic_metadata` topic and is consuming from the start production: * [ ] run SQL migrations (to create `origin_extrinsic_metadata` table and related functions) * [ ] update and restart idx-storage server * [ ] update and restart indexer journal client * [ ] make sure the indexer journal client subscribed to the `raw_extrinsic_metadata` topic and is consuming from the start [1] https://jenkins.softwareheritage.org/job/debian/job/packages/job/DCIDX/job/gbp-buildpackage/177/console [2] Existing deployed origin intrinsic metadata journal had a very large memory consumption when it ran. So instead of letting that service run all index computations, separate it into multiple ones dedicated to each index to compute so each can be parametrized differently if need be.

This is the version that brings extrinsic-metadata indexing * [x] tag swh-indexer v2.2.0 * [x] T4406#88618: unstuck failing debian (buster) build [1] - [ ] D8161: puppet: prepare new manfest for each indexer as journal client service [2] staging: * [x] run SQL migrations (to create `origin_extrinsic_metadata` table and related functions) [3] * [ ] update and restart idx-storage server * [ ] update and restart indexer journal client * [ ] make sure the indexer journal client subscribed to the `raw_extrinsic_metadata` topic and is consuming from the start production: * [ ] run SQL migrations (to create `origin_extrinsic_metadata` table and related functions) * [ ] update and restart idx-storage server * [ ] update and restart indexer journal client * [ ] make sure the indexer journal client subscribed to the `raw_extrinsic_metadata` topic and is consuming from the start [1] https://jenkins.softwareheritage.org/job/debian/job/packages/job/DCIDX/job/gbp-buildpackage/177/console [2] Existing deployed origin intrinsic metadata journal had a very large memory consumption when it ran. So instead of letting that service run all index computations, separate it into multiple ones dedicated to each index to compute so each can be parametrized differently if need be. [3] ``` swhstorage@storage1:~$ swh db --config-file /etc/softwareheritage/storage/indexer.yml version indexer --module-config-key=indexer_storage module: indexer current code version: 135 version: 134 swhstorage@storage1:~$ swh db --config-file /etc/softwareheritage/storage/indexer.yml upgrade indexer --to-version=135 --module-config-key=indexer_storage INFO:swh.core.db.db_utils:Executing migration script '/usr/lib/python3/dist-packages/swh/indexer/sql/upgrades/135.sql' Migration to version 135 done swhstorage@storage1:~$ swh db --config-file /etc/softwareheritage/storage/indexer.yml version indexer --module-config-key=indexer_storage module: indexer current code version: 135 version: 135 ```

This is the version that brings extrinsic-metadata indexing * [x] tag swh-indexer v2.2.0 * [x] T4406#88618: unstuck failing debian (buster) build [1] - [ ] D8161: puppet: prepare new manfest for each indexer as journal client service [2] staging: * [ * [x] run SQL migrations (to create `origin_extrinsic_metadata` table and related functions) [3] * [ ] update and restart idx-storage server * [ ] update and restart indexer journal client * [ ] make sure the indexer journal client subscribed to the `raw_extrinsic_metadata` topic and is consuming from the start production: * [ ] run SQL migrations (to create `origin_extrinsic_metadata` table and related functions) * [ ] update and restart idx-storage server * [ ] update and restart indexer journal client * [ ] make sure the indexer journal client subscribed to the `raw_extrinsic_metadata` topic and is consuming from the start [1] https://jenkins.softwareheritage.org/job/debian/job/packages/job/DCIDX/job/gbp-buildpackage/177/console [2] Existing deployed origin intrinsic metadata journal had a very large memory consumption when it ran. So instead of letting that service run all index computations, separate it into multiple ones dedicated to each index to compute so each can be parametrized differently if need be. [3] ``` swhstorage@storage1:~$ swh db --config-file /etc/softwareheritage/storage/indexer.yml version indexer --module-config-key=indexer_storage module: indexer current code version: 135 version: 134 swhstorage@storage1:~$ swh db --config-file /etc/softwareheritage/storage/indexer.yml upgrade indexer --to-version=135 --module-config-key=indexer_storage INFO:swh.core.db.db_utils:Executing migration script '/usr/lib/python3/dist-packages/swh/indexer/sql/upgrades/135.sql' Migration to version 135 done swhstorage@storage1:~$ swh db --config-file /etc/softwareheritage/storage/indexer.yml version indexer --module-config-key=indexer_storage module: indexer current code version: 135 version: 135 ```