Plan, out of [1] diff:
- [X] scheduler0.staging, workers.staging: Stop puppet [2]
- [X] scheduler0.staging: stop the old journal client [3]
- [X] workers.staging: Wait for all tasks to finish
- [X] Stop swh-worker@indexer_origin_intrinsic_metadata [4]
- [X] D7928: Rework puppet manifest to drop old services + update indexer service (as journal client)
- [X] ~~scheduler-nodes: Clean up old service~~ (previous diff ^ does it)
- [X] Remove celery workers and queues (click-click on rabbitmq ui from scheduler0.staging [5])
- [X] pergamon: Deploy diff [6]
- [X] scheduler0.staging: Apply puppet change (drop old journal client service)
- [X] workers.staging: Upgrade python3-swh.indexer to v1.1.0
- [X] P1370: Issue with that version [7] ^
- [X] w/ vlorentz: Package new python3-swh.indexer to v1.2.0
- [X] workers.staging: Upgrade python3-swh.indexer to v1.2.0
- [X] Unstuck next problem... (the configuration is now off) [9]
- [X] workers.staging: Apply puppet change (drop old service, deploy new journal client service) [10]
- [X] T4282#86233: Backing down: It's not ready so reverting the current deployment
- [ ] D7951: Actual deployment when it's ready
- [ ] Reindex everything from scratch (proposed implementations [11])?
[1] D7899
[2]
```
root@pergamon:~# clush -b -w scheduler0.internal.staging.swh.network -w @staging-workers 'puppet agent --disable "T4282: Migrate to origin intrinsic meta indexer as journal client"'
```
[3]
```
root@pergamon:~# clush -b -w scheduler0.internal.staging.swh.network systemctl stop swh-indexer-journal-client.service
```
[4]
```
root@pergamon:~# clush -b -w @staging-workers systemctl stop swh-worker@indexer_origin_intrinsic_metadata.service
```
[5] http://scheduler0.internal.staging.swh.network:15672/#/queues
[6]
```
root@pergamon:~# /usr/local/bin/deploy.sh
HEAD is now at eff3f30 Add snyk-stg-01 credentials
Already up to date.
HEAD is now at eff3f30 Add snyk-stg-01 credentials
Already up to date.
```
[7]
```
root@scheduler0:~# puppet agent --enable; puppet agent --test
... # it passed nonetheless beyond me and applied the stuff ¯\_(ツ)_/¯
root@scheduler0:~# systemctl list-units | grep swh-indexer-journal-client.service
root@scheduler0:~# # no longer present ^ vs saatchi
```
[8] prod (untouched for now)
```
root@saatchi:~# systemctl list-units | grep swh-indexer-journal-client.service
swh-indexer-journal-client.service loaded active running Software Heritage Indexer Journal Client
```
[9]
```
swhworker@worker0:~$ /usr/bin/swh indexer --config-file $SWH_CONFIG_FILENAME journal-client '*'
Traceback (most recent call last):
File "/usr/bin/swh", line 33, in <module>
sys.exit(load_entry_point('swh.core==2.8.0', 'console_scripts', 'swh')())
File "/usr/lib/python3/dist-packages/swh/core/cli/__init__.py", line 184, in main
return swh(auto_envvar_prefix="SWH")
File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/lib/python3/dist-packages/swh/indexer/cli.py", line 310, in journal_client
idx = OriginMetadataIndexer()
File "/usr/lib/python3/dist-packages/swh/indexer/metadata.py", line 325, in __init__
self.revision_metadata_indexer = RevisionMetadataIndexer(config=config)
File "/usr/lib/python3/dist-packages/swh/indexer/metadata.py", line 163, in __init__
super().__init__(*args, **kwargs)
File "/usr/lib/python3/dist-packages/swh/indexer/indexer.py", line 167, in __init__
self.check()
File "/usr/lib/python3/dist-packages/swh/indexer/indexer.py", line 202, in check
raise ValueError("Tools %s is unknown, cannot continue" % self.tools)
ValueError: Tools [] is unknown, cannot continue
swhworker@worker0:~$ /usr/bin/swh indexer --config-file $SWH_CONFIG_FILENAME journal-client '*'
```
[10]
```
root@pergamon:~# clush -b -w @staging-workers "systemctl status swh-indexer-journal-client" | grep -c running
4
```
[11] It can reuse either the group_id to avoid re-indexing, or use a new one to reindex
everything (to solve old previous temporary failures we ever had). Or we can reset the
topic to reindex everything.