A current implementation detail imposes to define an objstorage key in the storage key
entry. It's not needed for the current checker service so use a noop key entry for the
objstorage.
Related to T4324
Details
Details
- Reviewers
vlorentz - Group Reviewers
System administrators - Maniphest Tasks
- T4324: production: Deploy swh-scrubber database and checkers
- Commits
- rSPSITEdadeb097cba9: scrubber: Deactivate the unneeded objstorage configuration part
octo-diff drops the azure objstorage configuration and use a noop objstorage (unused anyway):
$ $SWH_PUPPET_ENVIRONMENT_HOME/bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to staging scrubber1.internal.softwarehe ritage.org Found host scrubber1.internal.softwareheritage.org ... ******************************************* File[/etc/softwareheritage/scrubber/postgres.yml] => parameters => content => @@ -13,74 +13,4 @@ - cls: filtered storage_conf: - cls: azure-prefixed - accounts: - '0': - account_name: 0euwestswh - api_secret_key: swh::azure::credentials::0euwestswh - container_name: contents - '1': - account_name: 1euwestswh - api_secret_key: swh::azure::credentials::1euwestswh - container_name: contents - '2': - account_name: 2euwestswh - api_secret_key: swh::azure::credentials::2euwestswh - container_name: contents - '3': - account_name: 3euwestswh - api_secret_key: swh::azure::credentials::3euwestswh - container_name: contents - '4': - account_name: 4euwestswh - api_secret_key: swh::azure::credentials::4euwestswh - container_name: contents - '5': - account_name: 5euwestswh - api_secret_key: swh::azure::credentials::5euwestswh - container_name: contents - '6': - account_name: 6euwestswh - api_secret_key: swh::azure::credentials::6euwestswh - container_name: contents - '7': - account_name: 7euwestswh - api_secret_key: swh::azure::credentials::7euwestswh - container_name: contents - '8': - account_name: 8euwestswh - api_secret_key: swh::azure::credentials::8euwestswh - container_name: contents - '9': - account_name: 9euwestswh - api_secret_key: swh::azure::credentials::9euwestswh - container_name: contents - a: - account_name: aeuwestswh - api_secret_key: swh::azure::credentials::aeuwestswh - container_name: contents - b: - account_name: beuwestswh - api_secret_key: swh::azure::credentials::beuwestswh - container_name: contents - c: - account_name: ceuwestswh - api_secret_key: swh::azure::credentials::ceuwestswh - container_name: contents - d: - account_name: deuwestswh - api_secret_key: swh::azure::credentials::deuwestswh - container_name: contents - e: - account_name: eeuwestswh - api_secret_key: swh::azure::credentials::eeuwestswh - container_name: contents - f: - account_name: feuwestswh - api_secret_key: swh::azure::credentials::feuwestswh - container_name: contents - filters_conf: - - type: readonly - - cls: filtered - storage_conf: - cls: remote - url: http://banco.internal.softwareheritage.org:5003/ - filters_conf: - - type: readonly - - cls: filtered - storage_conf: - cls: remote - url: http://saam.internal.softwareheritage.org:5003/ - filters_conf: - - type: readonly + cls: noop ******************************************* *** End octocatalog-diff on scrubber1.internal.softwareheritage.org
Manually updating the configuration in the node and starting one service, it's happy as is:
17:25:13 swh-scrubber@belvedere:5432=> select now(), count(*) from corrupt_object ; +-------------------------------+-------+ | now | count | +-------------------------------+-------+ | 2022-06-27 15:25:20.101532+00 | 139 | +-------------------------------+-------+ (1 row) Time: 5.087 ms
Diff Detail
Diff Detail
- Repository
- rSPSITE puppet-swh-site
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
After discussing with olasd, turns out that the part declaring the objstorage is not currently needed so i'll try to do better.
Comment Actions
You should set that in the scrubber config in common.yaml rather than using a per-host override
Comment Actions
You should set that in the scrubber config in common.yaml rather than using a per-host override
yes, totally, i'll attend to that later in the day.
Also, more generally, i wonder if we should not loosen the storage constructor regarding that objstorage key entry (defaulting to noop).
Before it made sense to enforce it since we were only using storage for writing, now that we start having those for reading as well, it'd make sense.
Comment Actions
You should set that in the scrubber config in common.yaml rather than using a per-host override
yes, totally, i'll attend to that later in the day.