Page MenuHomeSoftware Heritage

vault: Setup new vangogh server
ClosedPublic

Authored by ardumont on Wed, May 22, 9:58 AM.

Details

Summary

Prepare the vault vm to expose the vault api.

The vault api is plugged to:

  • db in prado
  • "azure" objstorage
  • "remote" storage already setuped in azure (storage0)

The objstorage is plugged to an azure blobstorage (as defined in D1495)

Note:

  • As the db used is no longer installed on the same node, the postgres profile has been removed from the vault role
  • The objstorage cache backend was exposed but unused, this was removed. As such, the vault role has been updated to remove the objstorage profile as well.

Related T1716

Test Plan
$ cd puppet-environment
$ bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to update_configuration vangogh.euwest.azure.internal.softwareheritage.org

Output: P404

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Wed, May 22, 9:58 AM
ardumont edited the test plan for this revision. (Show Details)Wed, May 22, 10:05 AM
ardumont retitled this revision from vault: Prepare new vangogh vault vm to vault: Setup new vangogh server.Wed, May 22, 10:12 AM
ardumont added projects: Vault, Puppet recipes.
olasd requested changes to this revision.Wed, May 22, 11:06 AM

Thanks for this change!

I think there's a lot more opportunity for cleanup here which I've outlined :)

data/defaults.yaml
1375

I don't think this comment is accurate (I'm also not sure it's all that useful)

data/hostname/vangogh.euwest.azure.internal.softwareheritage.org.yaml
3

Is this really used? I think all "cache objstorage" requests are proxied through the vault rpc server.

I don't think we need the objstorage rpc server deployed at all on this machine, actually, as your previous change makes the deployed vault backend use the objstorage backend config directly.

6

Does anything on that machine really use the swh::remote_service::vault variables?

19–21

Can be dropped if the objstorage rpc server deployment isn't needed.

25–26

Not sure that's needed?

site-modules/role/manifests/swh_vault.pp
5 ↗(On Diff #4926)

Can be dropped if we don't need remote access to the objstorage directly.

This revision now requires changes to proceed.Wed, May 22, 11:06 AM
ardumont marked 2 inline comments as done.Wed, May 22, 11:27 AM
ardumont added inline comments.
data/defaults.yaml
1375

The comment is wrong indeed.

data/hostname/vangogh.euwest.azure.internal.softwareheritage.org.yaml
3

Possibly, i ported the existing orangerie setup.
I'll check.

6

i misunderstood its use ;)
So to answer, no, nothing uses that.
I will remove it.

It will have to be defined in a next step in the defaults.yaml as the real switch between orangerie and this machine happens.

25–26

It's because that node runs on azure.
By default there, we have lots of indexer workers running.

site-modules/role/manifests/swh_vault.pp
5 ↗(On Diff #4926)

Right, that must be leftovers from some time ;)

ardumont updated this revision to Diff 4929.Wed, May 22, 11:52 AM
ardumont marked 2 inline comments as done.

Adapt according to review:

  • vault: Setup new vangogh server
  • vault: Remove no longer needed postgres profile
  • orangerie/orangeriedev: Remove duplicated setup
  • vault: Remove no longer needed objstorage profile
ardumont edited the summary of this revision. (Show Details)Wed, May 22, 11:55 AM
ardumont edited the summary of this revision. (Show Details)
ardumont added inline comments.Wed, May 22, 11:58 AM
data/defaults.yaml
1375

Mmm, yes about the usefulness...
I'll remove it when i'll effectively switch the vault from orangerie to vangogh.

olasd accepted this revision.Wed, May 22, 12:00 PM
This revision is now accepted and ready to land.Wed, May 22, 12:00 PM
ardumont updated this revision to Diff 4930.Wed, May 22, 12:05 PM

Rebase and plug to production branch

This revision was automatically updated to reflect the committed changes.