Page MenuHomeSoftware Heritage

vault: Setup new vangogh server
ClosedPublic

Authored by ardumont on May 22 2019, 9:58 AM.

Details

Summary

Prepare the vault vm to expose the vault api.

The vault api is plugged to:

  • db in prado
  • "azure" objstorage
  • "remote" storage already setuped in azure (storage0)

The objstorage is plugged to an azure blobstorage (as defined in D1495)

Note:

  • As the db used is no longer installed on the same node, the postgres profile has been removed from the vault role
  • The objstorage cache backend was exposed but unused, this was removed. As such, the vault role has been updated to remove the objstorage profile as well.

Related T1716

Test Plan
$ cd puppet-environment
$ bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to update_configuration vangogh.euwest.azure.internal.softwareheritage.org

Output: P404

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont retitled this revision from vault: Prepare new vangogh vault vm to vault: Setup new vangogh server.May 22 2019, 10:12 AM
ardumont added projects: Vault, Puppet recipes.
olasd requested changes to this revision.May 22 2019, 11:06 AM

Thanks for this change!

I think there's a lot more opportunity for cleanup here which I've outlined :)

data/defaults.yaml
1374

I don't think this comment is accurate (I'm also not sure it's all that useful)

data/hostname/vangogh.euwest.azure.internal.softwareheritage.org.yaml
2

Is this really used? I think all "cache objstorage" requests are proxied through the vault rpc server.

I don't think we need the objstorage rpc server deployed at all on this machine, actually, as your previous change makes the deployed vault backend use the objstorage backend config directly.

5

Does anything on that machine really use the swh::remote_service::vault variables?

18–20

Can be dropped if the objstorage rpc server deployment isn't needed.

24–25

Not sure that's needed?

site-modules/role/manifests/swh_vault.pp
5 ↗(On Diff #4926)

Can be dropped if we don't need remote access to the objstorage directly.

This revision now requires changes to proceed.May 22 2019, 11:06 AM
ardumont added inline comments.
data/defaults.yaml
1374

The comment is wrong indeed.

data/hostname/vangogh.euwest.azure.internal.softwareheritage.org.yaml
2

Possibly, i ported the existing orangerie setup.
I'll check.

5

i misunderstood its use ;)
So to answer, no, nothing uses that.
I will remove it.

It will have to be defined in a next step in the defaults.yaml as the real switch between orangerie and this machine happens.

24–25

It's because that node runs on azure.
By default there, we have lots of indexer workers running.

site-modules/role/manifests/swh_vault.pp
5 ↗(On Diff #4926)

Right, that must be leftovers from some time ;)

ardumont marked 2 inline comments as done.

Adapt according to review:

  • vault: Setup new vangogh server
  • vault: Remove no longer needed postgres profile
  • orangerie/orangeriedev: Remove duplicated setup
  • vault: Remove no longer needed objstorage profile
ardumont edited the summary of this revision. (Show Details)
data/defaults.yaml
1374

Mmm, yes about the usefulness...
I'll remove it when i'll effectively switch the vault from orangerie to vangogh.

This revision is now accepted and ready to land.May 22 2019, 12:00 PM

Rebase and plug to production branch

This revision was automatically updated to reflect the committed changes.