Move grafana on a dedicated server behind the admin RP
- use the swh postgresql profile to manage the datase
- install grafana, the plugin and the generated dashboard with puppet
- use the rp1 reverse proxy to expose the service
Related to T3817
Details
Details
- Reviewers
ardumont - Group Reviewers
System administrators - Maniphest Tasks
- T3817: Install grafana on its own server
- Commits
- rSPSITE8aa49be10300: Move grafana on a dedicated server behind the admin RP
This diff doesn't not impact the current grafana instance.
diff origin/production/pergamon.softwareheritage.org current/pergamon.softwareheritage.org
*******************************************
File[/etc/bind/keys/local-update] =>
parameters =>
content =>
@@ -2,4 +2,4 @@
key local-update {
algorithm hmac-sha256;
- secret "<redacted>";
+ secret "<redacted>";
};
*******************************************
File[/etc/bind/rndc.key] =>
parameters =>
content =>
@@ -2,4 +2,4 @@
key rndc-key {
algorithm hmac-md5;
- secret "<redacted>";
+ secret "<redacted>";
};
*******************************************
*** End octocatalog-diff on pergamon.softwareheritage.orgDiff Detail
Diff Detail
- Repository
- rSPSITE puppet-swh-site
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
Why the duplication of profile::grafana into profile::grafana::backend?
Do we really need the "swh postgresql" stuff for this very basic database? I find it much harder to follow than just having the existing two postgresql resources in profile::grafana.
Alternatively we should consider centralizing the postgresql databases used for admin stuff (we already have one for hedgedoc, one for netbox, and potentially the one for sentry) into a single dedicated host (on which we would be able to do more proper backups and monitoring).
| data/hostname/grafana0.internal.admin.swh.network.yaml | ||
|---|---|---|
| 14 ↗ | (On Diff #24918) | "storage"? |
| 20–21 ↗ | (On Diff #24918) | No such snippet exists. |
Comment Actions
Alternatively we should consider centralizing the postgresql databases used for admin stuff (we already have one for hedgedoc, one for netbox, and potentially the one for sentry) into a single dedicated host (on which we would be able to do more proper backups and monitoring).
I've opened T3833 for this good remark.
Comment Actions
- fix database name (not directly used by the configuration)
- fix the prometheus snippets configuration
Comment Actions
lgtm
one question inline about the pg version.
Also, in your vagrant tryouts, did you try to mount back the dump out of pergamon in the newly created grafana db?
| data/hostname/grafana0.internal.admin.swh.network.yaml | ||
|---|---|---|
| 2 ↗ | (On Diff #24969) | Since we are creating that new one, can't we directly go to 14 now (for one less db to upgrade later)? |
Comment Actions
thanks for the validation, I have some pending changes in progress and to reply to the olasd's remarks so I change the status to planned changes
Regarding the backup restoration in vagrant, yes I have successfully restored it locally, everything was good (except the data of the graphes because grafana request my empty local prometheus)
Comment Actions
- Fix the database password resolution after the database name update
- Restore the profile::grafana::objects call to manage the orgs and database declarations It's not ideal as it introduces a dependency with the reverse proxy
Comment Actions
- Use the postgresql module to configure the database Indeed, it's simpler
- fix a wrong hostname declaration for vagrant
- fix grafana plugin installation and configuration calls
Comment Actions
Why the duplication of profile::grafana into profile::grafana::backend?
The goal was to let the initial profile untouched during the migration and to clearly identify the public reverse_proxy is not installed by this part
Do we really need the "swh postgresql" stuff for this very basic database? I find it much harder to follow than just having the existing two postgresql resources in profile::grafana.
Nope you are right, it was clearly overkill
Comment Actions
- Rebase
- Use the centralized database instead of a local postgresql
- Move the prometheus sql exporter on dali