Page MenuHomeSoftware Heritage

thanos: Configure gateway service to expose historical data
ClosedPublic

Authored by ardumont on Jul 8 2022, 11:06 AM.

Details

Summary

This exposes the gateway service around the storage bucket where historical data is
stored (by sidecar services). This makes this service be referenced in the thanos query
configuration so it can be queried as well.

Related to T4385
Depends on D8092

Test Plan

vagrant testing.

root@thanos:~# cat /etc/thanos/query-sd.yaml
---
- targets:
  - mmca.softwareheritage.org:443
  - 10.168.100.29:19090
  - 10.168.50.90:19093
(reverse-i-search)`cat ': ^Ct /etc/thanos/query-sd.yaml
root@thanos:~# systemctl cat thanos-gateway | grep -C3 store
[Service]
Restart=on-failure
User=prometheus
ExecStart=/opt/thanos/current/thanos store --data-dir=/var/lib/prometheus/metrics2 \
    --objstore.config-file=/etc/thanos/objstore.yml \
    --http-address=10.168.50.90:19192 \
    --grpc-address=10.168.50.90:19093
ExecReload=/bin/kill -HUP $MAINPID

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

btw:

11:19 <+olasd> ardumont: we want the gateway services to run on the central Thanos host, don't we?
11:19 <+olasd> (to reduce the number of open ports and cross-tenant dependencies)
11:46 <+ardumont> Not sure we can? It wants access to the disk store? I'll double check
11:50 <+olasd> there's no disk store
11:50 <+olasd> only azure objstorage
ardumont edited the test plan for this revision. (Show Details)
ardumont edited the test plan for this revision. (Show Details)

Adapt according to suggestion.

Make the gateway service run on the thanos host.

Rebase and reuse adaptation from D8092

This revision was not accepted when it landed; it landed in state Needs Review.Jul 12 2022, 3:59 PM
This revision was automatically updated to reflect the committed changes.