Page MenuHomeSoftware Heritage

production: Deploy swh-scrubber database and checkers
Closed, MigratedEdits Locked

Description

Plan (inspired by T4102):

  • Install scrubber (postgres) db (for production manually done) [1]
  • D8034: Configure db connection (puppet)
  • Check connection is ok
  • D8035: Update documentation on how to connect to db
  • Install schema in swh-scrubber db [2]
  • Grand read-only access to guest user
  • Update inventory with new scrubber node [3]
  • D8038: Provision node for checkers services
  • D8039: Deploy checker services
  • D8041: Ensure services start properly
  • D8041 (test plan): Checks everything is fine

[1]

postgres@belvedere:~$ createuser -p 5434 -P swh-scrubber
Enter password for new role:
Enter it again:
postgres@belvedere:~$ createdb -p 5434 --lc-ctype=C.UTF-8 -T template1 -O swh-scrubber swh-scrubber

[2]

$ ~/work/inria/bin/setup-db.sh
+ DBPORT=5432
+ DBPASSWORD=<redacted>
+ DBUSER=swh-scrubber
+ DBNAME=swh-scrubber
+ DBHOST=belvedere.internal.softwareheritage.org
+ CONNSTRING=postgresql://swh-scrubber:<redacted>@belvedere.internal.softwareheritage.org:5432/swh-scrubber
+ swh db init-admin --dbname postgresql://swh-scrubber:<redacted>@belvedere.internal.softwareheritage.org:5432/swh-scrubber scrubber
+ swh db init --dbname postgresql://swh-scrubber:<redacted>@belvedere.internal.softwareheritage.org:5432/swh-scrubber scrubber
psql:/home/tony/work/inria/repo/swh/swh-environment/swh-core/swh/core/db/sql/36-dbmodule.sql:12: NOTICE:  relation "dbmodule" already exists, skipping
INFO:swh.core.cli.db:Initializing database version to 1 from the command line option --initial-version
DONE database for scrubber initialized at version 1

[3] https://inventory.internal.admin.swh.network/virtualization/virtual-machines/121/

Event Timeline

vlorentz triaged this task as Normal priority.Jun 10 2022, 11:53 AM
vlorentz created this task.
vlorentz updated the task description. (Show Details)
ardumont changed the task status from Open to Work in Progress.Jun 24 2022, 4:15 PM
ardumont updated the task description. (Show Details)
ardumont moved this task from Weekly backlog to in-progress on the System administration board.
ardumont updated the task description. (Show Details)
ardumont shifted this object from the S1 Public space to the Restricted Space space.
ardumont shifted this object from the Restricted Space space to the S1 Public space.Jun 27 2022, 2:37 PM
ardumont updated the task description. (Show Details)

Seems the job is well on its way:

07:59:35 swh-scrubber@belvedere:5432=> select now(), count(*) from corrupt_object ;
+-------------------------------+-------+
|              now              | count |
+-------------------------------+-------+
| 2022-06-28 05:59:39.490737+00 | 76511 |
+-------------------------------+-------+
(1 row)

Time: 25.700 ms
ardumont claimed this task.
ardumont moved this task from deployed/landed/monitoring to done on the System administration board.