Page MenuHomeSoftware Heritage

Deploy swh-scrubber on all storage instances
Closed, ResolvedPublic

Description

It's currently running on the main storage implementation belvedere.

It needs to be deployed on all storage instances independently on how storage instances are populated (wal replication, ...).

As a consequence, scrubber must be deployed to scrub somerset as well.

Event Timeline

ardumont triaged this task as Normal priority.Jul 4 2022, 10:24 AM
ardumont created this task.
ardumont updated the task description. (Show Details)

Deployed both in staging and production [1]:

15:48:20 swh-scrubber@belvedere:5432=> select now(), datastore, count(*) from corrupt_object group by datastore;
+-------------------------------+-----------+-------+
|              now              | datastore | count |
+-------------------------------+-----------+-------+
| 2022-08-04 13:48:26.124778+00 |      4997 |  4482 |
| 2022-08-04 13:48:26.124778+00 |         1 | 90899 |
+-------------------------------+-----------+-------+
(2 rows)

Time: 152.881 ms
15:48:26 swh-scrubber@belvedere:5432=> select now(), * from datastore;
+-------------------------------+------+---------+------------+-------------------------------------------------------------------------------------------------------+
|              now              |  id  | package |   class    |                                               instance                                                |
+-------------------------------+------+---------+------------+-------------------------------------------------------------------------------------------------------+
| 2022-08-04 13:48:35.116671+00 |    1 | storage | postgresql | user=guest password=xxx dbname=softwareheritage host=db.internal.softwareheritage.org port=5432       |
| 2022-08-04 13:48:35.116671+00 | 4997 | storage | postgresql | user=guest password=xxx dbname=softwareheritage host=somerset.internal.softwareheritage.org port=5432 |
+-------------------------------+------+---------+------------+-------------------------------------------------------------------------------------------------------+
(2 rows)

Time: 69.960 ms
ardumont claimed this task.
ardumont moved this task from deployed/landed/monitoring to done on the System administration board.