Page MenuHomeSoftware Heritage

Deploy swh.loader.core v2.1
Closed, ResolvedPublic

Description

It includes a fix to *again* unstuck the nixguix loader.
Previous fixes (T3763) only unstuck it once and then fails on subsequent ones.

  • Tag (@vlorentz did)
  • release package ok
  • Deploy on workers
  • Set tag on grafana about the deployment
  • Restart swh-worker@loader_nixguix service [1]
  • Trigger the loader nixguix again [2]
  • Restart other swh-workers@loader_$package for package in {deposit, debian, ...} [3]
  • Is the nixguix unstuck? -> No.
  • Report issue
  • D6816: Review fix
  • loop over the first instructions for v2.1.1 (when tagged)
  • Are nixguix loading visits ok?

[1]

root@pergamon:~# clush -b -w @staging-loader-workers -w @swh-workers "systemctl restart swh-worker@loader_nixguix"

[2]

09:16:33 softwareheritage-scheduler@belvedere:5432=> update task set next_run=now(), status='next_run_not_scheduled' where type='load-nixguix' and status!='disabled';
UPDATE 2
Time: 261.256 ms
09:17:12 softwareheritage-scheduler@belvedere:5432=> select * from task where type='load-nixguix' and status!='disabled';
+-[ RECORD 1 ]-----+------------------------------------------------------------------------------------------------------+
| id               | 334411727                                                                                            |
| type             | load-nixguix                                                                                         |
| arguments        | {"args": [], "kwargs": {"url": "https://nix-community.github.io/nixpkgs-swh/sources-unstable.json"}} |
| next_run         | 2021-12-10 08:17:12.764649+00                                                                        |
| current_interval | 1 day                                                                                                |
| status           | next_run_scheduled                                                                                   |
| policy           | recurring                                                                                            |
| retries_left     | 3                                                                                                    |
| priority         | (null)                                                                                               |
+-[ RECORD 2 ]-----+------------------------------------------------------------------------------------------------------+
| id               | 337282717                                                                                            |
| type             | load-nixguix                                                                                         |
| arguments        | {"args": [], "kwargs": {"url": "https://guix.gnu.org/sources.json"}}                                 |
| next_run         | 2021-12-10 08:17:12.764649+00                                                                        |
| current_interval | 1 day                                                                                                |
| status           | next_run_scheduled                                                                                   |
| policy           | recurring                                                                                            |
| retries_left     | 3                                                                                                    |
| priority         | (null)                                                                                               |
+------------------+------------------------------------------------------------------------------------------------------+

Time: 15.690 ms

[3]

root@pergamon:~# clush -b -w @swh-workers -w @staging-loader-workers 'for type in debian archive cran opam pypi npm deposit; do systemctl restart swh-worker@loader_${type}; done'

Event Timeline

ardumont changed the task status from Open to Work in Progress.Dec 9 2021, 4:26 PM
ardumont triaged this task as High priority.
ardumont created this task.
ardumont moved this task from Backlog to in-progress on the System administration board.
ardumont updated the task description. (Show Details)
ardumont added a subscriber: vlorentz.
ardumont updated the task description. (Show Details)
ardumont updated the task description. (Show Details)

Nixguix loader is ok.

root@pergamon:~# date; clush -b -w @swh-workers systemctl status swh-worker@loader_nixguix | grep succeeded
Sun 12 Dec 2021 04:14:37 PM UTC
Dec 11 19:10:47 worker13 python3[2780]: [2021-12-11 19:10:47,371: INFO/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[2541de72-6471-4766-960c-aba768aa6274] succeeded in 2493.3735876400024s: {'status': 'eventful', 'snapshot_id': 'be11862d047bcd7b6f8ebd8f417ffa6e545b7bc9'}
Dec 12 00:48:04 worker14 python3[2810]: [2021-12-12 00:48:04,783: INFO/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[c835fdd9-29e4-4584-a890-e38d1d7e845e] succeeded in 1039.6907857869955s: {'status': 'eventful', 'snapshot_id': '2893d092115f3d9b535d64c583f95b94e82f87f1'}
Dec 11 00:28:07 worker15 python3[2801]: [2021-12-11 00:28:07,621: INFO/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[eaa2e617-7855-40f7-a80b-456356f89e6c] succeeded in 24255.011169397003s: {'status': 'eventful', 'snapshot_id': '6f79b0c18dda0459b154003bbe899d50f183a8f8'}
Dec 10 18:27:12 worker16 python3[2797]: [2021-12-10 18:27:12,959: INFO/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[faf7a510-8548-443b-8d80-d27798a65a46] succeeded in 2600.335359801s: {'status': 'eventful', 'snapshot_id': '389ee393260c375915652389ce9ff4fd5712b9d7'}
ardumont claimed this task.
ardumont moved this task from deployed/landed/monitoring to done on the System administration board.