And the next visit run got hit by T2371.

As a workaround and tryout (unsure if it would have been enough), I stopped
other service workers and rescheduled one. And the run did finish this time
[1]

Heads up.

And the first run crashed...

May 22 03:38:53 worker01 python3[275651]: Process 'ForkPoolWorker-1' pid:275685 exited with 'signal 9 (SIGKILL)'                                                                                                                               May 22 03:38:54 worker01 python3[275651]: [2020-05-22 03:38:54,044: ERROR/MainProcess] Task handler raised error: WorkerLostError('Worker exited prematurely: signal 9 (SIGKILL).')
                                          Traceback (most recent call last):
                                            File "/usr/lib/python3/dist-packages/billiard/pool.py", line 1267, in mark_as_worker_lost
                                              human_status(exitcode)),
                                          billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 9 (SIGKILL).

Still running so more than a day indeed.

Started only 1 worker (01). The first run may take more than 1 day. And the
frequency is 1 day.

For people witch access to the staging infra, the latest snapshot is browsable [1]

For information, this has been running on staging daily for a while now without much issues. The sources.json are nix specifics for now.

The revisions in question were most probably already loaded with prior runs on
anterior nixguix loader version. So the metadata there was set. And now, it can
never change to the new format.

Deployed.

Fixed, landed, tagged and deployed.

mfw i noticed my typo in the git commit

Note that only the deposit loader is impacted and [1] fixes it.

Same as T2352#43469

and it finished alright \m/

while not finished, run is still happy so far

Deployed.

Deployed.