Build is green

Build is green

rework commit message again (last time!)

• Adapt according to review (log.exception, prefer unexpected to divergent)
• Rework commit message to align with the diff

D2949 updated accordingly

Build is green

Drop unnecessary log instructions

Build is green

• Fix existing source.json which should not have been modified in the first place
• Add test scenario around the modification the diff introduces

Build has FAILED

Rebase on latest master

It's not invalid in the end, we hit it in production now.

and done:

May 25 17:17:10 worker01 python3[603377]: [2020-05-25 17:17:10,297: INFO/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[5be5167d-3783-4fc9-b18e-116424949d0a] succeeded in 3153.633277214016s: {'status': 'eventful', 'snapshot_id': '76f38fb09efe9f10305c50f5cf083b008217256f'}

Even simpler than D2949, filtering out the culprit branch "evaluation" (targetting nixpkgs) and we are back on track.

Replaced by D3178

Build has FAILED

• nixguix: Add a test around the wrong metadata revision structure detection

Build is green

Rework according to review
(still working on a test)

Working on adding one test which pass in there (load from archive loader, then load from nixguix whose listing list the same artifact...)

I actually don't hit the issue in a test...

Well, i did through patching...

Build is green

Rework commit message

Build is green

• Rebase on latest master
• when a missing key is triggered, continue checking for other revisions of the snapshot

Working on adding one test which pass in there (load from archive loader, then load from nixguix whose listing list the same artifact...)

Working on adding one test which pass in there (load from archive loader, then load from nixguix whose listing list the same artifact...)

It's not invalid in the end, we hit it in production now.

And the next visit run got hit by T2371.

As a workaround and tryout (unsure if it would have been enough), I stopped
other service workers and rescheduled one. And the run did finish this time
[1]

Heads up.

And the first run crashed...

May 22 03:38:53 worker01 python3[275651]: Process 'ForkPoolWorker-1' pid:275685 exited with 'signal 9 (SIGKILL)'                                                                                                                               May 22 03:38:54 worker01 python3[275651]: [2020-05-22 03:38:54,044: ERROR/MainProcess] Task handler raised error: WorkerLostError('Worker exited prematurely: signal 9 (SIGKILL).')
                                          Traceback (most recent call last):
                                            File "/usr/lib/python3/dist-packages/billiard/pool.py", line 1267, in mark_as_worker_lost
                                              human_status(exitcode)),
                                          billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 9 (SIGKILL).

Still running so more than a day indeed.

Started only 1 worker (01). The first run may take more than 1 day. And the
frequency is 1 day.

For people witch access to the staging infra, the latest snapshot is browsable [1]

For information, this has been running on staging daily for a while now without much issues. The sources.json are nix specifics for now.

The revisions in question were most probably already loaded with prior runs on
anterior nixguix loader version. So the metadata there was set. And now, it can
never change to the new format.

Deployed.