Either the manifest referenced the wrong integrity for that urls (or someone updated in
place the urls...):

fwiw, this is working in docker ^.

Run through docker for directory:

Fix documentation references and tests according to latest change in visit_type

Use correct visit_type {content, directory} in the listed origins outputted from the
lister.

zut, forgot to update the mock setup ¯\_(ツ)_/¯

Use proper range of commits to update the diff

Fix missing integrity field (this makes for a redundant check in tests, adding
missing integrity to actual test correctly this entry)

Fix constructor mess

docker run on the lister:

17:36:23 swh-scheduler@localhost:5433=# select now(), visit_type, lister_id, count(*) from listed_origins where lister_id = ( select id from listers where name='nixguix' and instance_name='nix-community.github.io') group by visit_type, lister_id;
+-------------------------------+------------+--------------------------------------+-------+
|              now              | visit_type |              lister_id               | count |
+-------------------------------+------------+--------------------------------------+-------+
| 2022-10-03 15:44:20.179895+00 | git        | 3f5c040a-6247-4ef3-a812-36f4b9ceafeb |     1 |
| 2022-10-03 15:44:20.179895+00 | file       | 3f5c040a-6247-4ef3-a812-36f4b9ceafeb |    87 |
| 2022-10-03 15:44:20.179895+00 | tar        | 3f5c040a-6247-4ef3-a812-36f4b9ceafeb | 31130 |
+-------------------------------+------------+--------------------------------------+-------+
(3 rows)

• Adapt last review comment (reuse swh.core's mimetype dictionary)
• Format warning log
• Decrease log verbosity for one debug instruction (it was in info for test purposes)
• Skip artifacts with missing integrity field too (nixpkgs dataset has some)

ah! yeah, it's the first time we are scheduling ListedOrigins for non package loaders...
Thanks for raising this up!

Adapt according to review (except for reusing the mimetype from swh.core.tarballs, i'll
do that in another iteration if you don't mind)

Add missing coverage

oh yeah, correct, thx.
It's possible it used to so until it got extracted in the package loader instead.

Adapt according to latest review

Rebase

Rebase and adapt according to:

• reviews
• docker runs [1]

Tag and released v1.2.3, jenkins up to you now, tia!

Adapt according to reviews

ack on what you said both ;)

Docker run made apparent that determining the urls' nature is not so simple...
I'll amend accordingly and take into account review suggestions.

Fix build

Rebase (this includes a work around to fix master build)

Maybe a new upstream version bump on importlib-metadata?

[1]

...
importlib-metadata==5.0.0

Build has FAILED

Adapt both Content and Directory loaders to deal with checksums dict instead of
integrity.

• nixguix: Transform integrity field into a checksums dict
• nixguix: Register task
• nixguix: Add a small lister introduction

• Build content out of MultiHash
• Add support for sha512 hash
• Fix input test checksum

That's also something needed for the ongoing directory/content loader [1]

Refactoring step

Rebase

Thanks! I'll close this.

Model objects should map 1:1 with what is stored in the archive.

Why are the changes to the model object needed, instead of just hashing the file directly?

Refactoring steps

Compute expected checksum to check integrity outside the loop

I went for a minimal changes for that untested code as I am mostly planning a full
overhaul of that area. So I did not spent to effort to add new test (the code isn't
currently directly tested either as far as I understand).