Paths

Table of Contentst

Differential D2854

[wip] package.loader: introduce load_target and resolve_target_from
Changes PlannedPublic
Actions

Authored by lewo on Mar 19 2020, 10:51 AM.

Tags

None

Subscribers

Details

Reviewers

None

Group Reviewers

Summary

DO NOT REVIEW IT YET!

Diff Detail

Repository

rDLDBASE Generic VCS/Package Loader

Branch

functional-file

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 11218
Build 16940: tox-on-jenkins	Jenkins
Build 16939: arc lint + arc unit

Event Timeline

lewo created this revision.Mar 19 2020, 10:51 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptMar 19 2020, 10:51 AM

Harbormaster failed remote builds in B11217: Diff 10151!Mar 19 2020, 10:51 AM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tox/368/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tox/368/console

lewo edited the summary of this revision. (Show Details)Mar 19 2020, 10:52 AM

lewo mentioned this in T1991: Implement a Guix/Nix loader.Mar 19 2020, 11:09 AM

WIP

Harbormaster failed remote builds in B11218: Diff 10152!Mar 19 2020, 11:22 AM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tox/369/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tox/369/console

@ardumont I don't understand why the storage doesn't contains 2 contents (a tarball + a file). The test test_loader_one_visit is failing because there is only 1 content in the storage.

ardumont added inline comments.Mar 19 2020, 1:26 PM

swh/loader/package/functional/loader.py
144	beware `to_model` can return a `Content` or a `SkippedContent`. That will not be the same endpoint for the storage: Content: `self.storage.content_add` SkippedContent: `self.storage.skipped_content_add`

@ardumont I don't understand why the storage doesn't contains 2 contents (a tarball + a file). The test test_loader_one_visit is failing because there is only 1 content in the storage.

It happens to be the same content...
the 2 files you have in the revision.tar.gz and the second file targetted by url in your sample are the same.
It's then deduplicated in the storage...

Change the content of the targetted file
and you should be good to go ;)

This needs a rebase as there were quite some work on the loader core ;)

Cheers,

swh/loader/package/functional/loader.py
157	Discussing this case in [1], in the end, we could simply add a SnapshotBranch which targets a TargetType Content. No need to create a Release object. [1] https://forge.softwareheritage.org/D3745?id=13193#inline-25905

ardumont mentioned this in D3745: nixguix: Make the unsupported artifact extensions configurable.Aug 8 2020, 1:31 PM

ardumont mentioned this in T2879: Finalize nixguix loader implementation.Dec 11 2020, 3:42 PM

Revision Contents
Changeset List

Path

Size

swh/

loader/

package/

functional/

48 lines

tests/

test_functional.py

6 lines

33 lines

Diff 10152

swh/loader/package/functional/loader.py

Loading...

swh/loader/package/functional/tests/test_functional.py

Loading...

swh/loader/package/loader.py

Loading...