Differential D3667

nixguix: Load the content of sources.json as extrinsic snapshot metadata.
ClosedPublic
Actions

Authored by vlorentz on Jul 31 2020, 1:16 PM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Maniphest Tasks

T2512: Make all loaders write their extrinsic metadata to the appropriate storage

Commits

rDLDBASE55436f3925bf: nixguix: Load the content of sources.json as extrinsic snapshot metadata.

Test Plan

Tests will fail as they depend on swh.storage diffs: D3663 and D3666

Diff Detail

Repository

rDLDBASE Generic VCS/Package Loader

Branch

nixguix-sources-metadata

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 14141
Build 21726: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 21725: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	270 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.package.nixguix.tests.test_nixguix::test_loader_one_visit swh_config = '/tmp/pytest-of-jenkins/pytest-0/test_loader_one_visit0/loader.yml' requests_mock_datadir = <requests_mock.mocker.Mocker object at 0x7fcea2794198> raw_sources = b'{\n "sources": [\n {\n "type": "url",\n "urls": [ "https://github.com/owner-1/repository-1/revision-1....Mcx6QctleltspAgXlM="\n }\n ],\n "version": "1",\n "revision": "cc4e04c26672dd74e5fd0fecb78b435fb55368f7"\n}\n'
	4 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.core.tests.test_converters::test_content_for_storage_data
	12 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.core.tests.test_converters::test_content_for_storage_path
	4 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.core.tests.test_converters::test_content_for_storage_too_long
	4 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.core.tests.test_converters::test_prepare_contents
		View Full Test Results (1 Failed · 141 Passed)

Event Timeline

vlorentz created this revision.Jul 31 2020, 1:16 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJul 31 2020, 1:16 PM

Build is green

Patch application report for D3667 (id=12901)

Rebasing onto 3b97703d7f...

Current branch diff-target is up to date.

Changes applied before test

commit ddf55588d76d32e27d4f6374f22057b5c295daac
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 31 13:15:34 2020 +0200

    nixguix: Load the content of sources.json as extrinsic snapshot metadata.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/231/ for more details.

Harbormaster completed remote builds in B14139: Diff 12901.Jul 31 2020, 1:19 PM

oops, that was code that only works with the current version of swh.storage, not the next one.

Build has FAILED

Patch application report for D3667 (id=12903)

Rebasing onto 3b97703d7f...

Current branch diff-target is up to date.

Changes applied before test

commit 4e6d71e3d1f24e893865f33f4c2e267543090825
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 31 13:15:34 2020 +0200

    nixguix: Load the content of sources.json as extrinsic snapshot metadata.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/232/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/232/console

Harbormaster failed remote builds in B14141: Diff 12903!Jul 31 2020, 1:22 PM

vlorentz added a task: T2512: Make all loaders write their extrinsic metadata to the appropriate storage.Jul 31 2020, 2:25 PM

rebase on D3672

Build is green

Patch application report for D3667 (id=12922)

Rebasing onto 3b97703d7f...

Current branch diff-target is up to date.

Changes applied before test

commit 55436f3925bf2a0e37eb3a5f2a60fb98fd7c80fe
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 31 13:15:34 2020 +0200

    nixguix: Load the content of sources.json as extrinsic snapshot metadata.

commit d982889286dffe73c2003ff2252d6b1854f42402
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 31 14:54:41 2020 +0200

    Update tests to accept PagedResult from storage.raw_extrinsic_metadata_get.
    
    It will no longer return dicts in swh-storage v0.11.8.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/235/ for more details.

Harbormaster completed remote builds in B14160: Diff 12922.Jul 31 2020, 3:23 PM

vlorentz added a child revision: D3673: Make retrieve_sources use generic api_info instead of duplicating its code..Jul 31 2020, 3:27 PM

ardumont accepted this revision.Jul 31 2020, 3:31 PM

This revision is now accepted and ready to land.Jul 31 2020, 3:31 PM

Closed by commit rDLDBASE55436f3925bf: nixguix: Load the content of sources.json as extrinsic snapshot metadata. (authored by vlorentz). · Explain WhyJul 31 2020, 3:44 PM

This revision was automatically updated to reflect the committed changes.

vlorentz added a commit: rDLDBASE55436f3925bf: nixguix: Load the content of sources.json as extrinsic snapshot metadata..