Page MenuHomeSoftware Heritage

Use the new object_metadata_add endpoint instead of origin_metadata_add.
ClosedPublic

Authored by vlorentz on Jul 10 2020, 11:43 AM.

Details

Summary

Note: this changes the authority type from 'deposit_client' to 'deposit',
so we'll have to run a migration on the storage at some point to make
that consistent.

Test Plan

tests will fail until D3456 is merged

Diff Detail

Repository
rDLDBASE Generic VCS/Package Loader
Branch
object_metadata_add
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 13567
Build 20765: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 20764: arc lint + arc unit

Unit TestsFailed

TimeTest
949 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.package.deposit.tests.test_deposit::test_deposit_loading_failure_to_retrieve_1_artifact
self = <Retrying object at 0x7f09277204a8 (stop=<tenacity.stop.stop_after_attempt object at 0x7f0927656358>, wait=<tenacity.w...0x7f09276580d0>, before=<function before_nothing at 0x7f09276c6bf8>, after=<function after_nothing at 0x7f0927692620>)> fn = <function RetryingProxyStorage.metadata_authority_add at 0x7f092771d9d8> args = (<swh.storage.retry.RetryingProxyStorage object at 0x7f092502e978>, [MetadataAuthority(type=<MetadataAuthorityType.DEP...rl='https://hal-test.archives-ouvertes.fr/', metadata=<swh.model.collections.ImmutableDict object at 0x7f0925031b38>)])
1,580 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.package.deposit.tests.test_deposit::test_deposit_loading_ok
self = <Retrying object at 0x7f09277204a8 (stop=<tenacity.stop.stop_after_attempt object at 0x7f0927656358>, wait=<tenacity.w...0x7f09276580d0>, before=<function before_nothing at 0x7f09276c6bf8>, after=<function after_nothing at 0x7f0927692620>)> fn = <function RetryingProxyStorage.metadata_authority_add at 0x7f092771d9d8> args = (<swh.storage.retry.RetryingProxyStorage object at 0x7f09258fc5c0>, [MetadataAuthority(type=<MetadataAuthorityType.DEP...rl='https://hal-test.archives-ouvertes.fr/', metadata=<swh.model.collections.ImmutableDict object at 0x7f0925020898>)])
1,437 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.package.deposit.tests.test_deposit::test_deposit_loading_ok_2
self = <Retrying object at 0x7f09277204a8 (stop=<tenacity.stop.stop_after_attempt object at 0x7f0927656358>, wait=<tenacity.w...0x7f09276580d0>, before=<function before_nothing at 0x7f09276c6bf8>, after=<function after_nothing at 0x7f0927692620>)> fn = <function RetryingProxyStorage.metadata_authority_add at 0x7f092771d9d8> args = (<swh.storage.retry.RetryingProxyStorage object at 0x7f0924f4e1d0>, [MetadataAuthority(type=<MetadataAuthorityType.DEP...rl='https://hal-test.archives-ouvertes.fr/', metadata=<swh.model.collections.ImmutableDict object at 0x7f0924f656a0>)])
3,161 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.package.deposit.tests.test_deposit::test_revision_metadata_structure
self = <Retrying object at 0x7f09277204a8 (stop=<tenacity.stop.stop_after_attempt object at 0x7f0927656358>, wait=<tenacity.w...0x7f09276580d0>, before=<function before_nothing at 0x7f09276c6bf8>, after=<function after_nothing at 0x7f0927692620>)> fn = <function RetryingProxyStorage.metadata_authority_add at 0x7f092771d9d8> args = (<swh.storage.retry.RetryingProxyStorage object at 0x7f0925972160>, [MetadataAuthority(type=<MetadataAuthorityType.DEP...rl='https://hal-test.archives-ouvertes.fr/', metadata=<swh.model.collections.ImmutableDict object at 0x7f09259f1358>)])
2 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.core.tests.test_converters::test_content_for_storage_data
View Full Test Results (4 Failed · 138 Passed)

Event Timeline

Build has FAILED

Patch application report for D3491 (id=12344)

Rebasing onto ed4e0b38c1...

Current branch diff-target is up to date.
Changes applied before test
commit 10bf08d147443d8071cf5d20e2c56e0e94dc8ed2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 10 11:42:05 2020 +0200

    Use the new object_metadata_add endpoint instead of origin_metadata_add.
    
    Note: this changes the authority type from 'deposit_client' to 'deposit',
    so we'll have to run a migration on the storage at some point to make
    that consistent.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/128/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/128/console

This revision is now accepted and ready to land.Jul 10 2020, 12:02 PM

Build has FAILED

Patch application report for D3491 (id=12538)

Rebasing onto 3442689ecb...

Current branch diff-target is up to date.
Changes applied before test
commit 6400a426824d01661225ced4bf9a0208f42a23a7
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 10 11:42:05 2020 +0200

    Use the new object_metadata_add endpoint instead of origin_metadata_add.
    
    Note: this changes the authority type from 'deposit_client' to 'deposit',
    so we'll have to run a migration on the storage at some point to make
    that consistent.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/182/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/182/console

Build is green

Patch application report for D3491 (id=12538)

Rebasing onto 3442689ecb...

Current branch diff-target is up to date.
Changes applied before test
commit 6400a426824d01661225ced4bf9a0208f42a23a7
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 10 11:42:05 2020 +0200

    Use the new object_metadata_add endpoint instead of origin_metadata_add.
    
    Note: this changes the authority type from 'deposit_client' to 'deposit',
    so we'll have to run a migration on the storage at some point to make
    that consistent.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/183/ for more details.