Differential D5139

deposit: Update deposit status when the load status is 'partial'
ClosedPublic
Actions

Authored by vlorentz on Feb 25 2021, 11:45 AM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Maniphest Tasks

T3070: Fix longstanding deposit status verified which is usually a symptom of loading failure

Commits

rDLDBASEb5a7fee0725e: deposit: Update deposit status when the load status is 'partial'

Summary

Before this commit, load status 'success' and 'failed' updated the
deposit status accordingly, but other statuses were ignored.

Partially fixes T3070.

Diff Detail

Repository

rDLDBASE Generic VCS/Package Loader

Branch

deposit-partial

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 19456
Build 30183: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 30182: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	397 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.package.deposit.tests.test_deposit::test_deposit_loading_ok swh_storage = <swh.storage.retry.RetryingProxyStorage object at 0x7f770ea9afd0> deposit_client = <swh.loader.package.deposit.loader.ApiClient object at 0x7f770ea35668> requests_mock_datadir = <requests_mock.mocker.Mocker object at 0x7f770ea353c8>
	391 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.package.deposit.tests.test_deposit::test_deposit_loading_ok_2 swh_storage = <swh.storage.retry.RetryingProxyStorage object at 0x7f770ea8c400> deposit_client = <swh.loader.package.deposit.loader.ApiClient object at 0x7f770ea5bc88> requests_mock_datadir = <requests_mock.mocker.Mocker object at 0x7f770ea5bf28>
	2 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.core.tests.test_converters::test_content_for_storage_data
	4 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.core.tests.test_converters::test_content_for_storage_path
	1 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.core.tests.test_converters::test_content_for_storage_too_long
		View Full Test Results (2 Failed · 154 Passed)

Event Timeline

vlorentz created this revision.Feb 25 2021, 11:45 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptFeb 25 2021, 11:45 AM

Build has FAILED

Patch application report for D5139 (id=18378)

Rebasing onto 52626cbb04...

Current branch diff-target is up to date.

Changes applied before test

commit 9eb37bda3161705fb238227fed7e61a3eecb0426
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 25 11:44:24 2021 +0100

    deposit: Update deposit status when the load status is 'partial'
    
    Before this commit, load status 'success' and 'failed' updated the
    deposit status accordingly, but other statuses were ignored.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/381/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/381/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 25 2021, 11:47 AM

Harbormaster failed remote builds in B19456: Diff 18378!

fix successful case, using D5140

Build is green

Patch application report for D5139 (id=18381)

Could not rebase; Attempt merge onto 52626cbb04...

Updating 52626cb..b5a7fee
Fast-forward
 swh/loader/package/deposit/loader.py             |   7 +-
 swh/loader/package/deposit/tests/test_deposit.py |  18 ++++
 swh/loader/package/loader.py                     | 128 ++++++++++++++---------
 3 files changed, 102 insertions(+), 51 deletions(-)

Changes applied before test

commit b5a7fee0725e6be6e62715980a78bce4c39c9419
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 25 11:44:24 2021 +0100

    deposit: Update deposit status when the load status is 'partial'
    
    Before this commit, load status 'success' and 'failed' updated the
    deposit status accordingly, but other statuses were ignored.

commit e80919ed8e7bf882746faddda19706139ffa161f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 25 11:48:24 2021 +0100

    Make finalize_visit a method instead of nested function.
    
    I don't see the point of it being a nested function, it just makes load()
    larger.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/384/ for more details.

Harbormaster completed remote builds in B19459: Diff 18381.Feb 25 2021, 12:03 PM

vlorentz requested review of this revision.Feb 25 2021, 12:03 PM