Page MenuHomeSoftware Heritage

Add deposit metadata update step in deposit icinga check
ClosedPublic

Authored by ardumont on Tue, Oct 13, 6:23 PM.

Details

Summary

This extends the current icinga checks on deposit to trigger a metadata update when the main deposit is done.
It retrieves the swhid from the deposit status calls.
Then triggers a metadata update on the deposit using the latest deposit client abilities (0.3).

Related to T2685

Depends on D4270

Test Plan

tox

Diff Detail

Repository
rDICP Icinga plugins
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Tue, Oct 13, 6:23 PM

Build has FAILED

Patch application report for D4252 (id=15015)

Rebasing onto 06275d4314...

Current branch diff-target is up to date.
Changes applied before test
commit 78d1fc9863f51e2e896327bd2145986ff933f50e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 13 18:23:22 2020 +0200

    wip: Update deposit scenario to actually check metadata update
    
    Related to T2685

Link to build: https://jenkins.softwareheritage.org/job/DICP/job/tests-on-diff/7/
See console output for more information: https://jenkins.softwareheritage.org/job/DICP/job/tests-on-diff/7/console

ardumont planned changes to this revision.Tue, Oct 13, 6:24 PM
ardumont updated this revision to Diff 15018.Tue, Oct 13, 6:48 PM

Make the first test pass \m/
-> still wip though

ardumont planned changes to this revision.Tue, Oct 13, 6:48 PM

Build has FAILED

Patch application report for D4252 (id=15018)

Could not rebase; Attempt merge onto 06275d4314...

Updating 06275d4..793070d
Fast-forward
 requirements-swh.txt                     |  4 +--
 swh/icinga_plugins/deposit.py            | 52 +++++++++++++++++++++++++++++---
 swh/icinga_plugins/tests/test_deposit.py | 32 ++++++++++++++++++--
 3 files changed, 80 insertions(+), 8 deletions(-)
Changes applied before test
commit 793070d9d307ff28559fef55a303790d94a8f4e6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 13 18:23:22 2020 +0200

    wip: Update deposit scenario to actually check metadata update
    
    Related to T2685

commit 6f9b0d99ee5dd11823b58cf12156ea21fa093454
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 13 18:25:03 2020 +0200

    requirements-swh: Drop spurious v in the version

Link to build: https://jenkins.softwareheritage.org/job/DICP/job/tests-on-diff/8/
See console output for more information: https://jenkins.softwareheritage.org/job/DICP/job/tests-on-diff/8/console

ardumont retitled this revision from wip: Update deposit scenario to actually check metadata update to wip: Update deposit scenario to add metadata update query.Tue, Oct 13, 6:50 PM
ardumont updated this revision to Diff 15024.Wed, Oct 14, 10:26 AM

Make tests green

ardumont retitled this revision from wip: Update deposit scenario to add metadata update query to Update deposit scenario to add metadata update query.Wed, Oct 14, 10:26 AM
ardumont edited the test plan for this revision. (Show Details)

Build is green

Patch application report for D4252 (id=15024)

Rebasing onto 6f9b0d99ee...

Current branch diff-target is up to date.
Changes applied before test
commit 54952684f1e6dbdd9c6c8c850f26ee97c7cebe0c
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 13 18:23:22 2020 +0200

    Update deposit scenario to actually check metadata update
    
    Related to T2685

See https://jenkins.softwareheritage.org/job/DICP/job/tests-on-diff/9/ for more details.

ardumont added inline comments.Wed, Oct 14, 10:42 AM
swh/icinga_plugins/deposit.py
166

currently adding that missing test

ardumont edited the summary of this revision. (Show Details)Wed, Oct 14, 10:51 AM
ardumont added a project: SWORD deposit.
ardumont retitled this revision from Update deposit scenario to add metadata update query to Add deposit metadata update step in deposit icinga check.Wed, Oct 14, 10:58 AM
ardumont updated this revision to Diff 15026.Wed, Oct 14, 11:03 AM
ardumont edited the test plan for this revision. (Show Details)

Add missing scenario

Build is green

Patch application report for D4252 (id=15026)

Rebasing onto 6f9b0d99ee...

Current branch diff-target is up to date.
Changes applied before test
commit ea3456363bff2fb1afbbb77e8d27330eb400ca4f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 13 18:23:22 2020 +0200

    Add deposit metadata update step in deposit icinga check
    
    This extends the current icinga checks on deposit to trigger a metadata update
    when the main deposit is done. It retrieves the swhid from the deposit status
    calls. Then triggers a metadata update on the deposit using the latest deposit
    client abilities (0.3).
    
    Related to T2685

See https://jenkins.softwareheritage.org/job/DICP/job/tests-on-diff/10/ for more details.

vlorentz added a subscriber: vlorentz.EditedThu, Oct 15, 2:08 PM

Could you move the changes to the calls of add_step + status_template to a new diff, and rebase this one on top of that new diff?

swh/icinga_plugins/tests/test_deposit.py
93–95

ugh, we really have to do something about these tag names that don't belong in the Atom namespace.

But out of scope for this diff

Could you move the changes to the calls of add_step + status_template to a new diff, and rebase this one on top of that new diff?

yes, ok.

In the mean time, the gist of the diff is fine though, right?

swh/icinga_plugins/tests/test_deposit.py
93–95

yes, and yes.

does that mean, we need to start defining our own namespace and use it?
(if so, that will be a breaking change).

In the mean time, the gist of the diff is fine though, right?

yeah it is

ardumont updated this revision to Diff 15101.Thu, Oct 15, 2:53 PM
  • Rebase on top of stacked diffs on test module improvment
  • Depends on D4270

Build is green

Patch application report for D4252 (id=15101)

Could not rebase; Attempt merge onto 6f9b0d99ee...

Updating 6f9b0d9..f20b1bc
Fast-forward
 requirements-swh.txt                     |   2 +-
 swh/icinga_plugins/deposit.py            |  57 ++++++-
 swh/icinga_plugins/tests/test_deposit.py | 274 ++++++++++++++++++++++++-------
 3 files changed, 270 insertions(+), 63 deletions(-)
Changes applied before test
commit f20b1bc8ce07668d081c2a735064af6bf59074b6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 13 18:23:22 2020 +0200

    Add deposit metadata update step in deposit icinga check
    
    This extends the current icinga checks on deposit to trigger a metadata update
    when the main deposit is done. It retrieves the swhid from the deposit status
    calls. Then triggers a metadata update on the deposit using the latest deposit
    client abilities (0.3).
    
    Related to T2685

commit 5d43396856cf15f7989d5ce4e2ea8d289e9c05aa
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 15 14:50:52 2020 +0200

    test_deposit: Clarify unexpected output in case of assertion failures

commit 2c326c4a0a8b2f77bc7f73332301f8352b81b2c9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 15 14:44:42 2020 +0200

    test_deposit: Introduce status_template helper function
    
    To ease incoming scenario updates (notably the deposit update metadata).

commit be18e1f9dd4ac7220d573443492d66a57b4981b9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 15 14:42:18 2020 +0200

    test_deposit: Use f-string instead of concatenate strings

See https://jenkins.softwareheritage.org/job/DICP/job/tests-on-diff/15/ for more details.

Could you move the changes to the calls of add_step + status_template to a new diff, and rebase this one on top of that new diff?

yes

done through D4268 D4269 D4270, this one rebased on top of the latter.

vlorentz accepted this revision.Thu, Oct 15, 7:57 PM
This revision is now accepted and ready to land.Thu, Oct 15, 7:57 PM