Page MenuHomeSoftware Heritage
Differential D6514

Add docstring to runner and listener modules
ClosedPublic
Actions

Authored by ardumont on Oct 20 2021, 10:44 AM.

Details

Reviewers
anlambert
Group Reviewers
Reviewers
Maniphest Tasks
T3667: Orchestrate origins scheduling according to scheduler metrics feedback
Summary

Related to T3667

Diff Detail

Repository
rDSCH Scheduling utilities
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24558
Build 38320: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 38319: arc lint + arc unit

Event Timeline

ardumont created this revision.Oct 20 2021, 10:44 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptOct 20 2021, 10:44 AM
swh-public-ci added a comment.Oct 20 2021, 10:45 AM

Build has FAILED

Patch application report for D6514 (id=23652)

Could not rebase; Attempt merge onto 3aed6887fc...

Updating 3aed688..f14ee2c
Fast-forward
 requirements-test.txt                         |  1 +
 swh/scheduler/backend.py                      |  2 +-
 swh/scheduler/celery_backend/pika_listener.py |  7 ++++++-
 swh/scheduler/celery_backend/runner.py        | 10 ++++++++++
 4 files changed, 18 insertions(+), 2 deletions(-)
Changes applied before test
commit f14ee2c827d54b0341d0672108b1900e498dc76d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 20 10:42:34 2021 +0200

    Add docstring to runner and listener modules
    
    Related to T3667

commit 6e0a4b3088fca858b9914c41ecd0b8f6aaca853d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 20 10:41:38 2021 +0200

    backend: Add type ignore
    
    We cannot do much about that type issue since it's a dependency definition.

commit f15c51091439f499c5ab1c2cb75e3d058cb64341
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 20 09:36:27 2021 +0200

    scheduler: Deprecate unused main celery runner

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/459/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/459/console

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 20 2021, 10:45 AM
Harbormaster failed remote builds in B24542: Diff 23652!
ardumont updated this revision to Diff 23658.Oct 20 2021, 10:49 AM

Fix

swh-public-ci added a comment.Oct 20 2021, 10:52 AM

Build is green

Patch application report for D6514 (id=23658)

Could not rebase; Attempt merge onto 3aed6887fc...

Updating 3aed688..3d2d853
Fast-forward
 requirements-test.txt                         |  1 +
 swh/scheduler/celery_backend/pika_listener.py |  7 ++++++-
 swh/scheduler/celery_backend/runner.py        | 10 ++++++++++
 3 files changed, 17 insertions(+), 1 deletion(-)
Changes applied before test
commit 3d2d853cbce42fb632a95661245b1eb6cbdf515d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 20 10:42:34 2021 +0200

    Add docstring to runner and listener modules
    
    Related to T3667

commit f15c51091439f499c5ab1c2cb75e3d058cb64341
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 20 09:36:27 2021 +0200

    scheduler: Deprecate unused main celery runner

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/461/ for more details.

Harbormaster completed remote builds in B24548: Diff 23658.Oct 20 2021, 10:52 AM
ardumont requested review of this revision.Oct 20 2021, 10:52 AM
ardumont updated this revision to Diff 23670.Oct 20 2021, 11:10 AM

Improve sentence phrasing

swh-public-ci added a comment.Oct 20 2021, 11:13 AM

Build is green

Patch application report for D6514 (id=23670)

Could not rebase; Attempt merge onto f15c510914...

Updating f15c510..61655ab
Fast-forward
 swh/scheduler/celery_backend/listener.py      | 220 --------------------------
 swh/scheduler/celery_backend/pika_listener.py |  12 +-
 swh/scheduler/celery_backend/runner.py        |   9 ++
 3 files changed, 20 insertions(+), 221 deletions(-)
 delete mode 100644 swh/scheduler/celery_backend/listener.py
Changes applied before test
commit 61655ab2c5b40b5f17f286c191d3e8506e3213bd
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 20 10:42:34 2021 +0200

    Add docstring to runner and listener modules
    
    Related to T3667

commit 5b531964a45843eb4310c1fadd60958fba0b6041
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 20 10:44:34 2021 +0200

    Drop deprecated listener module
    
    It's been deprecated for enough time.
    
    Related to T3667

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/464/ for more details.

Harbormaster completed remote builds in B24558: Diff 23670.Oct 20 2021, 11:13 AM
anlambert accepted this revision.Oct 20 2021, 11:19 AM
anlambert added a subscriber: anlambert.

Looks goo to me.

swh/scheduler/celery_backend/pika_listener.py
9

It's the final step executed after a task is done. => to avoid the done repetition

This revision is now accepted and ready to land.Oct 20 2021, 11:19 AM
ardumont updated this revision to Diff 23672.Oct 20 2021, 11:25 AM

Adapt according to remark

Harbormaster completed remote builds in B24560: Diff 23672.Oct 20 2021, 11:25 AM
ardumont updated this revision to Diff 23673.Oct 20 2021, 11:26 AM

Use the correct repo to diff

ardumont marked an inline comment as done.Oct 20 2021, 11:26 AM
ardumont added inline comments.
swh/scheduler/celery_backend/pika_listener.py
9

lol, indeed, thx ;)

swh-public-ci added a comment.Oct 20 2021, 11:29 AM

Build is green

Patch application report for D6514 (id=23673)

Rebasing onto 5b531964a4...

Current branch diff-target is up to date.
Changes applied before test
commit 243a69fc357ade6d3d8e68cd5d45c6e47cb40d6d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 20 10:42:34 2021 +0200

    Add docstring to runner and listener modules
    
    Related to T3667

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/465/ for more details.

Harbormaster completed remote builds in B24561: Diff 23673.Oct 20 2021, 11:29 AM
ardumont updated this revision to Diff 23677.Oct 20 2021, 11:35 AM
ardumont marked an inline comment as done.

Use the correct keyname for swh-sysadm sphinx instance

Harbormaster completed remote builds in B24566: Diff 23677.Oct 20 2021, 11:35 AM
ardumont updated this revision to Diff 23678.Oct 20 2021, 11:39 AM

Use correct repo to diff

ardumont closed this revision.Oct 20 2021, 11:40 AM

Landed with 243a69f

swh-public-ci added a comment.Oct 20 2021, 11:43 AM

Build is green

Patch application report for D6514 (id=23678)

Rebasing onto 243a69fc35...

First, rewinding head to replay your work on top of it...
Fast-forwarded diff-target to base-revision-466-D6514.
Changes applied before test



See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/466/ for more details.
Harbormaster completed remote builds in B24568: Diff 23678.Oct 20 2021, 11:43 AM
Revision Contents
Changeset List
PathSizeCoverage (All)Coverage (Touched)
swh/
scheduler/
celery_backend/
12 lines21%
Loading...
9 lines85%
Loading...
Diff 23670
View Options
swh/scheduler/celery_backend/pika_listener.py
Loading...
View Options
swh/scheduler/celery_backend/runner.py
Loading...
About · Developer information · Legal