Refresh task type data from the database every time recurrent tasks are run
ClosedPublic
Actions

Authored by olasd on Oct 25 2022, 3:51 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Commits

rDSCH1f9109fa4d66: Refresh task type data from the database every time recurrent tasks are run

Summary

Avoids inconsistencies between the database state and an ongoing
recurrent task scheduler.

Diff Detail

Repository

rDSCH Scheduling utilities

Branch

detached-head

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 32571
Build 51020: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 51019: arc lint + arc unit

Event Timeline

olasd created this revision.Oct 25 2022, 3:51 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptOct 25 2022, 3:51 PM

Build is green

Patch application report for D8769 (id=31618)

Could not rebase; Attempt merge onto aeb870a700...

Updating aeb870a..5e3ecb3
Fast-forward
 swh/scheduler/celery_backend/config.py           | 4 ++--
 swh/scheduler/celery_backend/recurrent_visits.py | 6 ++++++
 2 files changed, 8 insertions(+), 2 deletions(-)

Changes applied before test

commit 5e3ecb339ed74fcbb59724dff3c7bb63217f16d3
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Oct 25 15:47:37 2022 +0200

    Refresh task type data from the database every time recurrent tasks are run
    
    Avoids inconsistencies between the database state and an ongoing
    recurrent task scheduler.

commit bde27a9e4262373486ed8c3e48bfbfe5171a9432
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Oct 25 15:46:26 2022 +0200

    Use json instead of msgpack for serializers
    
    Recent celery versions generate serialized messages with mime types incompatible
    with older versions when using msgpack

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/565/ for more details.

Harbormaster completed remote builds in B32571: Diff 31618.Oct 25 2022, 3:56 PM

olasd requested review of this revision.Oct 25 2022, 3:56 PM

youmight want to add a variable for the value of f"load-{visit_type}", it's used four times now

This revision is now accepted and ready to land.Oct 25 2022, 4:08 PM

Variabilize task_name

olasd mentioned this in D8770: Ensure origins are not visited faster than twice a day.Oct 25 2022, 4:50 PM

Build is green

Patch application report for D8769 (id=31625)

Could not rebase; Attempt merge onto aeb870a700...

Updating aeb870a..1f9109f
Fast-forward
 swh/scheduler/celery_backend/config.py           |  4 ++--
 swh/scheduler/celery_backend/recurrent_visits.py | 11 +++++++++--
 2 files changed, 11 insertions(+), 4 deletions(-)

Changes applied before test

commit 1f9109fa4d66f42fcc67bd6ba06c91d1eeffedec
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Oct 25 15:47:37 2022 +0200

    Refresh task type data from the database every time recurrent tasks are run
    
    Avoids inconsistencies between the database state and an ongoing
    recurrent task scheduler.

commit bde27a9e4262373486ed8c3e48bfbfe5171a9432
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Oct 25 15:46:26 2022 +0200

    Use json instead of msgpack for serializers
    
    Recent celery versions generate serialized messages with mime types incompatible
    with older versions when using msgpack

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/568/ for more details.

Harbormaster completed remote builds in B32578: Diff 31625.Oct 25 2022, 4:54 PM

Closed by commit rDSCH1f9109fa4d66: Refresh task type data from the database every time recurrent tasks are run (authored by olasd). · Explain WhyOct 25 2022, 4:58 PM

This revision was automatically updated to reflect the committed changes.

olasd added a commit: rDSCH1f9109fa4d66: Refresh task type data from the database every time recurrent tasks are run.