Page MenuHomeSoftware Heritage

Refresh task type data from the database every time recurrent tasks are run
ClosedPublic

Authored by olasd on Oct 25 2022, 3:51 PM.

Details

Summary

Avoids inconsistencies between the database state and an ongoing
recurrent task scheduler.

Diff Detail

Repository
rDSCH Scheduling utilities
Branch
detached-head
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 32578
Build 51033: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 51032: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8769 (id=31618)

Could not rebase; Attempt merge onto aeb870a700...

Updating aeb870a..5e3ecb3
Fast-forward
 swh/scheduler/celery_backend/config.py           | 4 ++--
 swh/scheduler/celery_backend/recurrent_visits.py | 6 ++++++
 2 files changed, 8 insertions(+), 2 deletions(-)
Changes applied before test
commit 5e3ecb339ed74fcbb59724dff3c7bb63217f16d3
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Oct 25 15:47:37 2022 +0200

    Refresh task type data from the database every time recurrent tasks are run
    
    Avoids inconsistencies between the database state and an ongoing
    recurrent task scheduler.

commit bde27a9e4262373486ed8c3e48bfbfe5171a9432
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Oct 25 15:46:26 2022 +0200

    Use json instead of msgpack for serializers
    
    Recent celery versions generate serialized messages with mime types incompatible
    with older versions when using msgpack

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/565/ for more details.

olasd requested review of this revision.Oct 25 2022, 3:56 PM
vlorentz added a subscriber: vlorentz.

youmight want to add a variable for the value of f"load-{visit_type}", it's used four times now

This revision is now accepted and ready to land.Oct 25 2022, 4:08 PM

Variabilize task_name

Build is green

Patch application report for D8769 (id=31625)

Could not rebase; Attempt merge onto aeb870a700...

Updating aeb870a..1f9109f
Fast-forward
 swh/scheduler/celery_backend/config.py           |  4 ++--
 swh/scheduler/celery_backend/recurrent_visits.py | 11 +++++++++--
 2 files changed, 11 insertions(+), 4 deletions(-)
Changes applied before test
commit 1f9109fa4d66f42fcc67bd6ba06c91d1eeffedec
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Oct 25 15:47:37 2022 +0200

    Refresh task type data from the database every time recurrent tasks are run
    
    Avoids inconsistencies between the database state and an ongoing
    recurrent task scheduler.

commit bde27a9e4262373486ed8c3e48bfbfe5171a9432
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Oct 25 15:46:26 2022 +0200

    Use json instead of msgpack for serializers
    
    Recent celery versions generate serialized messages with mime types incompatible
    with older versions when using msgpack

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/568/ for more details.