Diffusion Scheduling utilities d58776ab0b41

Introduce new scheduling policy to grab origins without last update
d58776ab0b41
Actions

Description

Introduce new scheduling policy to grab origins without last update

This is in charge of scheduling origins without last update. This also updates the
global queue position so the journal client can initialize correctly the next position
per origin and visit type.

Related to T2345

Details

Provenance

ardumont	Authored on Jul 22 2021, 12:22 PM
olasd	Committed on Jul 22 2021, 12:23 PM
olasd	Pushed on Jul 22 2021, 2:21 PM

Differential Revision

D5956: Introduce new scheduling policy to grab origins without last update

Parents

rDSCH825e8cfe7d24: grab_next_visits: make the handling of CTEs more modular

Branches

Unknown

Tags

Unknown

Tasks

T2345: Improve handling of recurrent loading tasks in scheduler

Build Status

Buildable 22713
Build 35419: test-and-build	Jenkins console · Jenkins

Event Timeline

olasd committed rDSCHd58776ab0b41: Introduce new scheduling policy to grab origins without last update (authored by ardumont).Jul 22 2021, 12:23 PM

olasd added a task: T2345: Improve handling of recurrent loading tasks in scheduler.Jul 22 2021, 2:22 PM

olasd added an edge: D5956: Introduce new scheduling policy to grab origins without last update.

Harbormaster completed building B22713: rDSCHd58776ab0b41: Introduce new scheduling policy to grab origins without last update.Jul 22 2021, 2:38 PM

swh-public-ci mentioned this in D6017: test: Refactor assert_visit_stats_ok to ignore_fields.Jul 23 2021, 9:28 AM

swh-public-ci mentioned this in D6018: Only record last_visited and last_successful in origin_visit_stats.Jul 23 2021, 12:00 PM

Changes (2)

Path

Size

swh/

scheduler/

backend.py

tests/

test_scheduler.py

rDSCHd58776ab0b41

View Options

swh/scheduler/backend.py