Useful for changes further down the line.
Details
Details
- Reviewers
olasd douardda - Group Reviewers
Reviewers - Commits
- rDSCH7ffbdd1b3eb5: Construct grab_next_visits query arguments incrementally
Diff Detail
Diff Detail
- Repository
- rDSCH Scheduling utilities
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
Build is green
Patch application report for D4910 (id=17489)
Could not rebase; Attempt merge onto 03460207a1...
Updating 0346020..9943195 Fast-forward swh/scheduler/backend.py | 67 +++++++++++++++++++------ swh/scheduler/interface.py | 30 +++++++---- swh/scheduler/model.py | 33 +------------ swh/scheduler/simulator/__init__.py | 18 +++---- swh/scheduler/simulator/origins.py | 83 +++++++++++++++++++++++++++++-- swh/scheduler/tests/test_scheduler.py | 93 ++++++++++++++++++++++++++++++----- swh/scheduler/tests/test_simulator.py | 6 +-- 7 files changed, 245 insertions(+), 85 deletions(-)
Changes applied before test
commit 9943195d31c51a44325cba09d07fb6e904d45a00
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Thu Jan 21 17:27:40 2021 +0100
Construct grab_next_visits query arguments incrementally
commit 72070b7bf628788b6872e90a3f8ac8f0c01b70d9
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Jan 21 14:57:42 2021 +0100
simulator: add simple lister simulation
commit 1f1aad459c4b0740ecbe96e9809e4b31f66bf999
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Jan 21 14:54:53 2021 +0100
Factor out ListedOrigin generation to use the OriginModel
This generates consistent last_update values according to the model and
simulated time.
commit b93aa5be2c2d5dc2130e1027698f3e1255052d8d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Jan 21 13:01:53 2021 +0100
Make PaginatedListedOriginList a concretization of PagedResult
1. consistent with swh-storage and swh-indexer-storage
2. we can use swh.core.api.classes.stream_results on scheduler.get_listed_origins.
commit 2f47936731cf438a5195978a2af3250597b693b5
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Wed Jan 20 17:29:16 2021 +0100
Add scheduling policy for already visited origins with known last update
This policy schedules origins by decreasing order of "visit lag" (that
is, origins with the most lag are scheduled first).
commit acad712ad3f71f88f99e45e9b4f571ad751945dc
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Wed Jan 20 17:25:46 2021 +0100
Add scheduling policy for never visited origins
This policy orders never visited origins by increasing date of last
update (scheduling the "oldest" never visited origins first).See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/232/ for more details.
Comment Actions
ok, but it would have been nice to have an explanation of why this is necessary in the commit message.
Comment Actions
Build is green
Patch application report for D4910 (id=17568)
Could not rebase; Attempt merge onto 2906b4e8a0...
Updating 2906b4e..a2dc724 Fast-forward swh/scheduler/backend.py | 21 ++++++--- swh/scheduler/simulator/__init__.py | 18 +++----- swh/scheduler/simulator/origins.py | 92 +++++++++++++++++++++++++++++++++++-- 3 files changed, 110 insertions(+), 21 deletions(-)
Changes applied before test
commit a2dc72474056c2f20e255acf13ec3e662e1aad7a
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Thu Jan 21 17:27:40 2021 +0100
Construct grab_next_visits query arguments incrementally
commit e5709214b4917a5fe3634d040da7a061f5978f66
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Jan 21 14:57:42 2021 +0100
simulator: add simple lister simulation
commit 7af98e2bc048c6946679e7d95cf8620e4a0ee4bf
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Jan 21 14:54:53 2021 +0100
Factor out ListedOrigin generation to use the OriginModel
This generates consistent last_update values according to the model and
simulated time.See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/272/ for more details.
Comment Actions
Build is green
Patch application report for D4910 (id=17608)
Could not rebase; Attempt merge onto 2906b4e8a0...
Updating 2906b4e..7ffbdd1 Fast-forward docs/simulator.rst | 15 ++++++ swh/scheduler/backend.py | 21 ++++++--- swh/scheduler/simulator/__init__.py | 18 +++----- swh/scheduler/simulator/origins.py | 92 +++++++++++++++++++++++++++++++++++-- 4 files changed, 125 insertions(+), 21 deletions(-)
Changes applied before test
commit 7ffbdd1b3eb579f43e8913ea11cfd916b2f3c457
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Thu Jan 21 17:27:40 2021 +0100
Construct grab_next_visits query arguments incrementally
commit ea068b46a89e07c60ad1233afd36afc6bb29031e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Jan 21 14:57:42 2021 +0100
simulator: add simple lister simulation
commit 7af98e2bc048c6946679e7d95cf8620e4a0ee4bf
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Jan 21 14:54:53 2021 +0100
Factor out ListedOrigin generation to use the OriginModel
This generates consistent last_update values according to the model and
simulated time.See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/281/ for more details.