Diffusion Scheduling utilities 081148ac15b4

Merge tag 'debian/0.9.0-1_swh2' into debian/buster-swh
081148ac15b4
Actions

Tags

None

Subscribers

None

Description

Merge tag 'debian/0.9.0-1_swh2' into debian/buster-swh

Details

Provenance

Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>	Authored on Jan 21 2021, 2:30 PM
swh-public-ci	Pushed on Jan 21 2021, 2:30 PM

Parents

rDSCH58ec03f0b4af: d/changelog: Bump new release
rDSCHdd4b6988a4e3: Updated backport on buster-swh from debian/0.8.2-1_swh2 (unstable-swh)

Branches

Unknown

Tags

Unknown

Event Timeline

Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSCH081148ac15b4: Merge tag 'debian/0.9.0-1_swh2' into debian/buster-swh (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).Jan 21 2021, 2:30 PM

Merged Changes

This commit merges a very large number of changes. Only the first 50 are shown.

58ec03f0b4af d/changelog: Bump new release
Author
ardumont
Browse Repository
No Builds
No Audits
b0e941de9589 d/control: Update dependencies
Author
ardumont
Browse Repository
No Builds
No Audits
d2c472549adb Updated debian changelog for version 0.9.0
Author
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>
Browse Repository
No Builds
No Audits
5a9c2e7596db Update upstream source from tag 'debian/upstream/0.9.0'
Author
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>
Browse Repository
No Builds
No Audits
2fd414f696e9 New upstream version 0.9.0
Author
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>
Browse Repository
No Builds
No Audits
b641ac83ebbf Make the grab_next_visits sql query modular
Author
olasd
Revisions
D4896: Make the grab_next_visits sql query modular
Browse Repository
Passed
No Audits
9fb0dd6c7cc3 journal_client: Read visit_stats entries by batch out of the loop
Author
ardumont
Revisions
D4890: journal_client: Read visit_stats entries by batch out of the loop
Browse Repository
Passed
No Audits
d464b4cc1f9a scheduler: Make origin_visit_stats_get read multiple entries
Author
ardumont
Revisions
D4888: scheduler: Make origin_visit_stats_get read multiple entries
Browse Repository
Failed
No Audits
ffe2aed2fa32 Simplify journal client tests
Author
douardda
Revisions
D4894: Simplify journal client tests
Browse Repository
Passed
No Audits
c7b740cafa64 Revert "Make sure swh.scheduler.cli.journal is loaded in test_cli_journal.py"
Author
douardda
Browse Repository
Passed
No Audits
898820fac52c simulator: collect and plot scheduler metrics over time
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
9ce68f8d0e0e simulator: stop using get_scheduler directly
Author
vlorentz
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
88e0b4280501 simulator: Add documentation.
Author
vlorentz
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
62c6d90867bc simulator: Make min_batch_size a parameter defined in the setup.
Author
vlorentz
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
9468bb9384f1 simulator: add basic tests for fill_test_data and run
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
ead7b347db9d simulator: implement a simulator for the "old" task-based scheduler
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
aecd27eee06a Move the simulator cli to the main cli module
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
05067e3ecc88 simulator: Replace attrs with dataclasses for consistency
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
24922fe2d995 simulator: wrap tasks and task events in typechecked objects
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
d5318aea0a93 simulator: also fill data for the task-based scheduler
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
22ebb7a9a4bc simulator: Split into smaller files in the same package
Author
vlorentz
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
ad7bfbe731da simulator: Make the run time a CLI argument
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
df34db0bfc61 simulator: tweak simulation environment constants
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
21ce2c88dddc simulator: generate more origins in fill_data
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
29204199774b simulator: add typing for Environment.scheduler
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
6433266106dd simulator: add support for a basic SimulationReport
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
c474a825336a simulator: refine origin model to follow an exponential distribution
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
2459badf0c05 simulator: Remove some debug statements and lower log level
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
cb12449e8f57 simulator: simulate the scheduler journal client
Author
vlorentz
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
20b7f9c68f83 simulator: generate OriginVisitStatus objects in modeled visits
Author
vlorentz
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
39ad47de2e75 simulator: Move scheduler into the simulation environment object
Author
vlorentz
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
31967fa850c3 simulator: Use datetimes instead of a floating point simulated time
Author
vlorentz
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
fc3f06bd1d77 Introduce scaffolding for a scheduler simulator
Author
olasd
Revisions
D4856: Introduce scaffolding for a scheduler simulator
Browse Repository
Passed
No Audits
7905a6bea4ba Add a cli for the scheduler metrics update endpoint
Author
vlorentz
Revisions
D4889: Add a cli for the scheduler metrics update endpoint
Browse Repository
Passed
No Audits
c386fdf3b9fc Make the max_date() helper function accept *dates as argument
Author
douardda
Revisions
D4893: Make the max_date() helper function accept *dates as argument
Browse Repository
Passed
No Audits
b03d978241a6 Make sure swh.scheduler.cli.journal is loaded in test_cli_journal.py
Author
douardda
Browse Repository
Passed
No Audits
737d12e5b9e6 Introduce a new lister_get endpoint
Author
olasd
Revisions
D4887: Introduce a new lister_get endpoint
Browse Repository
Passed
No Audits
114ed952e513 Implement some basic aggregated metrics on listed origins
Author
olasd
Revisions
D4880: Implement some basic aggregated metrics on listed origins
Browse Repository
Passed
No Audits
98526539a8e4 Import the journal subcommand in the main swh.scheduler cli
Author
olasd
Revisions
D4886: Import the journal subcommand in the main swh.scheduler cli
Browse Repository
Passed
No Audits
f8627a96fed6 Move the `last_scheduled` ts from ListedOrigin to OriginVisitStatus
Author
douardda
Revisions
D4881: Move the `last_scheduled` ts from ListedOrigin to OriginVisitStatus
Browse Repository
Passed
No Audits
0a32a31195f1 Make the journal-client cli subcommand automagically loaded
Author
douardda
Revisions
D4885: Make the journal-client cli subcommand automagically loaded
Browse Repository
Passed
No Audits
5e609d520592 requirements: Make swh.journal and optional dependency
Author
ardumont
Revisions
D4879: requirements: Make swh.journal and optional dependency
Browse Repository
Passed
No Audits
9395aa0763b7 scheduler.cli.journal: Add `swh scheduler journal-client` cli
Author
ardumont
Revisions
D4876: scheduler.cli.journal: Add `swh scheduler journal visit-stats` cli
Browse Repository
Passed
No Audits
58ca79622db0 journal_client: Improve stats detection
Author
ardumont
Revisions
D4873: journal_client: Improve stats detection
Browse Repository
Passed
No Audits
d3afd144af1d Use the recorded task end time for the task scheduler feedback loop
Author
olasd
Revisions
D4874: Use the recorded task end time for the task scheduler feedback loop
Browse Repository
Passed
No Audits
a5fb291703d2 backend: Make origin_visit_stats_upsert a batch api
Author
ardumont
Revisions
D4868: backend: Make origin_visit_stats_upsert a batch api
Browse Repository
Passed
No Audits
608aa2075619 Populate origin_visit_stats table out of the origin_visit_status topic
Author
ardumont
Revisions
D4866: Populate origin_visit_stats table out of the origin_visit_status topic
Browse Repository
Passed
No Audits
ca45d40f2a62 Filter origins by visit type when scheduling the next visits
Author
olasd
Revisions
D4855: Filter origins by visit type when scheduling the next visits
Browse Repository
Passed
No Audits
59b4cb3f1c7a Reorganize ListedOrigin fixtures to generate multiple visit_types
Author
olasd
Revisions
D4854: Reorganize ListedOrigin fixtures to generate multiple visit_types
Browse Repository
Passed
No Audits
4f5338f2aba3 Introduce a `swh scheduler origin schedule-next` cli
Author
olasd
Revisions
D4846: Introduce a `swh scheduler origin schedule-next` cli
Browse Repository
Passed
No Audits

Changes (49)

Path

Size

.pre-commit-config.yaml

debian/

docs/

requirements-journal.txt

requirements-simulator.txt

sql/

updates/

swh.scheduler.egg-info/

entry_points.txt

swh/

scheduler/

cli/

journal_client.py

origin_scheduler.py

task_scheduler.py

sql/

tests/

test_api_client.py

test_cli_journal.py

test_cli_origin.py

test_journal_client.py

test_scheduler.py

test_simulator.py

rDSCH081148ac15b4

.pre-commit-config.yaml

Loading...

PKG-INFO

Loading...

debian/changelog

Loading...

debian/control

Loading...

debian/rules

Loading...

docs/cli.rst

Loading...

docs/index.rst

Loading...

docs/simulator.rst

Loading...

mypy.ini

Loading...

requirements-journal.txt

Loading...

requirements-simulator.txt

Loading...

setup.py

Loading...

sql/updates/18.sql

Loading...

sql/updates/19.sql

Loading...

sql/updates/20.sql

Loading...

sql/updates/23.sql

Loading...

sql/updates/24.sql

Loading...

sql/updates/25.sql

Loading...

swh.scheduler.egg-info/PKG-INFO

Loading...

swh.scheduler.egg-info/SOURCES.txt

Loading...

swh.scheduler.egg-info/entry_points.txt

Loading...

swh.scheduler.egg-info/requires.txt

Loading...

swh/scheduler/backend.py

Loading...

swh/scheduler/cli/init.py

Loading...

swh/scheduler/cli/journal.py

Loading...

swh/scheduler/cli/origin.py

Loading...

swh/scheduler/cli/simulator.py

Loading...

swh/scheduler/exc.py

Loading...

swh/scheduler/interface.py

Loading...

swh/scheduler/journal_client.py

Loading...

swh/scheduler/model.py

Loading...

swh/scheduler/simulator/init.py

Loading...

swh/scheduler/simulator/common.py

Loading...

swh/scheduler/simulator/origin_scheduler.py

Loading...

swh/scheduler/simulator/origins.py

Loading...

swh/scheduler/simulator/task_scheduler.py

Loading...

swh/scheduler/sql/30-schema.sql

Loading...

swh/scheduler/sql/40-func.sql

Loading...

swh/scheduler/sql/60-indexes.sql

Loading...

swh/scheduler/tests/common.py

Loading...

swh/scheduler/tests/conftest.py

Loading...

swh/scheduler/tests/test_api_client.py

Loading...

swh/scheduler/tests/test_cli_journal.py

Loading...

swh/scheduler/tests/test_cli_origin.py

Loading...

swh/scheduler/tests/test_journal_client.py

Loading...

swh/scheduler/tests/test_model.py

Loading...

swh/scheduler/tests/test_scheduler.py

Loading...

swh/scheduler/tests/test_simulator.py

Loading...