Page MenuHomeSoftware Heritage
Feed All Stories

Today

ardumont added a comment to T2993: Deploy visit-stats journal client on production.

Everything looks fine and visit-stats is growing gently:

Tue, Jan 26, 7:08 PM · System administration, Scheduling utilities
ardumont closed T2987: Port gitlab lister to the new `swh.lister.pattern.Lister` API, a subtask of T2442: Provide a unified API for listers to interact with the scheduler, as Resolved.
Tue, Jan 26, 7:06 PM · Sprint 2021 01, Scheduling utilities
ardumont closed T2987: Port gitlab lister to the new `swh.lister.pattern.Lister` API as Resolved.
Tue, Jan 26, 7:06 PM · Lister
swh-public-ci added a comment to D4953: cgit: Add support for last_update information during listing.

Build is green

Tue, Jan 26, 6:57 PM
swh-public-ci added a comment to D4953: cgit: Add support for last_update information during listing.

Build is green

Tue, Jan 26, 6:53 PM
vsellier updated the diff for D4953: cgit: Add support for last_update information during listing.

Inline unecessary indirection

Tue, Jan 26, 6:52 PM
swh-public-ci added a comment to D4731: WIP Configuration system.

Build is green

Tue, Jan 26, 6:51 PM · feedback-required, Core & foundations
vsellier updated the diff for D4953: cgit: Add support for last_update information during listing.

Add missing test coverage

Tue, Jan 26, 6:49 PM
tenma updated the diff for D4731: WIP Configuration system.

Rename "singleton" to "record"

Tue, Jan 26, 6:49 PM · feedback-required, Core & foundations
vsellier requested review of D4953: cgit: Add support for last_update information during listing.
Tue, Jan 26, 6:36 PM
vsellier added a revision to T2988: Improve cgit lister to add last modification date of the repos: D4953: cgit: Add support for last_update information during listing.
Tue, Jan 26, 6:33 PM · CGit lister, Lister
vsellier changed the status of T2988: Improve cgit lister to add last modification date of the repos from Open to Work in Progress.
Tue, Jan 26, 6:04 PM · CGit lister, Lister
ardumont requested changes to D4951: Remove tests' journal_data.py in favor of the version in swh-model.

for the sake of my questions

Tue, Jan 26, 5:42 PM
anlambert closed D4952: tests/resources: Fix mypy 0.800 errors.
Tue, Jan 26, 5:37 PM
anlambert committed rDWAPPSab5401d73db1: tests/resources: Fix mypy 0.800 errors (authored by anlambert).
tests/resources: Fix mypy 0.800 errors
Tue, Jan 26, 5:37 PM
tenma closed D4949: tests: remove all unused conftests.
Tue, Jan 26, 5:36 PM
ardumont accepted D4952: tests/resources: Fix mypy 0.800 errors.

(i don't know why or if it's sensible that the decorator got dropped though ;)

Tue, Jan 26, 5:35 PM
douardda requested review of D4951: Remove tests' journal_data.py in favor of the version in swh-model.
Tue, Jan 26, 5:32 PM
swh-public-ci added a comment to D4949: tests: remove all unused conftests.

Build is green

Tue, Jan 26, 5:31 PM
anlambert closed D4925: debian: Reimplement lister using new Lister API.
Tue, Jan 26, 5:30 PM
anlambert closed T2979: Port debian lister to the new Lister API, a subtask of T2442: Provide a unified API for listers to interact with the scheduler, as Resolved.
Tue, Jan 26, 5:29 PM · Sprint 2021 01, Scheduling utilities
anlambert closed T2979: Port debian lister to the new Lister API as Resolved by committing rDLSbb0184c004d2: debian: Reimplement lister using new Lister API.
Tue, Jan 26, 5:29 PM · Lister, Sprint 2021 01
anlambert committed rDLSbb0184c004d2: debian: Reimplement lister using new Lister API (authored by anlambert).
debian: Reimplement lister using new Lister API
Tue, Jan 26, 5:29 PM
tenma updated the diff for D4949: tests: remove all unused conftests.

Rebase

Tue, Jan 26, 5:27 PM
swh-public-ci added a comment to D4925: debian: Reimplement lister using new Lister API.

Build is green

Tue, Jan 26, 5:27 PM
tenma reopened D4949: tests: remove all unused conftests.
Tue, Jan 26, 5:27 PM
anlambert updated the diff for D4925: debian: Reimplement lister using new Lister API.

Rebase

Tue, Jan 26, 5:23 PM
anlambert requested review of D4952: tests/resources: Fix mypy 0.800 errors.
Tue, Jan 26, 5:23 PM
douardda closed D4950: Add swh-journal's model-related test data set in swh-model.
Tue, Jan 26, 5:21 PM
douardda committed rDMODcad940dc8c07: Add swh-journal's model-related test data set in swh-model (authored by douardda).
Add swh-journal's model-related test data set in swh-model
Tue, Jan 26, 5:21 PM
swh-public-ci added a comment to D4925: debian: Reimplement lister using new Lister API.

Build is green

Tue, Jan 26, 5:12 PM
tenma closed D4949: tests: remove all unused conftests.
Tue, Jan 26, 5:10 PM
tenma committed rDLS6cd31769c102: tests: Remove no longer used conftest files (authored by tenma).
tests: Remove no longer used conftest files
Tue, Jan 26, 5:09 PM
douardda added a revision to T2970: Make swh-journal tests not depend on swh-model any more: D4951: Remove tests' journal_data.py in favor of the version in swh-model.
Tue, Jan 26, 5:09 PM · Journal
anlambert updated the diff for D4925: debian: Reimplement lister using new Lister API.

Rebase and remove conftest.py file

Tue, Jan 26, 5:08 PM
ardumont accepted D4950: Add swh-journal's model-related test data set in swh-model.

sgtm

Tue, Jan 26, 5:05 PM
swh-public-ci added a comment to D4949: tests: remove all unused conftests.

Build is green

Tue, Jan 26, 5:01 PM
anlambert accepted D4949: tests: remove all unused conftests.

Thanks !

Tue, Jan 26, 5:01 PM
ardumont closed D4948: gitlab: Implement keyset-based pagination listing.
Tue, Jan 26, 5:00 PM
ardumont committed rDLS97254a19f226: gitlab: Implement keyset-based pagination listing (authored by ardumont).
gitlab: Implement keyset-based pagination listing
Tue, Jan 26, 5:00 PM
tenma updated the diff for D4949: tests: remove all unused conftests.

Reword commit

Tue, Jan 26, 4:58 PM
swh-public-ci added a comment to D4948: gitlab: Implement keyset-based pagination listing.

Build is green

Tue, Jan 26, 4:57 PM
ardumont updated the diff for D4948: gitlab: Implement keyset-based pagination listing.

Rebase

Tue, Jan 26, 4:54 PM
anlambert requested changes to D4949: tests: remove all unused conftests.

Looks good but the commit message could be more detailed and rephrased to something like:

Tue, Jan 26, 4:51 PM
douardda requested review of D4950: Add swh-journal's model-related test data set in swh-model.
Tue, Jan 26, 4:47 PM
tenma requested review of D4949: tests: remove all unused conftests.
Tue, Jan 26, 4:46 PM
douardda added a revision to T2970: Make swh-journal tests not depend on swh-model any more: D4950: Add swh-journal's model-related test data set in swh-model.
Tue, Jan 26, 4:45 PM · Journal
douardda added a comment to T2970: Make swh-journal tests not depend on swh-model any more.

Back on this, the plan is now to make swh-journal not depend on the actual model definition, which is currently mostly due to the presence of the journal_data.py in swh-journal. So the plan is to move this file in swh-model so it's kept up to date with swh-model, even if it's mostly used for testing other packages (like swh-journal).

Tue, Jan 26, 4:41 PM · Journal
swh-public-ci added a comment to D4948: gitlab: Implement keyset-based pagination listing.

Build is green

Tue, Jan 26, 4:38 PM
ardumont updated the diff for D4948: gitlab: Implement keyset-based pagination listing.

Adapt according to review to improve readability

Tue, Jan 26, 4:34 PM
ardumont added inline comments to D4948: gitlab: Implement keyset-based pagination listing.
Tue, Jan 26, 4:25 PM
anlambert accepted D4948: gitlab: Implement keyset-based pagination listing.

Looks good to me ! I added a comment regarding further improvements that could be added to the lister but not related to that diff.

Tue, Jan 26, 4:19 PM
Harbormaster failed remote builds in B18766: Diff 17628 for D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error!
Tue, Jan 26, 3:48 PM
swh-public-ci added a comment to D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error.

Build has FAILED

Tue, Jan 26, 3:48 PM
ardumont requested review of D4948: gitlab: Implement keyset-based pagination listing.
Tue, Jan 26, 3:46 PM
DanSeraf updated the diff for D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error.

commit message

Tue, Jan 26, 3:45 PM
ardumont added a revision to T2994: Use keyset pagination in Gitlab lister: D4948: gitlab: Implement keyset-based pagination listing.
Tue, Jan 26, 3:43 PM · Origin-GitLab, Lister
swh-public-ci added a comment to D4925: debian: Reimplement lister using new Lister API.

Build is green

Tue, Jan 26, 3:29 PM
anlambert updated the diff for D4925: debian: Reimplement lister using new Lister API.

Rebase

Tue, Jan 26, 3:25 PM
anlambert closed T2989: Port CRAN lister to the new Lister API as Resolved by committing rDLS22eeb0956ec1: cran: Retrieve last update date for each listed package.
Tue, Jan 26, 3:24 PM · Lister
anlambert closed T2989: Port CRAN lister to the new Lister API, a subtask of T2442: Provide a unified API for listers to interact with the scheduler, as Resolved.
Tue, Jan 26, 3:24 PM · Sprint 2021 01, Scheduling utilities
anlambert committed rDLS22eeb0956ec1: cran: Retrieve last update date for each listed package (authored by anlambert).
cran: Retrieve last update date for each listed package
Tue, Jan 26, 3:24 PM
anlambert closed D4945: cran: Reimplement lister using new Lister API.
Tue, Jan 26, 3:24 PM
anlambert committed rDLS6f40ab4c5730: cran: Reimplement lister using new Lister API (authored by anlambert).
cran: Reimplement lister using new Lister API
Tue, Jan 26, 3:24 PM
anlambert closed T2911: Redirect https://www.softwareheritage.org/schema/2018/deposit to https://docs.softwareheritage.org/devel/swh-deposit/specs/protocol-reference.html, a subtask of T2625: create and publish xmlns schema for the specific swh-deposit metadata, as Resolved.
Tue, Jan 26, 3:22 PM · Restricted Project, SWORD deposit, Scientific Community Building
anlambert closed T2911: Redirect https://www.softwareheritage.org/schema/2018/deposit to https://docs.softwareheritage.org/devel/swh-deposit/specs/protocol-reference.html as Resolved.

Following URLs are now redirected to the desired page:

Tue, Jan 26, 3:22 PM · Website
swh-public-ci added a comment to D4945: cran: Reimplement lister using new Lister API.

Build is green

Tue, Jan 26, 3:18 PM
vlorentz added a comment to D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error.

oh, that explains the test failures, thanks

Tue, Jan 26, 3:18 PM
anlambert updated the diff for D4945: cran: Reimplement lister using new Lister API.

Rebase

Tue, Jan 26, 3:15 PM
DanSeraf retitled D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error from scanner-benchmark: algo_miv fixed, retry mechanism on request error to scanner-benchmark: algo_min fixed, retry mechanism on request error.
Tue, Jan 26, 3:15 PM
DanSeraf retitled D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error from scanner-benchmark: algo_mix fixed, retry mechanism on request error to scanner-benchmark: algo_miv fixed, retry mechanism on request error.
Tue, Jan 26, 3:15 PM
ardumont claimed T2994: Use keyset pagination in Gitlab lister.
Tue, Jan 26, 3:14 PM · Origin-GitLab, Lister
swh-public-ci added a comment to D4927: lister.docs: add a lister template for the new API.

Build is green

Tue, Jan 26, 3:12 PM · Sprint 2021 01, Lister
tenma accepted D4925: debian: Reimplement lister using new Lister API.

LGTM

Tue, Jan 26, 3:11 PM
tenma updated the diff for D4927: lister.docs: add a lister template for the new API.

Add implementation instructions to the template

Tue, Jan 26, 3:08 PM · Sprint 2021 01, Lister
ardumont closed D4944: gitlab: Add support for last_update information during listing.
Tue, Jan 26, 3:02 PM
ardumont committed rDLSaefb260f76bf: gitlab: Add support for last_update information during listing (authored by ardumont).
gitlab: Add support for last_update information during listing
Tue, Jan 26, 3:02 PM
ardumont closed D4940: gitlab: Support authentication.
Tue, Jan 26, 3:02 PM
ardumont committed rDLS1a19b2c74752: gitlab: Support authentication (authored by ardumont).
gitlab: Support authentication
Tue, Jan 26, 3:02 PM
ardumont added a comment to T2993: Deploy visit-stats journal client on production.
  • getty: Update swh dependencies (storage upgrade for the backfiller)
Tue, Jan 26, 3:01 PM · System administration, Scheduling utilities
zack added a comment to D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error.

for context: this change is only going to be committed in a separate branch that we're using for experiments with swh-scanner, hence the testing requirements are different/lower than for master

Tue, Jan 26, 3:00 PM
vlorentz added a comment to D4947: scanner-benchmark: algo_min fixed, retry mechanism on request error.

Nitpick: could you reword the commit to follow this guideline?

Tue, Jan 26, 2:51 PM
vlorentz added a project to T2937: Provide credentials to InvenioRDM for staging: System administration.
Tue, Jan 26, 2:45 PM · System administration, Restricted Project, SWORD deposit, Scientific Community Building
vlorentz renamed T2937: Provide credentials to InvenioRDM for staging from Provide credentials to InvenioRDM for stagging to Provide credentials to InvenioRDM for staging.
Tue, Jan 26, 2:45 PM · System administration, Restricted Project, SWORD deposit, Scientific Community Building
swh-public-ci added a comment to D4945: cran: Reimplement lister using new Lister API.

Build is green

Tue, Jan 26, 2:44 PM
anlambert added inline comments to D4945: cran: Reimplement lister using new Lister API.
Tue, Jan 26, 2:42 PM
anlambert updated the diff for D4945: cran: Reimplement lister using new Lister API.

Update: Add test for parse_packaged_date and fix implementation.

Tue, Jan 26, 2:41 PM
ardumont edited P927 backfill script.
Tue, Jan 26, 2:31 PM
ardumont updated the task description for T2993: Deploy visit-stats journal client on production.
Tue, Jan 26, 2:30 PM · System administration, Scheduling utilities
ardumont added a comment to T2993: Deploy visit-stats journal client on production.
  • Create consumer group swh.scheduler.journal_client so it starts at the end of the topics
Tue, Jan 26, 2:30 PM · System administration, Scheduling utilities
swh-public-ci added a comment to D4925: debian: Reimplement lister using new Lister API.

Build is green

Tue, Jan 26, 2:26 PM
anlambert updated the diff for D4925: debian: Reimplement lister using new Lister API.

Update: Address some @tenma comments

Tue, Jan 26, 2:23 PM
anlambert accepted D4940: gitlab: Support authentication.

Looks good !

Tue, Jan 26, 2:14 PM
swh-public-ci added a comment to D4944: gitlab: Add support for last_update information during listing.

Build is green

Tue, Jan 26, 2:12 PM
swh-public-ci added a comment to D4940: gitlab: Support authentication.

Build is green

Tue, Jan 26, 2:08 PM
ardumont updated the diff for D4944: gitlab: Add support for last_update information during listing.

address suggestion, ensure last_update field in listed_origin are not None

Tue, Jan 26, 2:06 PM
ardumont updated the diff for D4940: gitlab: Support authentication.

Use supported authentication scheme (bearer token in Authorization request header)

Tue, Jan 26, 2:05 PM
swh-public-ci added a comment to D4928: [wip] add materialized view origins_to_schedule and use it in grab_next_visits..

Build is green

Tue, Jan 26, 1:46 PM
swh-public-ci added a comment to D4917: simulator: stop validating the scheduling policy in the CLI.

Build is green

Tue, Jan 26, 1:44 PM
swh-public-ci added a comment to D4916: Run simulator tests on all known scheduling policies.

Build is green

Tue, Jan 26, 1:41 PM