Page MenuHomeSoftware Heritage

ardumont (Antoine R. Dumont)
UserAdministrator

Projects (10)

User Details

User Since
Sep 7 2015, 3:42 PM (280 w, 5 d)
Roles
Administrator

Recent Activity

Today

ardumont updated the diff for D4930: gitlab: Add tests on pagination.

Refactor existing test to reuse the same mechanism as the new one

Sat, Jan 23, 4:13 PM
ardumont requested review of D4930: gitlab: Add tests on pagination.
Sat, Jan 23, 4:06 PM
ardumont requested review of D4929: gitlab: Port to the new lister api.
Sat, Jan 23, 4:03 PM

Yesterday

ardumont committed rPPL4f32cc99e74f: Initial packaging backport for buster-swh (authored by ardumont).
Initial packaging backport for buster-swh
Fri, Jan 22, 6:48 PM
ardumont accepted D4924: model/ListedOrigin: Set extra_loader_arguments type to Dict[str, Any].
Fri, Jan 22, 5:15 PM
ardumont moved T2972: Port npm lister to the new Lister API from code review to done on the Sprint 2021 01 board.
Fri, Jan 22, 5:13 PM · Npm Lister, Lister, Sprint 2021 01
ardumont moved T2984: Port cgit lister to the new Lister API from in-progress to code review on the Sprint 2021 01 board.
Fri, Jan 22, 3:53 PM · Lister, CGit lister, Sprint 2021 01
ardumont committed rPPL74370a8d40b2: d/control: Drop unnecessary dependencies (authored by ardumont).
d/control: Drop unnecessary dependencies
Fri, Jan 22, 9:34 AM
ardumont committed rPPL209db950199a: d/copyright: Add file (authored by ardumont).
d/copyright: Add file
Fri, Jan 22, 9:34 AM
ardumont committed rCJSWH93f33c908f6e: jobs/dependency-packages: Add python3-plotille deps package (authored by ardumont).
jobs/dependency-packages: Add python3-plotille deps package
Fri, Jan 22, 9:34 AM

Thu, Jan 21

ardumont accepted D4898: Add scheduling policy for never visited origins.
Thu, Jan 21, 7:30 PM
ardumont committed rPPL99d125c15fbc: pristine-tar data for plotille_3.7.2.orig.tar.gz (authored by ardumont).
pristine-tar data for plotille_3.7.2.orig.tar.gz
Thu, Jan 21, 7:21 PM
ardumont committed rPPL907a1307efa1: Initial packaging for plotille (authored by ardumont).
Initial packaging for plotille
Thu, Jan 21, 7:21 PM
ardumont committed rPPLd132068dfe4c: New upstream version 3.7.2 (authored by ardumont).
New upstream version 3.7.2
Thu, Jan 21, 7:21 PM
ardumont accepted D4917: simulator: stop validating the scheduling policy in the CLI.
Thu, Jan 21, 7:17 PM
ardumont accepted D4918: Solve uneventful/eventful with unordered messages with snapshots.
Thu, Jan 21, 7:11 PM
ardumont accepted D4908: Do not consider duplicated messages as uneventful event.
Thu, Jan 21, 7:06 PM
ardumont accepted D4905: Make PaginatedListedOriginList a concretization of PagedResult.
Thu, Jan 21, 4:50 PM
ardumont accepted D4906: tests: small fixes to pypi and gitlab.
Thu, Jan 21, 4:48 PM
ardumont committed rDSCH58ec03f0b4af: d/changelog: Bump new release (authored by ardumont).
d/changelog: Bump new release
Thu, Jan 21, 2:28 PM
ardumont committed rDSCHb0e941de9589: d/control: Update dependencies (authored by ardumont).
d/control: Update dependencies
Thu, Jan 21, 2:28 PM
ardumont accepted D4902: Run Black..

As for the reason:

12:07 <+ardumont> why it was not formatted, might be because of the rebase
12:08 <+ardumont> the `git commit` now feels slugish since we started using pre-commit
12:08 <+ardumont> but the rebase in themselves are not
12:08 <+ardumont> so i don't think pre-commit is run during those
12:10 <+ardumont> and i did the last rebase actually focusing on the stability not the formatting so there you go, that must be how it passed through
12:10 <+olasd> git hooks don't fire on rebase
Thu, Jan 21, 12:11 PM
ardumont accepted D4897: Reorganize grab_next_visits tests to better check sorting behavior.
Thu, Jan 21, 12:06 PM
ardumont closed T2961: Add a "not_found" value in OriginVisitStatus.status as Resolved.
Thu, Jan 21, 11:43 AM · Sprint 2021 01, Data Model
ardumont closed T2967: Write journal client subcribed to origin_visit_status topics , a subtask of T2443: Implement a bulk-queryable cache of latest visits for use by the recurrent visit scheduler, as Resolved.
Thu, Jan 21, 11:43 AM · Sprint 2021 01, Scheduling utilities
ardumont closed T2967: Write journal client subcribed to origin_visit_status topics as Resolved.
Thu, Jan 21, 11:43 AM · Sprint 2021 01, Scheduling utilities
ardumont moved T2967: Write journal client subcribed to origin_visit_status topics from in-progress to code review on the Sprint 2021 01 board.
Thu, Jan 21, 11:42 AM · Sprint 2021 01, Scheduling utilities
ardumont closed D4890: journal_client: Read visit_stats entries by batch out of the loop.
Thu, Jan 21, 11:02 AM
ardumont committed rDSCH9fb0dd6c7cc3: journal_client: Read visit_stats entries by batch out of the loop (authored by ardumont).
journal_client: Read visit_stats entries by batch out of the loop
Thu, Jan 21, 11:02 AM
ardumont closed D4888: scheduler: Make origin_visit_stats_get read multiple entries.
Thu, Jan 21, 11:02 AM
ardumont committed rDSCHd464b4cc1f9a: scheduler: Make origin_visit_stats_get read multiple entries (authored by ardumont).
scheduler: Make origin_visit_stats_get read multiple entries
Thu, Jan 21, 11:02 AM
ardumont updated the diff for D4890: journal_client: Read visit_stats entries by batch out of the loop.

Rebase

Thu, Jan 21, 10:57 AM
ardumont updated the diff for D4888: scheduler: Make origin_visit_stats_get read multiple entries.

Rebase

Thu, Jan 21, 10:57 AM
ardumont updated the diff for D4862: varnish: Define vhost with forbidden access.

Use vmod var!

Thu, Jan 21, 10:38 AM
ardumont accepted D4900: deposit monitoring: Increase warning threshold.
Thu, Jan 21, 9:35 AM
ardumont added a comment to D4862: varnish: Define vhost with forbidden access.

...

All in all it might be easier to actually use the var vmod ! :D

Thu, Jan 21, 9:24 AM
ardumont accepted D4895: Add a successive_visits counter to OriginVisitStats.

lgtm

Thu, Jan 21, 8:50 AM

Wed, Jan 20

ardumont accepted D4896: Make the grab_next_visits sql query modular.
Wed, Jan 20, 7:18 PM
ardumont added a comment to D4896: Make the grab_next_visits sql query modular.

1 question about 'enabled' inlined there

Wed, Jan 20, 7:18 PM
ardumont updated the diff for D4888: scheduler: Make origin_visit_stats_get read multiple entries.

Rebase

Wed, Jan 20, 7:10 PM
ardumont updated the diff for D4890: journal_client: Read visit_stats entries by batch out of the loop.

Rebase

Wed, Jan 20, 7:10 PM
ardumont added a comment to T2961: Add a "not_found" value in OriginVisitStatus.status.

We also added a 'failed' status.

Wed, Jan 20, 6:53 PM · Sprint 2021 01, Data Model
ardumont updated the diff for D4890: journal_client: Read visit_stats entries by batch out of the loop.

Rebase

Wed, Jan 20, 6:46 PM
ardumont updated the diff for D4888: scheduler: Make origin_visit_stats_get read multiple entries.

Rebase with fix so tests are happy (i missed the modification last time i rebased and
updated this ¯\_(ツ)_/¯)

Wed, Jan 20, 6:46 PM
ardumont added a comment to D4888: scheduler: Make origin_visit_stats_get read multiple entries.

This needs a rebase to fix the build (i'm on it)

Wed, Jan 20, 6:42 PM
ardumont updated the diff for D4890: journal_client: Read visit_stats entries by batch out of the loop.

Rebase

Wed, Jan 20, 5:50 PM
ardumont updated the diff for D4888: scheduler: Make origin_visit_stats_get read multiple entries.

Rebase

Wed, Jan 20, 5:50 PM
ardumont accepted D4893: Make the max_date() helper function accept *dates as argument.
Wed, Jan 20, 4:33 PM
ardumont committed rDCORE76d2a0033f10: requirements-test: Restrict hypothesis <6 (authored by ardumont).
requirements-test: Restrict hypothesis <6
Wed, Jan 20, 4:03 PM
ardumont accepted D4884: Deploy the scheduler's journal client.
Wed, Jan 20, 3:56 PM
ardumont closed D4892: db: Allow new status values not_found and failed to OriginVisitStatus.
Wed, Jan 20, 3:42 PM
ardumont committed rDSTOe433255dabff: db: Allow new status values not_found, failed to OriginVisitStatus (authored by ardumont).
db: Allow new status values not_found, failed to OriginVisitStatus
Wed, Jan 20, 3:42 PM
ardumont updated the diff for D4892: db: Allow new status values not_found and failed to OriginVisitStatus.

Fix trailing comma which sql is not happy about ¯\_(ツ)_/¯

Wed, Jan 20, 3:36 PM
ardumont closed D4891: model: Allow new status values not_found and failed to OriginVisitStatus.
Wed, Jan 20, 3:31 PM
ardumont committed rDMOD9af451fd6248: model: Allow new status values not_found and failed to OriginVisitStatus (authored by ardumont).
model: Allow new status values not_found and failed to OriginVisitStatus
Wed, Jan 20, 3:31 PM
ardumont moved T2961: Add a "not_found" value in OriginVisitStatus.status from in-progress to code review on the Sprint 2021 01 board.
Wed, Jan 20, 3:20 PM · Sprint 2021 01, Data Model
ardumont updated the diff for D4890: journal_client: Read visit_stats entries by batch out of the loop.

Rebase

Wed, Jan 20, 3:19 PM
ardumont updated the diff for D4888: scheduler: Make origin_visit_stats_get read multiple entries.

Rebase

Wed, Jan 20, 3:19 PM
ardumont updated the diff for D4890: journal_client: Read visit_stats entries by batch out of the loop.

Adapt according to suggestion

Wed, Jan 20, 3:14 PM
ardumont updated the summary of D4890: journal_client: Read visit_stats entries by batch out of the loop.
Wed, Jan 20, 3:09 PM
ardumont updated the diff for D4890: journal_client: Read visit_stats entries by batch out of the loop.

Rebase

Wed, Jan 20, 3:07 PM
ardumont updated the diff for D4888: scheduler: Make origin_visit_stats_get read multiple entries.

Update interface docstring to mention the length of the output list could be less than
the one from the input.

Wed, Jan 20, 3:04 PM
ardumont updated the diff for D4888: scheduler: Make origin_visit_stats_get read multiple entries.

Adapt back to the first implementation, this returns a list of OriginVisitStats

Wed, Jan 20, 3:02 PM
ardumont retitled D4892: db: Allow new status values not_found and failed to OriginVisitStatus from db: Allow new status value 'not_found' to OriginVisitStatus to db: Allow new status values not_found and failed to OriginVisitStatus.
Wed, Jan 20, 2:05 PM
ardumont retitled D4891: model: Allow new status values not_found and failed to OriginVisitStatus from model: Allow new status value 'not_found' to OriginVisitStatus to model: Allow new status values not_found and failed to OriginVisitStatus.
Wed, Jan 20, 2:05 PM
ardumont updated the diff for D4892: db: Allow new status values not_found and failed to OriginVisitStatus.

Adapt diff according to the model update (adding the failed state as well)

Wed, Jan 20, 2:05 PM
ardumont updated the diff for D4891: model: Allow new status values not_found and failed to OriginVisitStatus.

Address reviews

  • add failed status
  • update swh.model.hypothesis_strategies module according to changes
Wed, Jan 20, 2:02 PM
ardumont updated the diff for D4888: scheduler: Make origin_visit_stats_get read multiple entries.

Adapt according to review (most are ok except for the returned type dict change)

Wed, Jan 20, 12:51 PM
ardumont added a comment to D4888: scheduler: Make origin_visit_stats_get read multiple entries.

Maybe it'd make sense to return a dict (url, type) -> OriginVisitStats instead?

Wed, Jan 20, 12:47 PM
ardumont added a comment to D4888: scheduler: Make origin_visit_stats_get read multiple entries.

You should consider using psycopg2.extras.execute_values instead of rolling your own implementation (see the implementation of record_listed_origins).

Wed, Jan 20, 11:50 AM
ardumont added inline comments to D4888: scheduler: Make origin_visit_stats_get read multiple entries.
Wed, Jan 20, 11:44 AM
ardumont added a comment to D4892: db: Allow new status values not_found and failed to OriginVisitStatus.

This reminds me that I noticed this morning that the bump to schema version 165 was forgotten in c24d35f86a06 (in db.py and 30-schema.sql).

Wed, Jan 20, 11:41 AM
ardumont updated subscribers of D4891: model: Allow new status values not_found and failed to OriginVisitStatus.

This makes me wonder if we shouldn't add an explicit failed status too, while we're at it, for explicit failures that couldn't generate a partial snapshot.

Wed, Jan 20, 11:40 AM
ardumont requested review of D4892: db: Allow new status values not_found and failed to OriginVisitStatus.
Wed, Jan 20, 10:52 AM
ardumont added a revision to T2961: Add a "not_found" value in OriginVisitStatus.status: D4892: db: Allow new status values not_found and failed to OriginVisitStatus.
Wed, Jan 20, 10:47 AM · Sprint 2021 01, Data Model
ardumont changed the status of T2961: Add a "not_found" value in OriginVisitStatus.status from Open to Work in Progress.
Wed, Jan 20, 10:43 AM · Sprint 2021 01, Data Model
ardumont requested review of D4891: model: Allow new status values not_found and failed to OriginVisitStatus.
Wed, Jan 20, 10:35 AM
ardumont added a revision to T2961: Add a "not_found" value in OriginVisitStatus.status: D4891: model: Allow new status values not_found and failed to OriginVisitStatus.
Wed, Jan 20, 10:33 AM · Sprint 2021 01, Data Model
ardumont moved T2978: Deploy visit-stats journal client on staging from in-progress to code review on the Sprint 2021 01 board.
Wed, Jan 20, 9:59 AM · Sprint 2021 01, Scheduling utilities
ardumont accepted D4887: Introduce a new lister_get endpoint.
Wed, Jan 20, 9:38 AM
ardumont accepted D4886: Import the journal subcommand in the main swh.scheduler cli.

Thanks.

Wed, Jan 20, 9:35 AM

Tue, Jan 19

ardumont updated the diff for D4890: journal_client: Read visit_stats entries by batch out of the loop.

Keep only interesting visits to ask for their visit_stats informations

Tue, Jan 19, 7:07 PM
ardumont requested review of D4890: journal_client: Read visit_stats entries by batch out of the loop.
Tue, Jan 19, 6:54 PM
ardumont added a revision to T2967: Write journal client subcribed to origin_visit_status topics : D4890: journal_client: Read visit_stats entries by batch out of the loop.
Tue, Jan 19, 6:52 PM · Sprint 2021 01, Scheduling utilities
ardumont requested review of D4888: scheduler: Make origin_visit_stats_get read multiple entries.
Tue, Jan 19, 6:34 PM
ardumont added a revision to T2967: Write journal client subcribed to origin_visit_status topics : D4888: scheduler: Make origin_visit_stats_get read multiple entries.
Tue, Jan 19, 6:32 PM · Sprint 2021 01, Scheduling utilities
ardumont accepted D4885: Make the journal-client cli subcommand automagically loaded.
Tue, Jan 19, 5:14 PM
ardumont added a comment to D4885: Make the journal-client cli subcommand automagically loaded.

I forgot that part.

Tue, Jan 19, 4:57 PM
ardumont accepted D4882: scheduler: Reorganize scheduler configuration files.

lgtm

Tue, Jan 19, 4:04 PM
ardumont accepted D4883: Stop recommending the Slug header as the alternative to <external_identifier>..
Tue, Jan 19, 3:53 PM
ardumont closed D4879: requirements: Make swh.journal and optional dependency.
Tue, Jan 19, 1:50 PM
ardumont committed rDSCH5e609d520592: requirements: Make swh.journal and optional dependency (authored by ardumont).
requirements: Make swh.journal and optional dependency
Tue, Jan 19, 1:50 PM
ardumont edited P924 journal as optional deps for swh.scheduler ends up in error.
Tue, Jan 19, 1:32 PM
ardumont closed D4876: scheduler.cli.journal: Add `swh scheduler journal visit-stats` cli.
Tue, Jan 19, 12:09 PM
ardumont committed rDSCH9395aa0763b7: scheduler.cli.journal: Add `swh scheduler journal-client` cli (authored by ardumont).
scheduler.cli.journal: Add `swh scheduler journal-client` cli
Tue, Jan 19, 12:09 PM
ardumont requested review of D4879: requirements: Make swh.journal and optional dependency.
Tue, Jan 19, 12:09 PM
ardumont added a comment to P924 journal as optional deps for swh.scheduler ends up in error.

out of pipdeptree:

Tue, Jan 19, 11:41 AM
ardumont updated the title for P924 journal as optional deps for swh.scheduler ends up in error from journal as optional deps for swh.scheduler to journal as optional deps for swh.scheduler ends up in error.
Tue, Jan 19, 11:35 AM
ardumont created P924 journal as optional deps for swh.scheduler ends up in error.
Tue, Jan 19, 11:34 AM
ardumont updated the diff for D4876: scheduler.cli.journal: Add `swh scheduler journal visit-stats` cli.

Adapt according to review (thanks ;)

Tue, Jan 19, 11:11 AM