Page MenuHomeSoftware Heritage

ardumont (Antoine R. Dumont)
UserAdministrator

User Details

User Since
Sep 7 2015, 3:42 PM (355 w, 3 d)
Roles
Administrator

Recent Activity

Yesterday

ardumont committed rCJSWHb0872b4ec80b: wip: Start swh-apps/update-dependencies job (authored by ardumont).
wip: Start swh-apps/update-dependencies job
Thu, Jun 30, 5:56 PM
ardumont committed rCJSWH6ce55e57218b: wip: Start swh-apps/update-dependencies job (authored by ardumont).
wip: Start swh-apps/update-dependencies job
Thu, Jun 30, 5:46 PM
ardumont committed rCJSWHdfac84b3435f: wip: Start swh-apps/update-dependencies job (authored by ardumont).
wip: Start swh-apps/update-dependencies job
Thu, Jun 30, 5:43 PM
ardumont committed rCJSWHe41c157273e8: wip: Start swh-apps/update-dependencies job (authored by ardumont).
wip: Start swh-apps/update-dependencies job
Thu, Jun 30, 5:41 PM
ardumont committed rCJSWHd25fc9bb8b51: wip: Start swh-apps/update-dependencies job (authored by ardumont).
wip: Start swh-apps/update-dependencies job
Thu, Jun 30, 5:37 PM
ardumont closed D8063: README.md: Adapt docker instructions.
Thu, Jun 30, 5:06 PM
ardumont committed rCJSWHdf549fba24d0: README.md: Adapt docker instructions (authored by ardumont).
README.md: Adapt docker instructions
Thu, Jun 30, 5:06 PM
ardumont requested review of D8063: README.md: Adapt docker instructions.
Thu, Jun 30, 5:06 PM
ardumont accepted D8062: fix a typo on the production objstorage vhost.

fix typo for fun and profit! ;)

Thu, Jun 30, 4:42 PM
ardumont accepted D8059: Generate .labelobl files to speed-up labelled graph loading.
Thu, Jun 30, 3:46 PM
ardumont added a comment to P1396 possible files in guix sources manifest.

manifests:

Thu, Jun 30, 3:35 PM
ardumont added a comment to P1396 possible files in guix sources manifest.

Related to T3781

Thu, Jun 30, 3:30 PM
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Thu, Jun 30, 3:25 PM · Data Model, Nixguix loader
ardumont created P1396 possible files in guix sources manifest.
Thu, Jun 30, 3:19 PM
ardumont accepted D8057: Add a docker environment to test the job-builder inside jenkins.

couple of questions inline.

Thu, Jun 30, 2:39 PM
ardumont closed T4324: production: Deploy swh-scrubber database and checkers as Resolved.
Thu, Jun 30, 2:36 PM · System administration, Datastore Scrubber
ardumont closed T4361: Unstuck provenance diff build hanging and then aborted as Resolved.
Thu, Jun 30, 2:24 PM · Continuous Integration
ardumont added a comment to T4361: Unstuck provenance diff build hanging and then aborted.

Fixes included in the diff landed [1].

Thu, Jun 30, 2:23 PM · Continuous Integration
ardumont closed D8023: Install `swh provenance origin from-journal` cli and tests.
Thu, Jun 30, 2:10 PM
ardumont committed rDPROVa23c22827dfa: Install `swh provenance origin from-journal` cli and tests (authored by douardda).
Install `swh provenance origin from-journal` cli and tests
Thu, Jun 30, 2:10 PM
ardumont added a comment to T4361: Unstuck provenance diff build hanging and then aborted.

Thanks!

Thu, Jun 30, 2:09 PM · Continuous Integration
ardumont added a comment to P1395 (An Untitled Masterwork).
14:07:39 swh-scrubber@belvedere:5432=> select * from corrupt_object where id='swh:1:rel:a03d79c816c7ffebe3e8d90d7d395482075b086f';
+-[ RECORD 1 ]-----+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id               | swh:1:rel:a03d79c816c7ffebe3e8d90d7d395482075b086f                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| datastore        | 1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| object           | \x88a46e616d65c41864656269616e2f302e32362e312d307562756e7475322e32a76d657373616765c44574616767696e67207061636b61676520707974686f6e2d676c616e63652d73746f72652076657273696f6e2064656269616e2f302e32362e312d307562756e7475322e320aa6746172676574c414e4c28d0c01ea1274e8e9b28f113572c36211d538ab7461726765745f74797065a87265766973696f6ea973796e746865746963c2a6617574686f7283a866756c6c6e616d65c429436f72657920427279616e74203c636f7265792e627279616e744063616e6f6e6963616c2e636f6d3ea46e616d65c40c436f72657920427279616e74a5656d61696cc41a636f7265792e627279616e744063616e6f6e6963616c2e636f6da46461746582a974696d657374616d7082a77365636f6e6473ce5cfe6f7bac6d6963726f7365636f6e647300ac6f66667365745f6279746573c4052d30343030a26964c414a03d79c816c7ffebe3e8d90d7d395482075b086f |
| first_occurrence | 2022-06-27 15:43:47.263775+00                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Thu, Jun 30, 2:07 PM
ardumont accepted D8057: Add a docker environment to test the job-builder inside jenkins.

Looks good enough to start having a local jenkins, that will ease tampering with whatever we need to develop on it.
This without having to tamper directly with the main swh ci instance.
So thanks.

Thu, Jun 30, 2:02 PM
ardumont added inline comments to D8057: Add a docker environment to test the job-builder inside jenkins.
Thu, Jun 30, 1:58 PM
ardumont accepted D8056: requirements-test: Ensure pinned mypy version is used with tox.

thx (and i guess that fixes the master build as well).

Thu, Jun 30, 1:54 PM
ardumont added inline comments to D8033: Arch User Repository (AUR) lister.
Thu, Jun 30, 11:18 AM
ardumont added inline comments to D8033: Arch User Repository (AUR) lister.
Thu, Jun 30, 10:37 AM
ardumont added inline comments to D8023: Install `swh provenance origin from-journal` cli and tests.
Thu, Jun 30, 8:10 AM

Wed, Jun 29

ardumont added inline comments to D8023: Install `swh provenance origin from-journal` cli and tests.
Wed, Jun 29, 7:51 PM
ardumont added inline comments to D8023: Install `swh provenance origin from-journal` cli and tests.
Wed, Jun 29, 6:47 PM
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.
  • Drop self.conn shenanigans (tests no longer stuck but journal client tests are not happy)
Wed, Jun 29, 6:43 PM
ardumont updated subscribers of D8023: Install `swh provenance origin from-journal` cli and tests.
Wed, Jun 29, 6:40 PM
ardumont added a comment to D8023: Install `swh provenance origin from-journal` cli and tests.

I'm of a mind of just push it and check whether that's not some discrepancy with the master build somehow.
Master build is fine and local build as well.

Wed, Jun 29, 6:16 PM
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.

Revert to original commit

Wed, Jun 29, 6:07 PM
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.
  • Tentatively work around the rabbitmq tests build failures 5/n
Wed, Jun 29, 5:48 PM
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.
  • Tentatively work around the rabbitmq tests build failures 4/n
Wed, Jun 29, 5:38 PM
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.
  • Tentatively work around the rabbitmq initialization 3/n
Wed, Jun 29, 5:11 PM
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.
  • Tentatively work around the rabbitmq initialization.
Wed, Jun 29, 4:37 PM
ardumont accepted D8052: crates: Remove redundant 'max_content_length' argument.
Wed, Jun 29, 4:08 PM
ardumont added a project to T4361: Unstuck provenance diff build hanging and then aborted: Continuous Integration.
Wed, Jun 29, 3:23 PM · Continuous Integration
ardumont accepted D8046: Remove SingleFileMapping from JsonMapping's base classes.
Wed, Jun 29, 3:21 PM
ardumont accepted D8049: Crates loader, use **kwargs on task and loader initialisation instead of named args.
Wed, Jun 29, 3:20 PM
ardumont updated the task description for T4104: Ingest crates.io (Rust).
Wed, Jun 29, 3:19 PM · Crates loader, Crates lister, Archive coverage
ardumont added a comment to T4104: Ingest crates.io (Rust).

What the next steps here?

Wed, Jun 29, 3:19 PM · Crates loader, Crates lister, Archive coverage
ardumont updated the task description for T4104: Ingest crates.io (Rust).
Wed, Jun 29, 3:15 PM · Crates loader, Crates lister, Archive coverage
ardumont updated the task description for T4104: Ingest crates.io (Rust).
Wed, Jun 29, 3:15 PM · Crates loader, Crates lister, Archive coverage
ardumont accepted D8050: Add missing copyright notices to the entire Java codebase.
Wed, Jun 29, 3:09 PM
ardumont published D8050: Add missing copyright notices to the entire Java codebase for review.

well, i'll trust the copyright date are ok according to the existing code ;)

Wed, Jun 29, 3:09 PM
ardumont committed rSPSITE2f8a63356a46: scrubber: Set objstorage key to noop unused by scrubber services (authored by ardumont).
scrubber: Set objstorage key to noop unused by scrubber services
Wed, Jun 29, 2:46 PM
ardumont closed D8042: scrubber: Set objstorage key to noop unused by scrubber services.
Wed, Jun 29, 2:45 PM
ardumont committed rSPSITE7e3d3922e6eb: scrubber: Set objstorage key to noop unused by scrubber services (authored by ardumont).
scrubber: Set objstorage key to noop unused by scrubber services
Wed, Jun 29, 2:45 PM
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.

including the right amount of commits

Wed, Jun 29, 11:54 AM
ardumont added a comment to T4361: Unstuck provenance diff build hanging and then aborted.

wip tryout [1] without those tests to ensure whether it's the rabbitmq fixture which hangs or not.

[1] D8023#209122

Wed, Jun 29, 11:52 AM · Continuous Integration
ardumont added a comment to T4361: Unstuck provenance diff build hanging and then aborted.

wip tryout [1] without those tests to ensure whether it's the rabbitmq fixture which hangs or not.

Wed, Jun 29, 11:49 AM · Continuous Integration
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.

wip tryout (drop rabbitmq from fixture) to check if that still hangs or not

Wed, Jun 29, 11:45 AM
ardumont updated the task description for T4361: Unstuck provenance diff build hanging and then aborted.
Wed, Jun 29, 11:40 AM · Continuous Integration
ardumont added a comment to T4361: Unstuck provenance diff build hanging and then aborted.

That also seem to incur a heavy load on the jenkins node thyssen:

11:28 <+swhbot> icinga PROBLEM: service load on thyssen.internal.softwareheritage.org is WARNING: WARNING - load average: 30.18, 22.54, 11.58
Wed, Jun 29, 11:35 AM · Continuous Integration
ardumont renamed T4361: Unstuck provenance diff build hanging and then aborted from Unstuck provenance diff build to Unstuck provenance diff build hanging and then aborted.
Wed, Jun 29, 11:34 AM · Continuous Integration
ardumont created T4361: Unstuck provenance diff build hanging and then aborted.
Wed, Jun 29, 11:34 AM · Continuous Integration
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.

Trigger build again

Wed, Jun 29, 11:19 AM
ardumont renamed T3781: Replace the Nixguix loader with a lister from Replace the Nixguix loader with a lister? to Replace the Nixguix loader with a lister.
Wed, Jun 29, 11:05 AM · Data Model, Nixguix loader
ardumont added a subtask for T1991: Implement a Guix/Nix loader: T2879: Finalize nixguix loader implementation.
Wed, Jun 29, 10:32 AM · Package Loader
ardumont added a parent task for T2879: Finalize nixguix loader implementation: T1991: Implement a Guix/Nix loader.
Wed, Jun 29, 10:32 AM · Nixguix loader, Package Loader
ardumont closed T2623: swh.core.tarball: Add support for more tarballs format as Resolved.

It's been done for a while now.
And there is a mechanism in place in swh.core.tarball to extend it further if need be.
Closing.

Wed, Jun 29, 10:31 AM · Core & foundations
ardumont closed T2687: nixguix: Add support for downloads over FTP as Resolved.
Wed, Jun 29, 10:30 AM · Nixguix loader, Easy hack
ardumont closed T2687: nixguix: Add support for downloads over FTP, a subtask of T2879: Finalize nixguix loader implementation, as Resolved.
Wed, Jun 29, 10:30 AM · Nixguix loader, Package Loader
ardumont added a subtask for T2879: Finalize nixguix loader implementation: T2687: nixguix: Add support for downloads over FTP.
Wed, Jun 29, 10:30 AM · Nixguix loader, Package Loader
ardumont added a parent task for T2687: nixguix: Add support for downloads over FTP: T2879: Finalize nixguix loader implementation.
Wed, Jun 29, 10:30 AM · Nixguix loader, Easy hack
ardumont added a subtask for T2879: Finalize nixguix loader implementation: T3294: nixguix: Add support for pseudo-URLs with a missing schema.
Wed, Jun 29, 10:29 AM · Nixguix loader, Package Loader
ardumont added a parent task for T3294: nixguix: Add support for pseudo-URLs with a missing schema: T2879: Finalize nixguix loader implementation.
Wed, Jun 29, 10:29 AM · Easy hack, Nixguix loader
ardumont added a subtask for T2879: Finalize nixguix loader implementation: T3781: Replace the Nixguix loader with a lister.
Wed, Jun 29, 10:28 AM · Nixguix loader, Package Loader
ardumont added a parent task for T3781: Replace the Nixguix loader with a lister: T2879: Finalize nixguix loader implementation.
Wed, Jun 29, 10:28 AM · Data Model, Nixguix loader

Tue, Jun 28

ardumont accepted D8044: tests: Update mock to work with objstorage >= v2.0.0.
Tue, Jun 28, 6:08 PM
ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

So taking a bit more look into this possible new lister, we'd end up with the following
possible outputs:

  • artifacts url which are mostly tarballs [1] and sometimes file [2]
  • dvcs repositories delegated to dedicated loader to ingestion: svn [3], hg [4], git [5] (out of guix manifest)
  • Other stuff can be ignored [6]
Tue, Jun 28, 6:03 PM · Data Model, Nixguix loader
ardumont added a comment to D8023: Install `swh provenance origin from-journal` cli and tests.

Build was aborted

Tue, Jun 28, 4:51 PM
ardumont committed rCJSWH0ca1e42bcb55: provenance: Increase build timeout (authored by ardumont).
provenance: Increase build timeout
Tue, Jun 28, 11:32 AM
ardumont updated the diff for D8023: Install `swh provenance origin from-journal` cli and tests.

Rebase

Tue, Jun 28, 10:24 AM
ardumont closed T4348: Deploy swh-indexer v2.0.2 on production and staging as Resolved.
Tue, Jun 28, 9:58 AM · Indexer, System administration
ardumont added a comment to D8041: scrubber: Deactivate the unneeded objstorage configuration part.

You should set that in the scrubber config in common.yaml rather than using a per-host override

yes, totally, i'll attend to that later in the day.

Tue, Jun 28, 9:50 AM
ardumont updated the test plan for D8042: scrubber: Set objstorage key to noop unused by scrubber services.
Tue, Jun 28, 9:49 AM
ardumont requested review of D8042: scrubber: Set objstorage key to noop unused by scrubber services.
Tue, Jun 28, 9:48 AM
ardumont added a comment to D8041: scrubber: Deactivate the unneeded objstorage configuration part.

You should set that in the scrubber config in common.yaml rather than using a per-host override

Tue, Jun 28, 8:02 AM
ardumont added a comment to T4324: production: Deploy swh-scrubber database and checkers.

Seems to have done the job:

07:59:35 swh-scrubber@belvedere:5432=> select now(), count(*) from corrupt_object ;
+-------------------------------+-------+
|              now              | count |
+-------------------------------+-------+
| 2022-06-28 05:59:39.490737+00 | 76511 |
+-------------------------------+-------+
(1 row)
Tue, Jun 28, 8:00 AM · System administration, Datastore Scrubber

Mon, Jun 27

ardumont moved T4348: Deploy swh-indexer v2.0.2 on production and staging from in-progress to deployed/landed/monitoring on the System administration board.
Mon, Jun 27, 5:48 PM · Indexer, System administration
ardumont changed the status of T4348: Deploy swh-indexer v2.0.2 on production and staging from Open to Work in Progress.
Mon, Jun 27, 5:37 PM · Indexer, System administration
ardumont moved T4324: production: Deploy swh-scrubber database and checkers from in-progress to deployed/landed/monitoring on the System administration board.
Mon, Jun 27, 5:33 PM · System administration, Datastore Scrubber
ardumont updated the task description for T4324: production: Deploy swh-scrubber database and checkers.
Mon, Jun 27, 5:33 PM · System administration, Datastore Scrubber
ardumont closed D8041: scrubber: Deactivate the unneeded objstorage configuration part.
Mon, Jun 27, 5:31 PM
ardumont committed rSPSITEdadeb097cba9: scrubber: Deactivate the unneeded objstorage configuration part (authored by ardumont).
scrubber: Deactivate the unneeded objstorage configuration part
Mon, Jun 27, 5:31 PM
ardumont updated the diff for D8041: scrubber: Deactivate the unneeded objstorage configuration part.

Amend commit message

Mon, Jun 27, 5:31 PM
ardumont updated the summary of D8041: scrubber: Deactivate the unneeded objstorage configuration part.
Mon, Jun 27, 5:30 PM
ardumont retitled D8041: scrubber: Deactivate the unneeded objstorage configuration part from scrubber: Use multiplexed objstorage without azure to scrubber: Deactivate the unneeded objstorage configuration part.
Mon, Jun 27, 5:30 PM
ardumont updated the task description for T4324: production: Deploy swh-scrubber database and checkers.
Mon, Jun 27, 5:29 PM · System administration, Datastore Scrubber
ardumont updated the test plan for D8041: scrubber: Deactivate the unneeded objstorage configuration part.
Mon, Jun 27, 5:26 PM
ardumont updated the diff for D8041: scrubber: Deactivate the unneeded objstorage configuration part.

Use proper repository to update diff

Mon, Jun 27, 5:23 PM
ardumont updated the test plan for D8041: scrubber: Deactivate the unneeded objstorage configuration part.
Mon, Jun 27, 5:21 PM
ardumont updated the diff for D8041: scrubber: Deactivate the unneeded objstorage configuration part.

Use noop objstorage

Mon, Jun 27, 5:20 PM
ardumont added a comment to D8041: scrubber: Deactivate the unneeded objstorage configuration part.

After discussing with olasd, turns out that the part declaring the objstorage is not currently needed so i'll try to do better.

Mon, Jun 27, 5:12 PM
ardumont accepted D8040: Limit the number of entries in the cache.
Mon, Jun 27, 5:10 PM
ardumont added inline comments to D8040: Limit the number of entries in the cache.
Mon, Jun 27, 5:07 PM