Page MenuHomeSoftware Heritage
Feed Advanced Search

Sep 30 2022

ardumont added inline comments to D8582: Allow more checksum computations in Content model.
Sep 30 2022, 9:23 AM

Sep 29 2022

ardumont added a comment to D8581: Add Content Loader to ingest raw content file.

@vlorentz I should have started with this... from the nixguix manifest, the integrity is for now only sha256... [1]
So not sure we need to touch the model after all [2], especially since that diff got a tad bigger since you reviewed it...

[2] D8582

$ jq . /var/tmp/sources.json | grep -c sha256
13629
$ jq . /var/tmp/sources.json | grep -c sha384
0
$ jq . /var/tmp/sources.json | grep -c sha512
0
Sep 29 2022, 7:20 PM
ardumont added a comment to D8581: Add Content Loader to ingest raw content file.

@vlorentz I should have started with this... from the nixguix manifest, the integrity is for now only sha256... [1]
So not sure we need to touch the model after all [2], especially since that diff got a tad bigger since you reviewed it...

Sep 29 2022, 7:16 PM
ardumont updated the diff for D8582: Allow more checksum computations in Content model.

Adding tests revealed it missed stuff...

Sep 29 2022, 7:13 PM
ardumont updated the diff for D8581: Add Content Loader to ingest raw content file.

Fix docstring which failed the build!

Sep 29 2022, 6:55 PM
ardumont added inline comments to D8581: Add Content Loader to ingest raw content file.
Sep 29 2022, 6:28 PM
ardumont updated the diff for D8581: Add Content Loader to ingest raw content file.

Address review

Sep 29 2022, 6:27 PM
ardumont updated the summary of D8581: Add Content Loader to ingest raw content file.
Sep 29 2022, 6:14 PM
ardumont updated the summary of D8581: Add Content Loader to ingest raw content file.
Sep 29 2022, 6:12 PM
ardumont requested review of D8582: Allow more checksum computations in Content model.
Sep 29 2022, 5:59 PM
ardumont added a revision to T3781: Replace the Nixguix loader with a lister: D8582: Allow more checksum computations in Content model.
Sep 29 2022, 5:57 PM · Data Model, Nixguix loader
ardumont added a comment to D8581: Add Content Loader to ingest raw content file.

Could you use a smaller test file? That one is really big...

Well, that's the sole one i found.

any file would do, you don't need to get one from Guix/Nix

Sep 29 2022, 5:45 PM
ardumont added a comment to D8581: Add Content Loader to ingest raw content file.

please document which terminal of the grammar you are aiming for. (the current implementation is hash-expression, but I don't know if that's intentional)

Sep 29 2022, 5:39 PM
ardumont created P1466 docker-compose.override.yml.
Sep 29 2022, 5:00 PM
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Sep 29 2022, 4:54 PM · Data Model, Nixguix loader
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Sep 29 2022, 4:53 PM · Data Model, Nixguix loader
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Sep 29 2022, 4:53 PM · Data Model, Nixguix loader
ardumont accepted D8535: docker/conf: Declare task queues for new listers and loaders.

ok then ;)

Sep 29 2022, 4:51 PM
ardumont accepted D8570: Index extrinsic metadata from the deposit.
Sep 29 2022, 4:49 PM
ardumont accepted D8576: conftest: Replace multiprocessing hack when pytest-cov >= 4 is installed.
Sep 29 2022, 4:44 PM
ardumont abandoned D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.

This cannot work as explained time and again.

Sep 29 2022, 4:37 PM
ardumont updated the summary of D8341: Add nixguix lister.
Sep 29 2022, 4:31 PM
ardumont retitled D8341: Add nixguix lister from [WIP] nixguix: Boostrap lister to nixguix: Boostrap lister.
Sep 29 2022, 4:30 PM
ardumont updated the summary of D8341: Add nixguix lister.
Sep 29 2022, 4:30 PM
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Sep 29 2022, 4:28 PM · Data Model, Nixguix loader
ardumont requested review of D8581: Add Content Loader to ingest raw content file.
Sep 29 2022, 4:24 PM
ardumont added a revision to T3781: Replace the Nixguix loader with a lister: D8581: Add Content Loader to ingest raw content file.
Sep 29 2022, 4:21 PM · Data Model, Nixguix loader
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Sep 29 2022, 4:19 PM · Data Model, Nixguix loader
ardumont updated the summary of D8341: Add nixguix lister.
Sep 29 2022, 11:52 AM
ardumont updated the summary of D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.
Sep 29 2022, 11:05 AM
ardumont updated the summary of D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.
Sep 29 2022, 11:05 AM
ardumont updated the summary of D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.
Sep 29 2022, 11:05 AM
ardumont updated the summary of D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.
Sep 29 2022, 11:04 AM
ardumont updated the summary of D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.
Sep 29 2022, 11:04 AM
ardumont updated the summary of D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.
Sep 29 2022, 11:01 AM
ardumont updated the summary of D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.
Sep 29 2022, 10:59 AM
ardumont updated the diff for D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.

Add tests that explicitly fail as it's missing the required version in the current
package loader implementation.

Sep 29 2022, 10:58 AM
ardumont accepted D8577: Disable ping on hosts/ips managed by metallb.

til

Sep 29 2022, 10:51 AM
ardumont accepted D8539: Add random directory sampling policy.

Fine, i've one comment i'd like others to have a look at though [1] regarding where
that new discovery (interface) code should go. It feels currently a bit off to me that this code
is in loader-core. Loaders are not the sole archive consumers (scanner, webapp, cli, indexer, cooker, ...).

Sep 29 2022, 10:47 AM
ardumont added 1 blocking reviewer(s) for D8539: Add random directory sampling policy: Reviewers.
Sep 29 2022, 10:44 AM
ardumont added inline comments to D8539: Add random directory sampling policy.
Sep 29 2022, 10:43 AM
ardumont added a comment to D8538: Setup async interface for discovery module.

Still a couple of remarks/questions.

  1. Isn't this missing an asyncio dependency requirements in /requirements.txt
Sep 29 2022, 10:40 AM
ardumont updated the task description for T1721: Implementation of Gogs Lister.
Sep 29 2022, 10:29 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
ardumont closed T4423: Gogs pagination API breaks because of fatal repos, a subtask of T1721: Implementation of Gogs Lister, as Resolved.
Sep 29 2022, 10:29 AM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
ardumont closed T4423: Gogs pagination API breaks because of fatal repos as Resolved.

That's been worked around in the lister to skip those 500 which were breaking the pagination listing.
There is no reply to the upstream bug yet.

Sep 29 2022, 10:29 AM · Gogs lister, Origin-Gitea/Gogs, Lister, Archive coverage
ardumont committed rSPREf463e3e9bf4c: staging: Migrate elastic worker to pompidou (authored by ardumont).
staging: Migrate elastic worker to pompidou
Sep 29 2022, 10:22 AM
ardumont committed rSPRE63d87f908275: staging: Decrease worker0 hardware (authored by ardumont).
staging: Decrease worker0 hardware
Sep 29 2022, 10:22 AM
ardumont accepted D8565: pattern: Ensure accurate origin counts returned by run method.

one question where i got confused by the old code inline ¯\_(ツ)_/¯ .

Sep 29 2022, 9:55 AM
ardumont accepted D8572: patterns: adds a list of common patterns to exclude.

"As usual", i don't know how well that module is covered, it'd be good to
have a test around the scan function to check it behaves as we want.

Sep 29 2022, 9:47 AM
ardumont requested changes to D8571: from_disks: fix some of the pattern checking logic.
Sep 29 2022, 9:42 AM
ardumont added a comment to D8571: from_disks: fix some of the pattern checking logic.

This lgtm overall.

Sep 29 2022, 9:42 AM
ardumont updated the summary of D8571: from_disks: fix some of the pattern checking logic.
Sep 29 2022, 9:37 AM
ardumont added inline comments to D8556: Small improvements to the mirror tests.
Sep 29 2022, 9:29 AM
ardumont added inline comments to D8556: Small improvements to the mirror tests.
Sep 29 2022, 9:23 AM

Sep 28 2022

ardumont updated the task description for T4475: Ingest bower.io (Javascript package manager).
Sep 28 2022, 7:27 PM · Bower lister, Archive coverage
ardumont closed T4459: Deploy swh-indexer > v2.6 on staging then production, a subtask of T4392: Metadata Indexer for NuGet (.nuspec), as Resolved.
Sep 28 2022, 7:22 PM · Indexer
ardumont closed T4459: Deploy swh-indexer > v2.6 on staging then production, a subtask of T4401: Index metadata from the deposit, as Resolved.
Sep 28 2022, 7:22 PM · SWORD deposit, Indexer, Metadata workflow
ardumont closed T4459: Deploy swh-indexer > v2.6 on staging then production as Resolved.
Sep 28 2022, 7:22 PM · Indexer, System administration
ardumont closed T4545: staging: Deploy golang stack, a subtask of T4124: Golang support, as Resolved.
Sep 28 2022, 7:21 PM · Golang loader, Golang lister, Archive coverage
ardumont closed T4545: staging: Deploy golang stack as Resolved.
Sep 28 2022, 7:21 PM · System administration, Golang loader, Golang lister, Archive coverage
ardumont updated the task description for T4545: staging: Deploy golang stack.
Sep 28 2022, 7:21 PM · System administration, Golang loader, Golang lister, Archive coverage
ardumont added a comment to T4545: staging: Deploy golang stack.

Ingestion ok

Sep 28 2022, 7:20 PM · System administration, Golang loader, Golang lister, Archive coverage
ardumont closed T4555: staging: Deploy bower lister, a subtask of T4475: Ingest bower.io (Javascript package manager), as Resolved.
Sep 28 2022, 7:19 PM · Bower lister, Archive coverage
ardumont closed T4555: staging: Deploy bower lister as Resolved.
Sep 28 2022, 7:19 PM · System administration, Bower lister, Archive coverage
ardumont updated the task description for T4555: staging: Deploy bower lister.
Sep 28 2022, 7:19 PM · System administration, Bower lister, Archive coverage
ardumont added a comment to T4555: staging: Deploy bower lister.

Some bower origins visited and ok:

Sep 28 2022, 7:18 PM · System administration, Bower lister, Archive coverage
ardumont closed T4540: staging: Migrate remaining static services to elastic workers as Resolved.
Sep 28 2022, 7:16 PM · meta-task, System administration, Roadmap 2022
ardumont closed T4540: staging: Migrate remaining static services to elastic workers, a subtask of T4144: Elastic worker infrastructure, as Resolved.
Sep 28 2022, 7:16 PM · meta-task, System administration, Roadmap 2022
ardumont updated the task description for T4540: staging: Migrate remaining static services to elastic workers.
Sep 28 2022, 7:16 PM · meta-task, System administration, Roadmap 2022
ardumont added inline comments to D8539: Add random directory sampling policy.
Sep 28 2022, 7:14 PM
ardumont added inline comments to D8314: add a documentation landing page.
Sep 28 2022, 7:07 PM
ardumont added a comment to D8538: Setup async interface for discovery module.

I'm coming late to the battle ¯\_(ツ)_/¯.

Sep 28 2022, 7:03 PM
ardumont accepted D8568: codemeta: Fix crash when translating PropertyValue objects from codemeta-in-SWORD.

one suggestion inline.

Sep 28 2022, 7:02 PM
ardumont accepted D8556: Small improvements to the mirror tests.

one non-blocking suggestion inline.

Sep 28 2022, 7:00 PM
ardumont accepted D8567: argocd: Remove public site.
Sep 28 2022, 6:42 PM
ardumont updated the diff for D8341: Add nixguix lister.

Use the right number of commits

Sep 28 2022, 5:05 PM
ardumont updated the diff for D8341: Add nixguix lister.

Rebase and adapt according to recent lister refactoring

Sep 28 2022, 5:04 PM
ardumont accepted D8560: add_forge_now: Add useful links in requests tables.
Sep 28 2022, 1:32 PM
ardumont added a comment to D8560: add_forge_now: Add useful links in requests tables.

Although, i only skimmed through the code. I was mostly sold by the screenshots ¯\_(ツ)_/¯.

Sep 28 2022, 1:31 PM
ardumont created P1462 gogs origins behind authentication.
Sep 28 2022, 10:47 AM
ardumont moved T4540: staging: Migrate remaining static services to elastic workers from in-progress to deployed/landed/monitoring on the System administration board.
Sep 28 2022, 10:45 AM · meta-task, System administration, Roadmap 2022
ardumont committed R260:cb922ca3892c: staging: Add missing cvs type in high priority loader (authored by ardumont).
staging: Add missing cvs type in high priority loader
Sep 28 2022, 10:29 AM
ardumont committed R260:95b83d335884: staging/git: Add replica (authored by ardumont).
staging/git: Add replica
Sep 28 2022, 10:29 AM
ardumont accepted D8559: argocd: Prepare the configuration to migrate to the internal admin network.
Sep 28 2022, 10:23 AM
ardumont committed R260:deee735c376c: staging/git: Add replica (authored by ardumont).
staging/git: Add replica
Sep 28 2022, 10:16 AM
ardumont committed rSPSITEb81df129d2c5: staging: Migrate last indexer journal client to elastic workers (authored by ardumont).
staging: Migrate last indexer journal client to elastic workers
Sep 28 2022, 10:10 AM
ardumont committed R260:5d292b66596a: staging/indexer: Deploy content fossology license indexer (authored by ardumont).
staging/indexer: Deploy content fossology license indexer
Sep 28 2022, 9:23 AM
ardumont accepted D8551: first add forge now requests added to changelog.

I think we can open the linked tasks. They are tagged staff but there is nothing inside preventing it from going public.
@vsellier do you concur about the task ^ ?

Sep 28 2022, 9:22 AM
ardumont committed R259:52e66acec254: indexer: Add swh bullseye debian repo for runtime dependency (authored by ardumont).
indexer: Add swh bullseye debian repo for runtime dependency
Sep 28 2022, 9:17 AM

Sep 27 2022

ardumont updated the task description for T4540: staging: Migrate remaining static services to elastic workers.
Sep 27 2022, 7:31 PM · meta-task, System administration, Roadmap 2022
ardumont committed rSPRE91a8a005d5b7: staging: Decomission worker1 (authored by ardumont).
staging: Decomission worker1
Sep 27 2022, 7:23 PM
ardumont committed rSPSITE2f67bfc5d310: staging: Migrate most indexer journal clients to elastic workers (authored by ardumont).
staging: Migrate most indexer journal clients to elastic workers
Sep 27 2022, 7:15 PM
ardumont updated the task description for T4540: staging: Migrate remaining static services to elastic workers.
Sep 27 2022, 7:09 PM · meta-task, System administration, Roadmap 2022
ardumont committed R260:a973c3232a86: staging/indexer: Add missing scheduler configuration entry (authored by ardumont).
staging/indexer: Add missing scheduler configuration entry
Sep 27 2022, 7:07 PM
ardumont added a reverting change for rSKCONFbff363cfff19: archive-staging: Switch to master branch: rSKCONF1831ef5159e4: Revert "archive-staging: Switch to master branch".
Sep 27 2022, 7:01 PM
ardumont committed rSKCONF1831ef5159e4: Revert "archive-staging: Switch to master branch" (authored by ardumont).
Revert "archive-staging: Switch to master branch"
Sep 27 2022, 7:01 PM
ardumont committed R259:9108900d9e05: swh.indexer: Fix again entrypoint (authored by ardumont).
swh.indexer: Fix again entrypoint
Sep 27 2022, 5:06 PM
ardumont committed rSKCONFbff363cfff19: archive-staging: Switch to master branch (authored by ardumont).
archive-staging: Switch to master branch
Sep 27 2022, 5:03 PM
ardumont committed R260:47c886e2097e: staging: Deploy indexer journal client in elastic stack (authored by ardumont).
staging: Deploy indexer journal client in elastic stack
Sep 27 2022, 5:00 PM
ardumont accepted D8552: docs: Update archive stats.
Sep 27 2022, 4:59 PM