Page MenuHomeSoftware Heritage
Feed Advanced Search

May 13 2022

ardumont moved T4238: Deploy latest loaders version (> v3.4) from Backlog to Weekly backlog on the System administration board.
May 13 2022, 10:19 AM · System administration, Core Loader
ardumont triaged T4238: Deploy latest loaders version (> v3.4) as High priority.
May 13 2022, 10:17 AM · System administration, Core Loader
ardumont requested review of D7827: Initialize the success boolean early to avoid unbound exception.
May 13 2022, 10:00 AM

May 12 2022

ardumont accepted D7824: misc/coverage: Add counts for origins listed from Packagist.
May 12 2022, 8:05 PM
ardumont committed R260:7004c152ebac: Install worker chart (authored by ardumont).
Install worker chart
May 12 2022, 5:53 PM
ardumont committed rDSNIPa7b19fc2f65d: elastic: Create the metadata-fetcher secret to all namespaces (authored by ardumont).
elastic: Create the metadata-fetcher secret to all namespaces
May 12 2022, 5:50 PM
ardumont committed rDSNIP410f50205dd2: elastic: Expose instance yaml files to install specific loaders (authored by ardumont).
elastic: Expose instance yaml files to install specific loaders
May 12 2022, 5:39 PM
ardumont committed rDSNIP66d01ed59456: elastic-worker: Declare services within namespaces (authored by ardumont).
elastic-worker: Declare services within namespaces
May 12 2022, 5:23 PM
ardumont accepted D7819: misc/coverage: Improve display of origin counters in production/staging.
May 12 2022, 4:44 PM
ardumont added a comment to D7821: misc/coverage: Update search pattern for sourceforge bzr/cvs origins.

sure

May 12 2022, 4:43 PM
ardumont accepted D7822: misc/coverage: Add counts for origins listed from maven repositories.
May 12 2022, 4:38 PM
ardumont published D7822: misc/coverage: Add counts for origins listed from maven repositories for review.
May 12 2022, 4:38 PM
ardumont accepted D7821: misc/coverage: Update search pattern for sourceforge bzr/cvs origins.
May 12 2022, 4:38 PM
ardumont published D7821: misc/coverage: Update search pattern for sourceforge bzr/cvs origins for review.
May 12 2022, 4:37 PM
ardumont accepted D7820: misc/coverage: Add count for origins listed from phabricator instances.
May 12 2022, 4:36 PM
ardumont published D7820: misc/coverage: Add count for origins listed from phabricator instances for review.
May 12 2022, 4:36 PM
ardumont committed rDSNIPae8800c987d1: elastic-worker: Fix amqp credentials to let keda access rabbitmq (authored by ardumont).
elastic-worker: Fix amqp credentials to let keda access rabbitmq
May 12 2022, 4:11 PM
ardumont committed rDSNIP8dd6f1e2f2a0: elastic-worker: Deal with amqp credentials as secrets (authored by ardumont).
elastic-worker: Deal with amqp credentials as secrets
May 12 2022, 4:11 PM
ardumont committed rDSNIP7a6ea5da7010: elastic-worker: Plug sentry setup as secret for loader git (authored by ardumont).
elastic-worker: Plug sentry setup as secret for loader git
May 12 2022, 4:11 PM
ardumont committed rDSNIP1e2c9088e33d: Bump new release for elastic workers (authored by ardumont).
Bump new release for elastic workers
May 12 2022, 4:11 PM
ardumont accepted D7795: Test to detect uppercase NPM metadata file.
May 12 2022, 3:34 PM
ardumont committed R260:f5beb6e25a74: Initial commit: Bootstrap the README (authored by ardumont).
Initial commit: Bootstrap the README
May 12 2022, 2:01 PM
ardumont accepted D7818: webapp: Add missing conf for swh_raw_object throttle scope.

ah hence the issue @zack mentioned in irc.

May 12 2022, 1:57 PM
ardumont committed R261:79c9192cd7f6: Reference sysadm repositories in .mrconfig (authored by ardumont).
Reference sysadm repositories in .mrconfig
May 12 2022, 1:55 PM
ardumont committed R261:b142e95a8ec9: Initial commit: Bootstrap the README (authored by ardumont).
Initial commit: Bootstrap the README
May 12 2022, 1:48 PM
ardumont accepted D7816: interface: Return enabled origins only by default in get_listed_origins.

Even better ;)

May 12 2022, 1:21 PM
ardumont created P1362 (An Untitled Masterwork).
May 12 2022, 11:54 AM
ardumont added a comment to T4235: [As a temporary solution] deploy the statsd-exporter binary published by prometheus.

Note:

11:01 <+douardda> (note that another temporary solution would be to craft a deb using the published binary instead of building it in d/rules, not sure if it's simpler and acceptable)
11:02 <+vlorentz> and it would work better with the mirror stack
May 12 2022, 11:03 AM · System administration
ardumont added a comment to D7814: Remove dead code from the Python interface.

I'm fine with this code cleanup, with one caveat: that we document/ship the systemd startup service (and its meaning, including some intuitions about the trade-offs you mention) somewhere, in replacement of the cachemount command.

May 12 2022, 11:02 AM
ardumont added a comment to D7795: Test to detect uppercase NPM metadata file.

I missed it yesterday, can you please squash your commits into 1 (including the add to contributor one, it's fine ;) ?
(And then update the diff, wait for tests to go green and push, the diff will close then).

May 12 2022, 11:01 AM
ardumont accepted D7815: Add swh-graphql application.
May 12 2022, 10:59 AM

May 11 2022

ardumont accepted D7795: Test to detect uppercase NPM metadata file.

Thanks!

May 11 2022, 6:36 PM
ardumont accepted D7814: Remove dead code from the Python interface.
May 11 2022, 5:55 PM
ardumont added a comment to D7814: Remove dead code from the Python interface.

@seirl: So, for future version deployments, nothing changes, right?

Nope, nothing at all. It's literally dead code.

May 11 2022, 5:55 PM
ardumont accepted D7813: Make 'python -m swh' work as cli entry point.

Interesting.

May 11 2022, 5:32 PM
ardumont added a comment to D7814: Remove dead code from the Python interface.

@seirl: So, for future version deployments, nothing changes, right?

May 11 2022, 5:31 PM
ardumont planned changes to D7809: elastic-workers: Let the loader some time to finish gracefully.

In any case, my current implementation is not working ¯\_(ツ)_/¯.
That fails the restart of the pods somehow (segregating out that commit and everyting gets back to normal).

May 11 2022, 4:00 PM
ardumont updated subscribers of T4233: Ingest Arch Linux.
May 11 2022, 3:26 PM · Arch loader, Arch Lister, Archive coverage
ardumont shifted T4233: Ingest Arch Linux from the Restricted Space space to the S1 Public space.
May 11 2022, 3:26 PM · Arch loader, Arch Lister, Archive coverage
ardumont added a revision to T4233: Ingest Arch Linux: D7812: [WIP] Add arch lister module..
May 11 2022, 3:26 PM · Arch loader, Arch Lister, Archive coverage
ardumont updated the summary of D7812: [WIP] Add arch lister module..
May 11 2022, 3:26 PM
ardumont triaged T4233: Ingest Arch Linux as Normal priority.
May 11 2022, 3:17 PM · Arch loader, Arch Lister, Archive coverage
ardumont triaged T4232: Listers: Canonicalize listed github origins as Normal priority.
May 11 2022, 11:55 AM · Lister
ardumont closed T4214: Production : Deploy bzr/bazaar Loader, a subtask of T3610: Bazaar/Breezy loader, as Resolved.
May 11 2022, 10:20 AM · BZR loader
ardumont closed T4214: Production : Deploy bzr/bazaar Loader as Resolved.
May 11 2022, 10:20 AM · System administration
ardumont accepted D7808: Add add-forge-now inbound email address.
May 11 2022, 10:09 AM
ardumont added a comment to D7809: elastic-workers: Let the loader some time to finish gracefully.

What component is providing the health http endpoint in the pod?

May 11 2022, 10:03 AM
ardumont added a project to T4229: Properly declare a centralized logging configuration and then reuse: Core & foundations.
May 11 2022, 9:25 AM · Core & foundations
ardumont abandoned D7805: Allow setting up logging level in core loader.

Abandoned in favor of the future work in T4229

May 11 2022, 9:20 AM
ardumont added a revision to T4229: Properly declare a centralized logging configuration and then reuse: D7805: Allow setting up logging level in core loader.
May 11 2022, 9:20 AM · Core & foundations
ardumont updated the summary of D7805: Allow setting up logging level in core loader.
May 11 2022, 9:20 AM
ardumont added a comment to D7805: Allow setting up logging level in core loader.
In D7805#203180, @olasd wrote:

Yeah, I'm not saying it should happen now, but the way this stuff has been handled has been inconsistent/painful/annoying me for a while,

May 11 2022, 9:20 AM
ardumont renamed T4229: Properly declare a centralized logging configuration and then reuse from Properly centralize celery tasks logging configuration to Properly declare a centralized logging configuration and then reuse.
May 11 2022, 9:19 AM · Core & foundations
ardumont lowered the priority of T4229: Properly declare a centralized logging configuration and then reuse from High to Normal.
May 11 2022, 9:19 AM · Core & foundations
ardumont triaged T4229: Properly declare a centralized logging configuration and then reuse as High priority.
May 11 2022, 9:18 AM · Core & foundations
ardumont closed D7806: cvs.loader: Decrease log level verbosity to debug.
May 11 2022, 9:06 AM
ardumont committed rDLDCVS3909cb79ae42: cvs.loader: Decrease log level verbosity to debug (authored by ardumont).
cvs.loader: Decrease log level verbosity to debug
May 11 2022, 9:06 AM

May 10 2022

ardumont requested review of D7809: elastic-workers: Let the loader some time to finish gracefully.
May 10 2022, 6:55 PM
ardumont added a revision to T4144: Elastic worker infrastructure: D7809: elastic-workers: Let the loader some time to finish gracefully.
May 10 2022, 6:55 PM · meta-task, System administration, Roadmap 2022
ardumont planned changes to D7805: Allow setting up logging level in core loader.

I'm a bit uncomfortable with adding more ad-hoc logger.setLevel calls deep in SWH code (which then make reasoning about the logging config harder),

May 10 2022, 6:52 PM
ardumont removed a task from D7805: Allow setting up logging level in core loader: T4214: Production : Deploy bzr/bazaar Loader.
May 10 2022, 6:11 PM
ardumont removed a revision from T4214: Production : Deploy bzr/bazaar Loader: D7805: Allow setting up logging level in core loader.
May 10 2022, 6:11 PM · System administration
ardumont updated the summary of D7805: Allow setting up logging level in core loader.
May 10 2022, 6:10 PM
ardumont updated the diff for D7805: Allow setting up logging level in core loader.

Amend commit message (remove related to)

May 10 2022, 6:10 PM
ardumont requested review of D7806: cvs.loader: Decrease log level verbosity to debug.
May 10 2022, 6:07 PM
ardumont requested review of D7805: Allow setting up logging level in core loader.
May 10 2022, 6:06 PM
ardumont added a revision to T4214: Production : Deploy bzr/bazaar Loader: D7805: Allow setting up logging level in core loader.
May 10 2022, 6:03 PM · System administration
ardumont closed D7803: Ensure the loader/lister packages are installed on scheduler node.
May 10 2022, 4:03 PM
ardumont committed rSPSITE5e9a46881ce5: Ensure the loader/lister packages are installed on scheduler node (authored by ardumont).
Ensure the loader/lister packages are installed on scheduler node
May 10 2022, 4:03 PM
ardumont added a project to T4228: scrubber: Investigate the apparent lock (staging): Archive integrity.
May 10 2022, 3:37 PM · Archive integrity, System administration
ardumont added a hashtag to Archive integrity: #archive-integrity.
May 10 2022, 3:37 PM
ardumont set the icon for Archive integrity to Tag.
May 10 2022, 3:36 PM
ardumont created Archive integrity.
May 10 2022, 3:36 PM
ardumont added a project to T4228: scrubber: Investigate the apparent lock (staging): System administration.
May 10 2022, 3:34 PM · Archive integrity, System administration
ardumont updated the task description for T4228: scrubber: Investigate the apparent lock (staging).
May 10 2022, 3:33 PM · Archive integrity, System administration
ardumont triaged T4228: scrubber: Investigate the apparent lock (staging) as Normal priority.
May 10 2022, 3:30 PM · Archive integrity, System administration
ardumont accepted D7764: icinga checks: Activate the prometheus export on e2e tests.
May 10 2022, 3:23 PM
ardumont added a comment to T4214: Production : Deploy bzr/bazaar Loader.

To avoid infinite growing queue message, i also limited the max queue length:

14:48:26 softwareheritage-scheduler@belvedere:5432=> begin; update task_type set max_queue_length=1000 where max_queue_length is null and type like 'load-%';
BEGIN
Time: 5.700 ms
UPDATE 3
Time: 5.069 ms
14:48:37 *softwareheritage-scheduler@belvedere:5432=> commit;
COMMIT
Time: 5.631 ms
May 10 2022, 3:18 PM · System administration
ardumont updated the summary of D7803: Ensure the loader/lister packages are installed on scheduler node.
May 10 2022, 3:13 PM
ardumont updated the diff for D7803: Ensure the loader/lister packages are installed on scheduler node.

Amend commit message

May 10 2022, 3:12 PM
ardumont requested review of D7803: Ensure the loader/lister packages are installed on scheduler node.
May 10 2022, 3:11 PM
ardumont added a revision to T4214: Production : Deploy bzr/bazaar Loader: D7803: Ensure the loader/lister packages are installed on scheduler node.
May 10 2022, 3:11 PM · System administration
ardumont closed T3945: staging: List and ingest bzr origins, analyse and fix paper cuts, a subtask of T3610: Bazaar/Breezy loader, as Resolved.
May 10 2022, 3:04 PM · BZR loader
ardumont closed T3945: staging: List and ingest bzr origins, analyse and fix paper cuts as Resolved.

Considering this done, closing.

May 10 2022, 3:04 PM · BZR loader
ardumont accepted D7801: docs: Describe metadata formats more precisely, and mention github and gitlab's.
May 10 2022, 2:54 PM
ardumont added a comment to T4214: Production : Deploy bzr/bazaar Loader.

New triggered listing for sourceforge is doing its job:

May 10 2022, 2:54 PM · System administration
ardumont moved T4214: Production : Deploy bzr/bazaar Loader from in-progress to deployed/landed/monitoring on the System administration board.
May 10 2022, 2:44 PM · System administration
ardumont updated the task description for T4214: Production : Deploy bzr/bazaar Loader.
May 10 2022, 2:44 PM · System administration
ardumont updated the task description for T4214: Production : Deploy bzr/bazaar Loader.
May 10 2022, 2:40 PM · System administration
ardumont updated the task description for T4214: Production : Deploy bzr/bazaar Loader.
May 10 2022, 2:27 PM · System administration
ardumont committed rSPSITEb605038f0f6e: worker::instances: Explicit only staging services (authored by ardumont).
worker::instances: Explicit only staging services
May 10 2022, 2:23 PM
ardumont closed D7802: Deploy bzr loader in production.
May 10 2022, 2:23 PM
ardumont committed rSPSITE831dff13410b: Deploy bzr loader in production (authored by ardumont).
Deploy bzr loader in production
May 10 2022, 2:23 PM
ardumont updated the task description for T4214: Production : Deploy bzr/bazaar Loader.
May 10 2022, 2:20 PM · System administration
ardumont requested review of D7802: Deploy bzr loader in production.
May 10 2022, 2:17 PM
ardumont added a revision to T4214: Production : Deploy bzr/bazaar Loader: D7802: Deploy bzr loader in production.
May 10 2022, 2:17 PM · System administration
ardumont updated the task description for T4214: Production : Deploy bzr/bazaar Loader.
May 10 2022, 11:46 AM · System administration
ardumont changed the status of T4214: Production : Deploy bzr/bazaar Loader, a subtask of T3610: Bazaar/Breezy loader, from Open to Work in Progress.
May 10 2022, 11:34 AM · BZR loader
ardumont changed the status of T4214: Production : Deploy bzr/bazaar Loader from Open to Work in Progress.
May 10 2022, 11:34 AM · System administration
ardumont updated the task description for T4214: Production : Deploy bzr/bazaar Loader.
May 10 2022, 11:33 AM · System administration