Page MenuHomeSoftware Heritage

olasd (Nicolas Dandrimont)
UserAdministrator

Projects (9)

User Details

User Since
Sep 7 2015, 3:25 PM (449 w, 5 d)
Roles
Administrator

Recent Activity

Feb 20 2024

olasd committed R261:bb1c42b9e70b: Pin gitignore entries to the toplevel directory (authored by olasd).
Pin gitignore entries to the toplevel directory
Feb 20 2024, 10:31 AM
olasd committed R261:098a1b04ec87: Add objstorage-ansible repository (authored by olasd).
Add objstorage-ansible repository
Feb 20 2024, 10:31 AM

Jan 8 2023

olasd added a member for Paper authors: gitlab-migration.
Jan 8 2023, 11:21 PM
olasd added a member for Web masters: gitlab-migration.
Jan 8 2023, 4:52 PM
olasd added a member for Developers: gitlab-migration.
Jan 8 2023, 4:50 PM

Jan 7 2023

olasd committed rSPSITE6598bef89f65: Phabricator: disable POST on edition/creation endpoints (authored by olasd).
Phabricator: disable POST on edition/creation endpoints
Jan 7 2023, 12:24 AM
olasd committed rSPSITE68a8e962132c: Phabricator: disable POST on /edit/ endpoints (authored by olasd).
Phabricator: disable POST on /edit/ endpoints
Jan 7 2023, 12:15 AM
olasd committed rSPSITE9c69febdf3ae: Move repository references to new GitLab hierarchy (authored by olasd).
Move repository references to new GitLab hierarchy
Jan 7 2023, 12:15 AM
olasd committed rSPSITEe7ad6af551a8: kafka: min_compaction_lag_ms makes kafka run out of memory (authored by olasd).
kafka: min_compaction_lag_ms makes kafka run out of memory
Jan 7 2023, 12:15 AM
olasd committed rSPSITE7989b6c0b46a: kafka: tune number of threads (authored by olasd).
kafka: tune number of threads
Jan 7 2023, 12:15 AM
olasd committed rSPSITE327f35afa79e: kafka: use a less aggressive dedupe buffer size (authored by olasd).
kafka: use a less aggressive dedupe buffer size
Jan 7 2023, 12:15 AM
olasd committed rSPSITEf558c90486f1: Set kafka log cleaner settings in the broker config (authored by olasd).
Set kafka log cleaner settings in the broker config
Jan 7 2023, 12:15 AM
olasd committed rSPSITE3f1d216ec0da: Disable automatic topic creation on kafka brokers (authored by olasd).
Disable automatic topic creation on kafka brokers
Jan 7 2023, 12:15 AM
olasd committed rSPSITE128b31c7b884: met: clean up unused settings (authored by olasd).
met: clean up unused settings
Jan 7 2023, 12:15 AM
olasd committed rSPSITE506929c1bb6d: Drop postgresql server profiles from swh_provenance (authored by olasd).
Drop postgresql server profiles from swh_provenance
Jan 7 2023, 12:15 AM
olasd committed rSPSITE41adb333a0d0: pve-exporter: disable config collector (authored by olasd).
pve-exporter: disable config collector
Jan 7 2023, 12:15 AM
olasd committed rSPSITE1288af2e8aa8: Set host variables for mucem (authored by olasd).
Set host variables for mucem
Jan 7 2023, 12:15 AM
olasd committed rSPSITE6214025b6c71: Add mucem to the hypervisor profile (authored by olasd).
Add mucem to the hypervisor profile
Jan 7 2023, 12:14 AM
olasd committed rSPSITE613019ecbb63: Push profile::zfs::docker zpool configuration to hiera (authored by olasd).
Push profile::zfs::docker zpool configuration to hiera
Jan 7 2023, 12:14 AM
olasd committed rSPSITE18ac3e185761: Add rancher nodes with apt config (authored by olasd).
Add rancher nodes with apt config
Jan 7 2023, 12:14 AM
olasd committed rSPSITEdb12c44fe87b: Set MTU to 1500 for WAN interfaces on kafka brokers (authored by olasd).
Set MTU to 1500 for WAN interfaces on kafka brokers
Jan 7 2023, 12:14 AM
olasd committed rSPSITEef0f4352abf8: vault: force smtp through localhost:25 (authored by olasd).
vault: force smtp through localhost:25
Jan 7 2023, 12:14 AM
olasd committed rSPSITE471341055764: Add a kubenfs group for kubernetes nfs mounts (authored by olasd).
Add a kubenfs group for kubernetes nfs mounts
Jan 7 2023, 12:14 AM
olasd committed rSPSITEa27c053c2155: Make counters fetch their historical data from thanos (authored by olasd).
Make counters fetch their historical data from thanos
Jan 7 2023, 12:14 AM
olasd committed rSPSITE011274768c55: Kafka upgrade: make the log message format version 3.3 (authored by olasd).
Kafka upgrade: make the log message format version 3.3
Jan 7 2023, 12:14 AM
olasd committed rSPSITE277b7cc70be7: Make sure only the main kafka process is SIGTERMed (authored by olasd).
Make sure only the main kafka process is SIGTERMed
Jan 7 2023, 12:14 AM
olasd committed rSPSITE3e38fe9007fb: Drop the kafka log rotation crontabs (authored by olasd).
Drop the kafka log rotation crontabs
Jan 7 2023, 12:14 AM
olasd committed rSPSITEb9f5a8ff359d: Bump kafka inter broker protocol version to 3.3 (authored by olasd).
Bump kafka inter broker protocol version to 3.3
Jan 7 2023, 12:14 AM
olasd committed rSPSITEc1a8b9190508: Bump all kafka brokers to 3.3.1 (authored by olasd).
Bump all kafka brokers to 3.3.1
Jan 7 2023, 12:14 AM
olasd committed rSPSITE2e2c7470e7ba: Fetch kafka from the main apache dlcdn (authored by olasd).
Fetch kafka from the main apache dlcdn
Jan 7 2023, 12:14 AM
olasd committed rSPSITEa1b5ff8a41b5: Bump kafka to 3.3.1 on kafka1 (authored by olasd).
Bump kafka to 3.3.1 on kafka1
Jan 7 2023, 12:14 AM
olasd committed rSPSITE7e8b5e9925a8: kafka: Update jmx exporter metric mapping (authored by olasd).
kafka: Update jmx exporter metric mapping
Jan 7 2023, 12:14 AM
olasd committed rSPSITE646a29a6891f: Bump jmx exporter to 0.17.2 (authored by olasd).
Bump jmx exporter to 0.17.2
Jan 7 2023, 12:14 AM
olasd committed rSPSITE58b45154473c: Ensure java is installed before installing kafka (authored by olasd).
Ensure java is installed before installing kafka
Jan 7 2023, 12:14 AM
olasd committed rSPSITEcb603442ede2: graph: Ensure the shm-mounted files are copied properly (authored by olasd).
graph: Ensure the shm-mounted files are copied properly
Jan 7 2023, 12:14 AM
olasd committed rSPSITE4ed51ac6ccbf: Bump kafka to 3.3.1 in staging (authored by olasd).
Bump kafka to 3.3.1 in staging
Jan 7 2023, 12:14 AM
olasd committed rSPSITEd25d42d02b24: Move the weekly planning bot on Friday afternoon (authored by olasd).
Move the weekly planning bot on Friday afternoon
Jan 7 2023, 12:14 AM
olasd committed rSPSITE64107664f02c: Drop monthly snapshot on dali (authored by olasd).
Drop monthly snapshot on dali
Jan 7 2023, 12:14 AM
olasd committed rSPSITEa0331f841baa: Adapt the weekly plan script to work on fridays too (authored by olasd).
Adapt the weekly plan script to work on fridays too
Jan 7 2023, 12:14 AM
olasd committed rSPSITEb4822612382f: Enable borgmatic backups for the migration.internal.swh.org host (authored by olasd).
Enable borgmatic backups for the migration.internal.swh.org host
Jan 7 2023, 12:14 AM

Jan 6 2023

olasd committed R261:2964611239c2: Move repository references to new GitLab hierarchy (authored by olasd).
Move repository references to new GitLab hierarchy
Jan 6 2023, 5:17 PM

Jan 5 2023

olasd committed R263:697caf670363: Ensure more git pushes to gitlab are successful (authored by olasd).
Ensure more git pushes to gitlab are successful
Jan 5 2023, 6:46 PM
olasd committed R263:8ed4dab6d903: Fix inverted logic on recreation of gitlab checkouts (authored by olasd).
Fix inverted logic on recreation of gitlab checkouts
Jan 5 2023, 6:46 PM

Dec 21 2022

olasd committed R263:b89bb2406250: Add variable to create issue labels in default group (authored by olasd).
Add variable to create issue labels in default group
Dec 21 2022, 12:20 PM
olasd committed R263:507cce6048a1: Parallelize some phabricator export operations (authored by olasd).
Parallelize some phabricator export operations
Dec 21 2022, 12:20 PM
olasd committed R263:f05f13490941: Use task view policy to make it restricted (authored by olasd).
Use task view policy to make it restricted
Dec 21 2022, 12:20 PM
olasd set the icon for Has docker image declaration to Tag.
Dec 21 2022, 11:33 AM

Dec 20 2022

olasd accepted D8960: tox: Use upstream release of jenkins-job-builder.

Great, thanks!

Dec 20 2022, 6:05 PM
olasd added a comment to D8947: Document statsd metrics and link to dashboards.

I like the idea (I'm a bit worried about the list becoming stale really fast, but...)

Dec 20 2022, 6:04 PM
olasd accepted D8959: github: Export statsd metrics about API requests and token usage.

This looks sensible. When this is deployed, we'll need to monitor how the label set grows over time, of course!

Dec 20 2022, 6:01 PM
olasd updated the diff for D8964: Add a backfiller cli command.

Apply @vlorentz's comments

Dec 20 2022, 5:27 PM
olasd closed T4066: Migrate sysadmin projects to GitLab as Resolved.

Sysadmin projects have been migrated on October 18

Dec 20 2022, 4:01 PM · GitLab migration, Roadmap 2020
olasd closed T4066: Migrate sysadmin projects to GitLab, a subtask of T2225: Migrate to GitLab, as Resolved.
Dec 20 2022, 4:01 PM · meta-task, Roadmap 2022, GitLab migration, Roadmap 2020
olasd closed T4701: GitHub mirror of swh-graph is not updated as Resolved.

We've dropped the pristine-tar branch which was causing a bunch of issues (and wasn't being used anyway). The github mirroring from phabricator should now work automatically.

Dec 20 2022, 4:00 PM · System administration, Compressed graph service

Dec 16 2022

olasd committed R263:3aa286ab5f4d: Make sure assignees are properly recorded (authored by olasd).
Make sure assignees are properly recorded
Dec 16 2022, 7:15 PM
olasd committed R263:84e007afa934: Only remove GitLab merge request branches once (authored by olasd).
Only remove GitLab merge request branches once
Dec 16 2022, 7:15 PM
olasd committed R263:f6717b8a6e53: Add a variable to avoid writing out completed mappings (authored by olasd).
Add a variable to avoid writing out completed mappings
Dec 16 2022, 7:15 PM
olasd committed R263:8960a8991ba9: Don't require all users to be admins to migrate objects (authored by olasd).
Don't require all users to be admins to migrate objects
Dec 16 2022, 7:15 PM
olasd committed R263:64b3d844bab0: Move local checkouts to a separate directory (authored by olasd).
Move local checkouts to a separate directory
Dec 16 2022, 7:15 PM
olasd committed R263:8b703a51a020: Don't pull namespace ids out of the mapping (authored by olasd).
Don't pull namespace ids out of the mapping
Dec 16 2022, 7:15 PM
olasd committed R263:484079022aa9: Add more expected retries to gitlab API requests (authored by olasd).
Add more expected retries to gitlab API requests
Dec 16 2022, 7:15 PM
olasd committed R263:ce6c9aa774fa: Assign tickets that have multiple repos to the migration project (authored by olasd).
Assign tickets that have multiple repos to the migration project
Dec 16 2022, 7:15 PM
olasd committed R263:18d1285f700e: gitlab: support nil users in `create-user` (authored by olasd).
gitlab: support nil users in `create-user`
Dec 16 2022, 7:15 PM
olasd committed R263:8fd2bc29b070: GitLab un-approvals do not generate an event (authored by olasd).
GitLab un-approvals do not generate an event
Dec 16 2022, 7:15 PM
olasd committed R263:47096ce64394: gitlab: only tweak approvals generated by the migration user (authored by olasd).
gitlab: only tweak approvals generated by the migration user
Dec 16 2022, 7:15 PM
olasd committed R263:c26be2646412: phabricator: Ensure null branchnames are handled properly (authored by olasd).
phabricator: Ensure null branchnames are handled properly
Dec 16 2022, 7:15 PM
olasd committed R263:af2d02a1d363: Add newline after mapping errors (authored by olasd).
Add newline after mapping errors
Dec 16 2022, 7:15 PM
olasd committed R263:33f030cdd3ce: Merge duplicate actions when converting a diff to core (authored by olasd).
Merge duplicate actions when converting a diff to core
Dec 16 2022, 7:15 PM
olasd committed R263:f46bd1f3626b: Merge changes between phabricator and gitlab when creating local checkout (authored by olasd).
Merge changes between phabricator and gitlab when creating local checkout
Dec 16 2022, 7:15 PM
olasd committed R263:b2b6d8b7b0b2: Always use mapped-item-iids for ticket fetches (authored by olasd).
Always use mapped-item-iids for ticket fetches
Dec 16 2022, 7:15 PM
olasd committed R263:7410d52ccf89: Always force-push to gitlab (authored by olasd).
Always force-push to gitlab
Dec 16 2022, 7:15 PM
olasd committed R263:eb3886dbf3b8: Properly merge temporary filenames for link paths (authored by olasd).
Properly merge temporary filenames for link paths
Dec 16 2022, 7:15 PM

Dec 14 2022

olasd accepted D8877: Fix incorrect error messages when failing to connect.
Dec 14 2022, 2:46 PM

Dec 12 2022

olasd added a comment to D8952: Add missing __init__.py so find_packages keep finding sql modules.

Thanks for looking at this (fwiw, this warning is pretty innocuous for now, and is not what prevents the Debian build; the issue seems to be in pytest).

Dec 12 2022, 2:45 PM

Dec 9 2022

olasd accepted D8940: cli.add_forge_now: Open `register-lister` with sensible defaults.

I like it, thanks!

Dec 9 2022, 2:54 PM

Dec 8 2022

olasd added inline comments to D8940: cli.add_forge_now: Open `register-lister` with sensible defaults.
Dec 8 2022, 5:28 PM
olasd added inline comments to D8940: cli.add_forge_now: Open `register-lister` with sensible defaults.
Dec 8 2022, 11:38 AM
olasd added inline comments to D8940: cli.add_forge_now: Open `register-lister` with sensible defaults.
Dec 8 2022, 11:31 AM
olasd added a comment to D8940: cli.add_forge_now: Open `register-lister` with sensible defaults.

For --staging, we just want to create a single oneshot full listing task.

Why bother with a full listing?

Since we now got the means to limit the listing, i'd use it. That's a faster feedback
loop and without stressing twice the upstream forge (one round for staging and another
for production at a relatively small intervals of time).

Maybe I need to configure the default to a more sensible defaults though. Like 3 pages
with 10 results (so that we can see that the pagination works too). Currently it's a tad
small (1 page of 15 or something).

Dec 8 2022, 11:28 AM

Dec 7 2022

olasd added a comment to D8940: cli.add_forge_now: Open `register-lister` with sensible defaults.

So, I don't think we want to have to remember if there's full or incremental versions of the lister.

Dec 7 2022, 6:10 PM
olasd accepted D8936: cli.add_forge_now: Open `schedule-first-visits` with sensible defaults.

A couple of suggestions inline, but lgtm otherwise, thanks!

Dec 7 2022, 4:38 PM
olasd accepted D8931: Prevent incorrect warning from being printed to output files.

Why not just touch all the files?

Dec 7 2022, 1:37 PM
olasd accepted D8933: task add: Ensure task type provided exist and raise otherwise.
Dec 7 2022, 12:56 PM

Dec 6 2022

olasd accepted D8922: send-to-celery: Adapt to schedule from lister name & instance_name.

Great, thanks!

Dec 6 2022, 5:53 PM
olasd added a comment to D8922: send-to-celery: Adapt to schedule from lister name & instance_name.

Either way, the uuid argument in the cli endpoint should go away!

Dec 6 2022, 4:13 PM
olasd added a comment to D8922: send-to-celery: Adapt to schedule from lister name & instance_name.

Does it seem like we're going to use these arguments in another caller of grab_next_visits?

That's a good question. I think not.

Dec 6 2022, 4:10 PM
olasd added a comment to D8922: send-to-celery: Adapt to schedule from lister name & instance_name.

So, instead of adding more stuff to the grab_next_visits signature, I would have suggested just calling lister_get in swh/scheduler/cli/origin.py to get the lister uuid, and replacing the --lister-uuid CLI argument with --lister-name and --lister-instance-name (so, only changing the CLI function).

Dec 6 2022, 4:05 PM
olasd added a comment to D8922: send-to-celery: Adapt to schedule from lister name & instance_name.

As I mentioned on IRC, I think we should do that "join" directly in swh scheduler origin send-to-celery (and have it error out if the lister name/instance provided don't match an existing lister); using the uuid was an "easy hack" to extend the API of grab_next_visits, but using lister name/instance in the CLI interface makes more sense.

Dec 6 2022, 12:37 PM
olasd closed D8921: Hook up recently introduced options to all listers.
Dec 6 2022, 10:47 AM
olasd committed rDLSe785e67315c6: Hook up recently introduced options to all listers (authored by olasd).
Hook up recently introduced options to all listers
Dec 6 2022, 10:47 AM
olasd closed D8918: gitlab: allow ignoring projects with certain path prefixes.
Dec 6 2022, 10:47 AM
olasd committed rDLS5ea79ee3e0de: gitlab: allow ignoring projects with certain path prefixes (authored by olasd).
gitlab: allow ignoring projects with certain path prefixes
Dec 6 2022, 10:47 AM

Dec 5 2022

olasd added a comment to D8906: [WIP] :Make numeric cursors a bit more opaque to avoid cursor hacks.

So, overall, what I'm uncomfortable with is decoding incoming cursors without validating that we have output and generated them ourselves. They're currently untrusted user input, that we're decoding with a very simple algorithm, and it'd be easy enough for anyone to fiddle with them (even with the prefix/suffix addition).

Dec 5 2022, 5:55 PM
olasd added a comment to D8918: gitlab: allow ignoring projects with certain path prefixes.

Could you add this check?

for prefix in ignored_project_prefixes:
    if not prefix.endswith("/"):
        raise ValueError("Prefixes must end with /")

Otherwise we are going to forget to add slashes at the end, and it will match too many namespaces.

Dec 5 2022, 4:46 PM
olasd requested review of D8921: Hook up recently introduced options to all listers.
Dec 5 2022, 4:44 PM
olasd requested review of D8918: gitlab: allow ignoring projects with certain path prefixes.
Dec 5 2022, 3:45 PM
olasd added a revision to T4680: GitLab lister: allow ignoring origins contained in given namespace prefixes: D8918: gitlab: allow ignoring projects with certain path prefixes.
Dec 5 2022, 3:40 PM · Easy hack, Lister
olasd closed D8916: Add a flag to not enable origins listed by a lister.
Dec 5 2022, 3:25 PM
olasd committed rDLS64267f8f50a8: Add a flag to not enable origins listed by a lister (authored by olasd).
Add a flag to not enable origins listed by a lister
Dec 5 2022, 3:25 PM
olasd closed D8915: Add built-in page and origin count limit to listers.
Dec 5 2022, 3:25 PM