Page MenuHomeSoftware Heritage

olasd (Nicolas Dandrimont)
UserAdministrator

Projects (9)

User Details

User Since
Sep 7 2015, 3:25 PM (369 w, 3 d)
Roles
Administrator

Recent Activity

Today

olasd added a comment to D8597: Add method 'origin_get' endpoint.

I'm not too fond of the List[str] -> List[Dict[str, Any]] signature, with the "not found" requested documents just disappearing from the result list.

Thu, Oct 6, 5:22 PM
olasd added inline comments to D8635: gitlab-staging: Add the configuration to install the gitlab operator.
Thu, Oct 6, 3:16 PM
olasd added inline comments to D8635: gitlab-staging: Add the configuration to install the gitlab operator.
Thu, Oct 6, 12:33 PM

Mon, Oct 3

olasd added a comment to T4534: Evaluate MetalLB as inbound loadbalancer.

FWIW, we didn't manage to replicate the timeout issue when manually killing and/or bringing down the network on the node currently responding to the MetalLB IP address... Every time, the failover happened within 10 seconds.

Mon, Oct 3, 4:11 PM · System administration
olasd committed rSPSITE4a28fe53994c: Add thanos store for archive-production rancher cluster (authored by olasd).
Add thanos store for archive-production rancher cluster
Mon, Oct 3, 3:54 PM
olasd accepted D8593: Reorganize the code.

I think this makes sense, thanks!

Mon, Oct 3, 11:17 AM
olasd added a comment to T4527: scrubber: keep a state file for postgresql datastores.

I think I'll implement it as a table in the scrubber's DB, this will make it easier to query the current status of scrubbing and add it to the Grafana dashboard

Mon, Oct 3, 11:09 AM · Datastore Scrubber
olasd accepted D8591: Adapt postgresql backend to swh.core.db >= 2.0.
Mon, Oct 3, 10:57 AM
olasd accepted D8590: Upgrade to swh.graph 2.0.
Mon, Oct 3, 10:55 AM

Fri, Sep 30

olasd accepted D8564: github: Ignore archive_url and issues_url; use custom codemeta:issueTracker.

Ah, d'oh, it is (but your test fixture lies :-P)

Fri, Sep 30, 4:32 PM
olasd added a comment to D8564: github: Ignore archive_url and issues_url; use custom codemeta:issueTracker.

Generating the issueTracker should probably be gated on the has_issues field (which is false, for this specific repository).

Fri, Sep 30, 4:31 PM
olasd accepted D8574: postgresql: Remove merge join with origin_visit in origin_visit_get_latest.

That seems sensible, thanks.

Fri, Sep 30, 4:30 PM
olasd added a comment to D8582: Allow more checksum computations in Content model.

It was initially to just compute that new hash at the same time as the standard ones we store but that's getting out of hand.

then use MultiHash directly in the Content loader, instead of Content.from_data

Fri, Sep 30, 10:47 AM
olasd added a comment to D8582: Allow more checksum computations in Content model.

Why are the changes to the model object needed, instead of just hashing the file directly?

You mean using MultiHash directly. Yes, that feels more and more like the right way to do it.

It was initially to just compute that new hash at the same time as the standard ones we store but that's getting out of hand.

Fri, Sep 30, 10:47 AM
olasd requested changes to D8582: Allow more checksum computations in Content model.

I don't agree with the idea of adding additional hashes, that will not be stored, to the Content model object. Model objects should map 1:1 with what is stored in the archive.

Fri, Sep 30, 10:42 AM

Thu, Sep 29

olasd accepted D8580: Apply 'max_matching_nodes' restriction after 'return_types' filter.

The logic seems sound, but I'm proposing an (untested) edit to make the code clearer, WDYT?

Thu, Sep 29, 5:55 PM
olasd added inline comments to D8581: Add Content Loader to ingest raw content file.
Thu, Sep 29, 5:45 PM
olasd added a comment to D8539: Add random directory sampling policy.

Fine, i've one comment i'd like others to have a look at though [1] regarding where
that new discovery (interface) code should go. It feels currently a bit off to me that this code
is in loader-core. Loaders are not the sole archive consumers (scanner, webapp, cli, indexer, cooker, ...).

[1] https://forge.softwareheritage.org/D8539?id=30916#inline-60774

@vlorentz @douardda @olasd @anlambert ^

Thu, Sep 29, 4:03 PM
olasd committed rCJSWH32b1ad3332b3: Make all docs jobs only clone a single branch for all repos (authored by olasd).
Make all docs jobs only clone a single branch for all repos
Thu, Sep 29, 3:25 PM

Wed, Sep 28

olasd created P1465 requirement hack.
Wed, Sep 28, 6:03 PM
olasd created P1464 Command-Line Input.
Wed, Sep 28, 6:03 PM
olasd committed rDGRPH8549673d02e4: Properly set the port from CLI when running swh graph grpc-serve (authored by olasd).
Properly set the port from CLI when running swh graph grpc-serve
Wed, Sep 28, 3:55 PM
olasd accepted D8561: Add debug logging to grpc-serve initialization.

The hang really is in the /java/ process, so that's not going to help me much, but it won't hurt either.

Wed, Sep 28, 1:38 PM

Tue, Sep 27

olasd accepted D8468: github and gitea: Use html_url as @id and clone_url as codeRepository.

This seems reasonable in terms of ""ontology design"".

Tue, Sep 27, 4:59 PM
olasd accepted D8460: Add Gitea metadata mapping.

At a glance, this seems okay.

Tue, Sep 27, 4:55 PM
olasd accepted D8447: Add max_matching_nodes parameter to /leaves.
Tue, Sep 27, 3:10 PM
olasd committed rDSTO1281ee7b7eeb: Add date-based index to origin_visit (authored by olasd).
Add date-based index to origin_visit
Tue, Sep 27, 3:04 PM
olasd closed D8466: Add date-based index to origin_visit.
Tue, Sep 27, 3:03 PM
olasd committed rDSTO0786ab174902: SQL upgrade scripts don't need to bump dbversion anymore (authored by olasd).
SQL upgrade scripts don't need to bump dbversion anymore
Tue, Sep 27, 3:03 PM
olasd updated the diff for D8466: Add date-based index to origin_visit.

Rebase

Tue, Sep 27, 2:44 PM
olasd accepted D8547: Exclude protobuf 4.21.*.

fun with inconsistent side effects during imports

Tue, Sep 27, 2:41 PM
olasd added inline comments to D8546: Add more origins to savecodenow::e2e..
Tue, Sep 27, 2:40 PM
olasd accepted D8543: Handle errors raised by fromisoformat..
Tue, Sep 27, 2:31 PM
olasd accepted D8546: Add more origins to savecodenow::e2e..
Tue, Sep 27, 2:28 PM
olasd accepted D8548: developer-setup: Add instructions to install latest yarn version.

Thanks.

Tue, Sep 27, 2:25 PM

Mon, Sep 26

olasd accepted D8527: model: avoid another extra creation of Model object.
Mon, Sep 26, 1:35 PM
olasd accepted D8510: from_disk: only build a model object once.
Mon, Sep 26, 1:31 PM
olasd added inline comments to D8521: Use a Merkle discovery algorithm with archives.
Mon, Sep 26, 12:10 PM
olasd accepted D8532: Set rancher-node-production worker to high load profile.

(indeed, collected resources)

Mon, Sep 26, 12:07 PM

Wed, Sep 21

olasd committed R260:99f6518e769c: storage-replayer: Disable sentry logging events (authored by olasd).
storage-replayer: Disable sentry logging events
Wed, Sep 21, 11:15 AM

Thu, Sep 15

olasd committed rSPSITE4790ce76ffde: Make swh.graph http listen on all ip addresses (authored by olasd).
Make swh.graph http listen on all ip addresses
Thu, Sep 15, 3:05 PM

Wed, Sep 14

olasd committed rSPSITEb56179cd700c: swh::deploy::graph: revert unexport of icinga checks (authored by olasd).
swh::deploy::graph: revert unexport of icinga checks
Wed, Sep 14, 7:44 PM
olasd committed rSPSITEce5de80db7fa: Clarify swh-graph icinga checks between http and grpc (authored by olasd).
Clarify swh-graph icinga checks between http and grpc
Wed, Sep 14, 7:40 PM
olasd committed rSPSITE3e45c4317622: Split deployment of swh.graph grpc and http (authored by olasd).
Split deployment of swh.graph grpc and http
Wed, Sep 14, 7:40 PM
olasd committed rCDFJ3784c57af69b: Update jenkins agents 4.13 (authored by olasd).
Update jenkins agents 4.13
Wed, Sep 14, 6:53 PM
olasd added a member for Staff: lunar.
Wed, Sep 14, 5:27 PM
olasd committed rDGRPH7432bb9f5d87: swh graph grpc-serve doesn't take a host parameter (authored by olasd).
swh graph grpc-serve doesn't take a host parameter
Wed, Sep 14, 4:24 PM
olasd committed rSPSITE892f68fe1e95: Prometheus's 1m is one minute, not one month (authored by olasd).
Prometheus's 1m is one minute, not one month
Wed, Sep 14, 3:03 PM
olasd committed rSPSITE7447a5122848: Use the proper option to get perfdata out of prometheus metrics (authored by olasd).
Use the proper option to get perfdata out of prometheus metrics
Wed, Sep 14, 2:14 PM
olasd committed rSPSITE07b107e99d77: Overhaul variables for prometheus metric icinga check (authored by olasd).
Overhaul variables for prometheus metric icinga check
Wed, Sep 14, 2:09 PM
olasd committed rSPSITEcf64f9b3843b: Use proper name for the check_prometheus_metric_query var (authored by olasd).
Use proper name for the check_prometheus_metric_query var
Wed, Sep 14, 1:54 PM
olasd committed rSPSITE781960c26298: Bump local prometheus retention down to 1 month (authored by olasd).
Bump local prometheus retention down to 1 month
Wed, Sep 14, 1:44 PM
olasd closed D8471: Summary: Bump local prometheus retention down to 1 month.
Wed, Sep 14, 1:44 PM
olasd committed rSPSITE78153d4a666f: Add all arguments of the check_prometheus_metric icinga check (authored by olasd).
Add all arguments of the check_prometheus_metric icinga check
Wed, Sep 14, 1:44 PM
olasd committed rSPSITE009710953dfe: Add icinga2 warning for thanos sidecar shipper (authored by olasd).
Add icinga2 warning for thanos sidecar shipper
Wed, Sep 14, 1:44 PM
olasd committed rSPSITE193e0af8e8c4: Rename check_prometheus_metric without shell suffix (authored by olasd).
Rename check_prometheus_metric without shell suffix
Wed, Sep 14, 1:44 PM
olasd committed rSPSITE73e4d70f8817: Use the generic check_prometheus_metric icinga check for belvedere lag (authored by olasd).
Use the generic check_prometheus_metric icinga check for belvedere lag
Wed, Sep 14, 1:44 PM
olasd closed D8470: Improve icinga2 prometheus metric checks.
Wed, Sep 14, 1:44 PM
olasd committed rSPSITEb4810292860d: Don't hardcode FQDN of prometheus server (authored by olasd).
Don't hardcode FQDN of prometheus server
Wed, Sep 14, 1:44 PM
olasd added inline comments to D8470: Improve icinga2 prometheus metric checks.
Wed, Sep 14, 1:35 PM

Tue, Sep 13

olasd requested review of D8471: Summary: Bump local prometheus retention down to 1 month.
Tue, Sep 13, 7:27 PM
olasd requested review of D8470: Improve icinga2 prometheus metric checks.
Tue, Sep 13, 7:26 PM
olasd accepted D8464: Add a 'swh graph grpc-serve' command.
Tue, Sep 13, 4:29 PM
olasd requested review of D8466: Add date-based index to origin_visit.
Tue, Sep 13, 4:26 PM
olasd requested changes to D8464: Add a 'swh graph grpc-serve' command.
Tue, Sep 13, 3:44 PM
olasd accepted D8448: Reorganize configuration of the http server building.

I think this makes sense.

Tue, Sep 13, 2:02 PM
olasd created P1451 (An Untitled Masterwork).
Tue, Sep 13, 1:29 PM
olasd accepted D8420: auth: Improve login management and configuration.

Very nice!

Tue, Sep 13, 12:00 PM
olasd added a reverting change for rSPSITE77b70356949d: Switch moma over to the main db for the somerset/beaubourg migration: rSPSITE558efbe435fb: Revert "Switch moma over to the main db for the somerset/beaubourg migration".
Tue, Sep 13, 11:00 AM
olasd committed rSPSITE558efbe435fb: Revert "Switch moma over to the main db for the somerset/beaubourg migration" (authored by olasd).
Revert "Switch moma over to the main db for the somerset/beaubourg migration"
Tue, Sep 13, 11:00 AM
olasd committed rSPSITE58b7480744bc: Drop legacy thanos gateway definitions (authored by olasd).
Drop legacy thanos gateway definitions
Tue, Sep 13, 11:00 AM

Mon, Sep 12

olasd updated subscribers of T4459: Deploy swh-indexer > v2.6 on staging then production.

There's a few issues with the configuration of these indexer clients:

Mon, Sep 12, 8:10 PM · Indexer, System administration
olasd added a member for Speakers: vlorentz.
Mon, Sep 12, 4:36 PM
olasd added a comment to D8420: auth: Improve login management and configuration.

Is there anything preventing swh-auth from using next= instead of next_path=?

Mon, Sep 12, 4:20 PM
olasd added a comment to D8420: auth: Improve login management and configuration.
In D8420#219363, @olasd wrote:

While you're refactoring this, is there a reason for not using the default REDIRECT_FIELD_NAME from django.contrib.auth ("next") for the login/logout views, instead of "next_path"? This would save us from having to override it on every user_passes_test call.

Oh I missed that django setting, let me check.

That's much better indeed, thanks ! Diff updated.

Mon, Sep 12, 3:44 PM
olasd added a comment to D8420: auth: Improve login management and configuration.

While you're refactoring this, is there a reason for not using the default REDIRECT_FIELD_NAME from django.contrib.auth ("next") for the login/logout views, instead of "next_path"? This would save us from having to override it on every user_passes_test call.

Mon, Sep 12, 2:46 PM
olasd accepted D8444: tests/conftest: Remove sentry fixtures.

Thanks! I guess this needs a version bump on swh.core in requirements-swh.txt

Mon, Sep 12, 2:40 PM
olasd committed rSPSITEe1c7c8dbda5e: Ensure upgrades of thanos restart all relevant services (authored by olasd).
Ensure upgrades of thanos restart all relevant services
Mon, Sep 12, 12:51 PM
olasd closed D8442: Upgrade thanos to 0.28.0.
Mon, Sep 12, 12:51 PM
olasd closed D8443: Bump max postgres connections on db1.staging: 200 is too low.
Mon, Sep 12, 12:51 PM
olasd committed rSPSITEeec2835f4c16: Bump max postgres connections on db1.staging: 200 is too low (authored by olasd).
Bump max postgres connections on db1.staging: 200 is too low
Mon, Sep 12, 12:51 PM
olasd committed rSPSITEa2233d54120e: Upgrade thanos to 0.28.0 (authored by olasd).
Upgrade thanos to 0.28.0
Mon, Sep 12, 12:51 PM
olasd closed D8440: Split thanos objstore configuration management to separate class.
Mon, Sep 12, 12:51 PM
olasd closed D8441: Add configuration for thanos compact.
Mon, Sep 12, 12:51 PM
olasd committed rSPSITE76213b1ab932: Split thanos objstore configuration management to separate class (authored by olasd).
Split thanos objstore configuration management to separate class
Mon, Sep 12, 12:51 PM
olasd committed rSPSITEca4bc487479d: Add configuration for thanos compact (authored by olasd).
Add configuration for thanos compact
Mon, Sep 12, 12:51 PM
olasd closed D8439: Rename thanos-gateway service to thanos-store to match upstream.
Mon, Sep 12, 12:51 PM
olasd committed rSPSITE6246518e45fd: Rename thanos-gateway service to thanos-store to match upstream (authored by olasd).
Rename thanos-gateway service to thanos-store to match upstream
Mon, Sep 12, 12:51 PM

Fri, Sep 9

olasd added a comment to D8443: Bump max postgres connections on db1.staging: 200 is too low.
18:43:28         ⤷ ╡ icinga PROBLEM: service check_systemd on scrubber0.internal.staging.swh.network is CRITICAL: SYSTEMD CRITICAL - swh-scrubber-checker-postgres@primary-snapshot-0.service: failed, swh-scrubber-checker-postgres@primary-snapshot-1.service: failed
Fri, Sep 9, 6:54 PM
olasd requested review of D8443: Bump max postgres connections on db1.staging: 200 is too low.
Fri, Sep 9, 6:53 PM
olasd triaged T4527: scrubber: keep a state file for postgresql datastores as High priority.
Fri, Sep 9, 6:49 PM · Datastore Scrubber
olasd requested review of D8442: Upgrade thanos to 0.28.0.
Fri, Sep 9, 6:35 PM
olasd requested review of D8441: Add configuration for thanos compact.
Fri, Sep 9, 6:31 PM
olasd requested review of D8440: Split thanos objstore configuration management to separate class.
Fri, Sep 9, 6:31 PM
olasd requested review of D8439: Rename thanos-gateway service to thanos-store to match upstream.
Fri, Sep 9, 6:30 PM
olasd accepted D8434: pytest_plugin: Add sentry_events fixture from swh-indexer.

Indeed this seems useful; I've made a couple of naming nitpicks, but I don't care very strongly about them.

Fri, Sep 9, 6:28 PM
olasd closed D8433: Add scratch disk to thanos host (for compaction).
Fri, Sep 9, 6:20 PM
olasd committed rSPREd97fe4b50a47: Add scratch disk to thanos host (for compaction) (authored by olasd).
Add scratch disk to thanos host (for compaction)
Fri, Sep 9, 6:20 PM
olasd updated the diff for D8433: Add scratch disk to thanos host (for compaction).

Rebase

Fri, Sep 9, 6:19 PM
olasd added a project to T4526: New model for the origin layer: Provenance database.
Fri, Sep 9, 6:07 PM · Provenance database