Page MenuHomeSoftware Heritage

olasd (Nicolas Dandrimont)Administrator
User

Projects (7)

User Details

User Since
Sep 7 2015, 3:25 PM (180 w, 4 d)
Roles
Administrator

Recent Activity

Yesterday

olasd added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

Grafana postgresql overview: https://grafana.softwareheritage.org/d/PEKz-Ygiz/postgresql-server-overview

Thu, Feb 21, 11:29 AM · Sprint 2018 12, System administration

Wed, Feb 20

olasd accepted D1161: deposit: Use explicit configuration.
Wed, Feb 20, 6:00 PM · SWORD deposit, Puppet recipes
olasd accepted D1162: rpc_server: Set the configuration file for the gunicorn service.

(god I hate how noisy the puppet coding style makes diffs)

Wed, Feb 20, 5:31 PM
olasd accepted D1158: manifest: Latest refactoring/cleanup .
Wed, Feb 20, 4:59 PM · Puppet recipes
olasd accepted D1163: Fix crash of SchedulerBackend.search_tasks when no argument is given..
Wed, Feb 20, 4:58 PM
olasd accepted D1160: swh-deposit: Adapt swh.deposit.wsgi starting point.
Wed, Feb 20, 4:33 PM · SWORD deposit, Development environment
olasd accepted D1159: settings.prod: Use SWH_CONFIG_FILENAME to load & check swh config.
Wed, Feb 20, 4:33 PM · SWORD deposit
olasd committed rSPMW41123572f90b: Add SVG rendering support to mediawiki (authored by olasd).
Add SVG rendering support to mediawiki
Wed, Feb 20, 4:10 PM
olasd requested changes to D1159: settings.prod: Use SWH_CONFIG_FILENAME to load & check swh config.
Wed, Feb 20, 3:52 PM · SWORD deposit
olasd added a comment to D1159: settings.prod: Use SWH_CONFIG_FILENAME to load & check swh config.

From what I can tell, in your make_app_from_configfile function, the conf object is never used, only checked. That feels very brittle to me and I think the checks should happen in the swh.deposit.settings.production module directly.

Wed, Feb 20, 3:52 PM · SWORD deposit
olasd updated subscribers of T1534: PostgreSQL replication issues between prado and somerset.

After some more stewing and discussion with @zack, we'll be going for the "upgrade to pg 11 and restart replication from scratch" route;

Wed, Feb 20, 2:32 PM · System administration, Archive content
olasd accepted D1155: worker/instance: Refactor worker systemd initialization.

Looks fine.

Wed, Feb 20, 8:55 AM
olasd accepted D1154: Refactor: synchronize workers' service names with configuration yaml key.

At that point I guess the storage_archiver worker should just be called archiver (the name dates back from the point the archiver was still in the storage namespace), but as it's not in use currently I don't think it's really worth bothering.

Wed, Feb 20, 8:51 AM · Puppet recipes
olasd accepted D1153: worker: Remove no longer used worker/<service>.ini config.

Looks fine to me considering ::profile::swh::deploy::worker::base is properly setup to clean up the /etc/softwareheritage/worker directory (as by itself, removing a resource definition does not remove the resource).

Wed, Feb 20, 8:45 AM · Puppet recipes

Tue, Feb 19

olasd committed rSPSITE43a79303d450: Add a static icinga check for softwareheritage.org (no www) (authored by olasd).
Add a static icinga check for softwareheritage.org (no www)
Tue, Feb 19, 8:15 PM
olasd updated subscribers of T1534: PostgreSQL replication issues between prado and somerset.

After reading some mailing list posts discussing the error message, and discussion with @ftigeot:

Tue, Feb 19, 6:22 PM · System administration, Archive content
olasd added a comment to T1534: PostgreSQL replication issues between prado and somerset.

Logs on primary:

2019-02-19 14:27:44 UTC [15973]: [1-1] user=postgres,db=softwareheritage LOG:  starting logical decoding for slot "pgl_softwareheritage_prado_somerset"
2019-02-19 14:27:44 UTC [15973]: [2-1] user=postgres,db=softwareheritage DETAIL:  streaming transactions committing after 18607/9189A578, reading WAL from 18607/9189A578
2019-02-19 14:27:44 UTC [15973]: [3-1] user=postgres,db=softwareheritage ERROR:  record with incorrect prev-link 5403A/2E2F1829 at 18607/9189A578 
2019-02-19 14:27:44 UTC [15973]: [4-1] user=postgres,db=softwareheritage LOG:  could not receive data from client: Connection reset by peer
Tue, Feb 19, 3:29 PM · System administration, Archive content
olasd renamed T1534: PostgreSQL replication issues between prado and somerset from PostgreSQL replication issues between prado and beaubourg to PostgreSQL replication issues between prado and somerset.
Tue, Feb 19, 3:22 PM · System administration, Archive content
olasd added a comment to T1526: Install a new VPN endpoint at Rocquencourt.

I'm slightly reordering what you wrote here, sorry!

Tue, Feb 19, 2:58 PM · System administration
olasd updated subscribers of T1534: PostgreSQL replication issues between prado and somerset.
Tue, Feb 19, 2:26 PM · System administration, Archive content
olasd added a parent task for T1461: Add loader-related metrics to swh-loader-core: T1535: Deploy prometheus-statsd-exporter to gather per-worker metrics.
Tue, Feb 19, 2:25 PM · Core Loader, Metrics/monitoring
olasd added a parent task for T1460: Add task related metrics to swh-scheduler: T1535: Deploy prometheus-statsd-exporter to gather per-worker metrics.
Tue, Feb 19, 2:25 PM · Scheduling utilities, Metrics/monitoring
olasd added subtasks for T1535: Deploy prometheus-statsd-exporter to gather per-worker metrics: T1460: Add task related metrics to swh-scheduler, T1461: Add loader-related metrics to swh-loader-core.
Tue, Feb 19, 2:25 PM · System administration, Metrics/monitoring
olasd triaged T1535: Deploy prometheus-statsd-exporter to gather per-worker metrics as High priority.
Tue, Feb 19, 2:25 PM · System administration, Metrics/monitoring
olasd triaged T1534: PostgreSQL replication issues between prado and somerset as High priority.
Tue, Feb 19, 2:10 PM · System administration, Archive content
olasd added a comment to T1491: Connection problems with latest celery (4.2.1; docker env, production).

I've pushed an updated kombu to our repository.

Tue, Feb 19, 9:55 AM · Docker environment, Scheduling utilities

Mon, Feb 18

olasd committed rCENV0e7a670eeb3a: No need to add the file to mrtrust (authored by olasd).
No need to add the file to mrtrust
Mon, Feb 18, 7:40 PM
olasd closed D1107: No need to add the file to mrtrust.
Mon, Feb 18, 7:40 PM
olasd added a comment to T1526: Install a new VPN endpoint at Rocquencourt.

Thanks for recording this task; I'll use the opportunity to document the reasoning behind the current internal networking setup, to try and make sure nothing is forgotten before migrating it.

Mon, Feb 18, 7:30 PM · System administration
olasd added a comment to D398: [WIP] "packing" object storage design documentation.

Some related work : https://fosdem.org/2019/schedule/event/small_files_in_swift_cluster/

Mon, Feb 18, 11:48 AM

Fri, Feb 8

olasd added a comment to T1522: external contributions: permission issue with staging area when `arc diff`.

I've documented the extra hoops people need to jump through for git push to work on https://wiki.softwareheritage.org/index.php?title=Code_review_in_Phabricator&type=revision&diff=964&oldid=899

Fri, Feb 8, 1:15 PM · Phabricator forge
olasd added a comment to T1522: external contributions: permission issue with staging area when `arc diff`.

I've moved the repositories that need it to have their staging area set to https.

Fri, Feb 8, 12:24 PM · Phabricator forge
olasd created D1107: No need to add the file to mrtrust.
Fri, Feb 8, 12:21 PM
olasd added a comment to T1522: external contributions: permission issue with staging area when `arc diff`.

use https for the staging area configuration (which will allow people to push it even if nothing is configured, by just typing their password)

That'd mean also to change that uri to be read/write, it's read-only for now.

Fri, Feb 8, 11:54 AM · Phabricator forge
olasd added a comment to T1522: external contributions: permission issue with staging area when `arc diff`.

One more possibility is to document that people need to setup an ssh key on their account to be able to push their changes.

Fri, Feb 8, 10:18 AM · Phabricator forge
olasd committed rCJSWHe9532ad741fd: Support building dependency packages (authored by olasd).
Support building dependency packages
Fri, Feb 8, 10:03 AM
olasd closed D1104: Support building dependency packages.
Fri, Feb 8, 10:03 AM
olasd committed rCJSWH64662eee8a18: Trigger automatic backport job at the end of a successful build (authored by olasd).
Trigger automatic backport job at the end of a successful build
Fri, Feb 8, 10:03 AM
olasd committed rCJSWH4942769a4537: Use a job group to avoid repeating the same jobs in twenty different files (authored by olasd).
Use a job group to avoid repeating the same jobs in twenty different files
Fri, Feb 8, 10:03 AM
olasd closed D1102: Trigger automatic backport job at the end of a successful build.
Fri, Feb 8, 10:03 AM
olasd closed D1103: Use a job group to avoid repeating the same jobs in twenty different files.
Fri, Feb 8, 10:03 AM
olasd closed D1101: Trigger debian builds on tags rather than on commits.
Fri, Feb 8, 10:03 AM
olasd committed rCJSWHaada5d8a7217: Trigger debian builds on tags rather than on commits (authored by olasd).
Trigger debian builds on tags rather than on commits
Fri, Feb 8, 10:03 AM
olasd committed rCJSWHe804a7cb9890: Clearer label for the packaging stage of the incoming-tag job (authored by olasd).
Clearer label for the packaging stage of the incoming-tag job
Fri, Feb 8, 10:03 AM
olasd closed D1100: Clearer label for the packaging stage of the incoming-tag job.
Fri, Feb 8, 10:03 AM
olasd committed rCJSWH5578d430498f: Move job_exists to an external library (authored by olasd).
Move job_exists to an external library
Fri, Feb 8, 10:03 AM
olasd closed D1099: Move job_exists to an external library.
Fri, Feb 8, 10:03 AM
olasd committed rCJSWHd1adff73efa0: Add documentation for the (current status of) automated debian jobs (authored by olasd).
Add documentation for the (current status of) automated debian jobs
Fri, Feb 8, 10:03 AM
olasd closed D1098: Add documentation for the (current status of) automated debian jobs.
Fri, Feb 8, 10:03 AM

Thu, Feb 7

olasd added a comment to T1520: Numerous dm device failures on louvre.

During the pvmove off of /dev/md3, the root filesystem for uffizi ended up being remounted r/o. I've shut it down, fsck'd it, and booted it back up.

Thu, Feb 7, 9:38 PM · System administration
olasd added a comment to D1099: Move job_exists to an external library.

sounds good

from where does the jobExists function come from though? (is that coming from some jenkins setup for example).

Thu, Feb 7, 8:37 PM
olasd created D1104: Support building dependency packages.
Thu, Feb 7, 7:33 PM
Herald added a reviewer for D1103: Use a job group to avoid repeating the same jobs in twenty different files: Reviewers.
Thu, Feb 7, 7:33 PM
olasd added a child revision for D1101: Trigger debian builds on tags rather than on commits: D1102: Trigger automatic backport job at the end of a successful build.
Thu, Feb 7, 7:32 PM
Herald added a reviewer for D1102: Trigger automatic backport job at the end of a successful build: Reviewers.
Thu, Feb 7, 7:32 PM
olasd removed a child revision for D1097: Fix the DumpMountAndLoadSvnRepository task: D1101: Trigger debian builds on tags rather than on commits.
Thu, Feb 7, 7:32 PM
olasd removed parent revisions for D1101: Trigger debian builds on tags rather than on commits: D1100: Clearer label for the packaging stage of the incoming-tag job, D1097: Fix the DumpMountAndLoadSvnRepository task.
Thu, Feb 7, 7:32 PM
olasd removed a child revision for D1100: Clearer label for the packaging stage of the incoming-tag job: D1101: Trigger debian builds on tags rather than on commits.
Thu, Feb 7, 7:32 PM
olasd added a child revision for D1098: Add documentation for the (current status of) automated debian jobs: D1101: Trigger debian builds on tags rather than on commits.
Thu, Feb 7, 7:31 PM
olasd added parent revisions for D1101: Trigger debian builds on tags rather than on commits: D1100: Clearer label for the packaging stage of the incoming-tag job, D1098: Add documentation for the (current status of) automated debian jobs.
Thu, Feb 7, 7:31 PM
olasd added a child revision for D1100: Clearer label for the packaging stage of the incoming-tag job: D1101: Trigger debian builds on tags rather than on commits.
Thu, Feb 7, 7:31 PM
olasd created D1101: Trigger debian builds on tags rather than on commits.
Thu, Feb 7, 7:31 PM
olasd added a child revision for D1099: Move job_exists to an external library: D1101: Trigger debian builds on tags rather than on commits.
Thu, Feb 7, 7:31 PM
olasd added a child revision for D1097: Fix the DumpMountAndLoadSvnRepository task: D1101: Trigger debian builds on tags rather than on commits.
Thu, Feb 7, 7:31 PM
olasd created D1100: Clearer label for the packaging stage of the incoming-tag job.
Thu, Feb 7, 7:29 PM
Herald added a reviewer for D1099: Move job_exists to an external library: Reviewers.
Thu, Feb 7, 7:28 PM
olasd created D1098: Add documentation for the (current status of) automated debian jobs.
Thu, Feb 7, 7:27 PM
olasd committed rPMK4661f696f94a: Bump package version again (authored by olasd).
Bump package version again
Thu, Feb 7, 7:07 PM
olasd committed rPMKdc3d832baf6e: Fix distribution for upload (authored by olasd).
Fix distribution for upload
Thu, Feb 7, 6:56 PM
olasd committed rPMK4deb724109e5: Bump package version (authored by olasd).
Bump package version
Thu, Feb 7, 6:55 PM
olasd committed rCJLIBf2772702d995: Add stub license information (same as Jenkins upstream) and README (authored by olasd).
Add stub license information (same as Jenkins upstream) and README
Thu, Feb 7, 2:59 PM
olasd committed rCJLIB8e92756eb8e5: Create a stub library for jenkins patterns (authored by olasd).
Create a stub library for jenkins patterns
Thu, Feb 7, 2:54 PM

Wed, Feb 6

olasd committed rDSCHc29d3834978d: Add basic stats to tasks (authored by olasd).
Add basic stats to tasks
Wed, Feb 6, 1:12 PM
olasd closed T1460: Add task related metrics to swh-scheduler, a subtask of T1408: More/better Metrics, as Resolved.
Wed, Feb 6, 1:12 PM · Metrics/monitoring, Sprint 2018 12
olasd closed T1460: Add task related metrics to swh-scheduler as Resolved by committing rDSCHc29d3834978d: Add basic stats to tasks.
Wed, Feb 6, 1:12 PM · Scheduling utilities, Metrics/monitoring
olasd closed D1007: Add basic stats to tasks.
Wed, Feb 6, 1:12 PM
olasd updated the diff for D1007: Add basic stats to tasks.

Fix copyright year

Wed, Feb 6, 1:06 PM
olasd updated the diff for D1007: Add basic stats to tasks.

Rebase

Wed, Feb 6, 1:06 PM

Tue, Feb 5

olasd added a comment to T1514: MemoryError in loader-git.

That's a fairly large repo (as seen with how the content bundles get spread out to limit their size). It looks like it has some large directories (e.g. the .bugs directory looks like it has a lot of entries) so I'm not too surprised.

Tue, Feb 5, 3:40 PM · Git loader

Thu, Jan 24

olasd updated the diff for D1007: Add basic stats to tasks.

Reference T1460 in the commit

Thu, Jan 24, 5:55 PM
Herald added a reviewer for D1007: Add basic stats to tasks: Reviewers.
Thu, Jan 24, 5:52 PM
olasd abandoned D1006: Add a cadvisor instance.
Thu, Jan 24, 5:49 PM
olasd committed rCDFD6df984762351: Add a prometheus instance (authored by olasd).
Add a prometheus instance
Thu, Jan 24, 5:48 PM
olasd closed D1005: Add a statsd-exporter instance.
Thu, Jan 24, 5:48 PM
olasd committed rCDFDa2778431afc1: Add a statsd-exporter instance (authored by olasd).
Add a statsd-exporter instance
Thu, Jan 24, 5:48 PM
olasd closed D1004: Add a prometheus instance.
Thu, Jan 24, 5:48 PM
olasd committed rCDFDb3c973594ab0: nginx: Add error logging at info level on stderr (authored by olasd).
nginx: Add error logging at info level on stderr
Thu, Jan 24, 5:48 PM
olasd committed rCDFD8fa21cc9d1cf: Move nginx to port 5080 (authored by olasd).
Move nginx to port 5080
Thu, Jan 24, 5:48 PM
olasd closed D1003: nginx: Add error logging at info level on stderr.
Thu, Jan 24, 5:48 PM
olasd closed D1002: Move nginx to port 5080.
Thu, Jan 24, 5:48 PM
olasd committed rCDFDb35153426aed: Avoid nginx depending on all the web-facing containers (authored by olasd).
Avoid nginx depending on all the web-facing containers
Thu, Jan 24, 5:48 PM
olasd closed D1001: Avoid nginx depending on all the web-facing containers.
Thu, Jan 24, 5:48 PM
olasd committed rCDFDcc26d77abd4a: Add a Makefile with a check-staged target to please the pre-commit hook (authored by olasd).
Add a Makefile with a check-staged target to please the pre-commit hook
Thu, Jan 24, 5:48 PM
olasd closed D1000: Add a Makefile with a check-staged target to please the pre-commit hook.
Thu, Jan 24, 5:48 PM
olasd updated the diff for D1006: Add a cadvisor instance.

Re-re-rebase

Thu, Jan 24, 5:46 PM
olasd updated the diff for D1005: Add a statsd-exporter instance.

Re-rebase

Thu, Jan 24, 5:46 PM
olasd updated the diff for D1004: Add a prometheus instance.

Rebasé

Thu, Jan 24, 5:46 PM
olasd updated the diff for D1003: nginx: Add error logging at info level on stderr.

Add comment + rebase

Thu, Jan 24, 5:46 PM
olasd updated the diff for D1002: Move nginx to port 5080.

Rebase

Thu, Jan 24, 5:45 PM
olasd updated the diff for D1001: Avoid nginx depending on all the web-facing containers.

Add all the comments + rebase

Thu, Jan 24, 5:45 PM