Page MenuHomeSoftware Heritage

vlorentz (Valentin Lorentz)
User

User Details

User Since
Oct 1 2018, 11:23 AM (147 w, 4 d)

Recent Activity

Yesterday

vlorentz requested review of D6049: Do not exclude falsy git objects from being added..
Fri, Jul 30, 6:14 PM
vlorentz closed D6048: Prevent swh_ql.so from being discovered by sphinx-autodoc.
Fri, Jul 30, 6:12 PM
vlorentz committed rDSEA5093dd70db21: Prevent swh_ql.so from being discovered by sphinx-autodoc (authored by vlorentz).
Prevent swh_ql.so from being discovered by sphinx-autodoc
Fri, Jul 30, 6:12 PM
vlorentz requested review of D6048: Prevent swh_ql.so from being discovered by sphinx-autodoc.
Fri, Jul 30, 5:23 PM
vlorentz closed D6047: Add stream_results_optional.
Fri, Jul 30, 4:44 PM
vlorentz committed rDCORE3337e79642e0: Add stream_results_optional (authored by vlorentz).
Add stream_results_optional
Fri, Jul 30, 4:44 PM
vlorentz updated the diff for D6047: Add stream_results_optional.

oops I didn't mean to commit this yet

Fri, Jul 30, 4:12 PM
vlorentz updated the diff for D6047: Add stream_results_optional.

add test_stream_results_kwarg

Fri, Jul 30, 4:09 PM
vlorentz requested review of D6047: Add stream_results_optional.
Fri, Jul 30, 3:59 PM
vlorentz added a comment to P1110 bad stream_results_optional.

Because, if there is a single yield expression in the function's AST, Python parses the function as a generator, so it can only return generator objects None.

Fri, Jul 30, 3:39 PM
vlorentz edited P1110 bad stream_results_optional.
Fri, Jul 30, 3:33 PM
vlorentz updated the language for P1110 bad stream_results_optional from autodetect to python.
Fri, Jul 30, 3:31 PM
vlorentz created P1110 bad stream_results_optional.
Fri, Jul 30, 3:31 PM
vlorentz created P1109 stream_results_optional.
Fri, Jul 30, 3:21 PM
vlorentz added a project to T3452: Replication lag between the dbs should raise icinga alerts: Monitoring.
Fri, Jul 30, 1:32 PM · Monitoring, System administration
vlorentz closed D6045: converters: Preserve GPG signatures on releases.
Fri, Jul 30, 12:09 PM
vlorentz committed rDLDGc67ab0269ef2: converters: Preserve GPG signatures on releases (authored by vlorentz).
converters: Preserve GPG signatures on releases
Fri, Jul 30, 12:09 PM
vlorentz accepted D6025: translator.py: Translate search query language to ES DSL.
Fri, Jul 30, 12:08 PM
vlorentz added inline comments to D6025: translator.py: Translate search query language to ES DSL.
Fri, Jul 30, 12:08 PM
vlorentz added a comment to D6004: misc/coverage: Revamp archive coverage widget (WIP).

Below is how it looks so far:

Fri, Jul 30, 11:39 AM
vlorentz added a project to T3450: 404 error when visiting a successfully archived repository: Storage manager.
Fri, Jul 30, 11:07 AM · Storage manager, System administration
vlorentz added a comment to T3450: 404 error when visiting a successfully archived repository.

@ardumont noticed the replication was blocked, but our automated monitoring didn't alert us. He unblocked the replication, so your code should appear in the next hours.

Fri, Jul 30, 11:07 AM · Storage manager, System administration
vlorentz removed a revision from T75: Check integrity of Revisions and Releases: D6045: converters: Preserve GPG signatures on releases.
Fri, Jul 30, 10:59 AM · Archive content, Restricted Project
vlorentz edited tasks for D6045: converters: Preserve GPG signatures on releases, added: T3135: Improve integrity of ingested content; removed T75: Check integrity of Revisions and Releases.
Fri, Jul 30, 10:59 AM
vlorentz added a revision to T3135: Improve integrity of ingested content: D6045: converters: Preserve GPG signatures on releases.
Fri, Jul 30, 10:59 AM · Storage manager, Roadmap 2021, meta-task
vlorentz added a revision to T75: Check integrity of Revisions and Releases: D6045: converters: Preserve GPG signatures on releases.
Fri, Jul 30, 10:58 AM · Archive content, Restricted Project
vlorentz added a task to D6045: converters: Preserve GPG signatures on releases: T75: Check integrity of Revisions and Releases.
Fri, Jul 30, 10:58 AM
vlorentz added a comment to T75: Check integrity of Revisions and Releases.

Dulwich 0.19.10 (released in january 2019) changed the way they handle signatures on annotated tags, so we silently drop all signatures since we started using it (probably whenever we upgraded loaders to Buster).

Fri, Jul 30, 10:58 AM · Archive content, Restricted Project
vlorentz requested review of D6045: converters: Preserve GPG signatures on releases.
Fri, Jul 30, 10:57 AM
vlorentz triaged T3451: Convert the refresh-savecodenow-statuses cron to a systemd timer as Low priority.
Fri, Jul 30, 10:25 AM · Web app, System administration
vlorentz added a project to T1510: Have a look at openAPI and decide whether we want to follow these specs: Web app.
Fri, Jul 30, 10:16 AM · Web app
vlorentz added a project to T3282: Add support for "uninterpreted upstream object" in SWH model and storage: Data Model.
Fri, Jul 30, 10:16 AM · Data Model
vlorentz added a project to T3112: Provenance index for the full archive: Provenance database.
Fri, Jul 30, 10:15 AM · Provenance database, Roadmap 2021, meta-task
vlorentz added a project to T2193: Add provenance feature: Provenance database.
Fri, Jul 30, 10:15 AM · Provenance database, Roadmap 2020
vlorentz added a project to T3431: Implement a MongoDB backend for SWH-provenance : Provenance database.
Fri, Jul 30, 10:15 AM · Provenance database
vlorentz created Provenance database.
Fri, Jul 30, 10:14 AM
vlorentz triaged T3450: 404 error when visiting a successfully archived repository as High priority.

This may simply be lag between the loader's and the frontend's databases.

Fri, Jul 30, 10:12 AM · Storage manager, System administration

Thu, Jul 29

vlorentz added inline comments to D6025: translator.py: Translate search query language to ES DSL.
Thu, Jul 29, 9:12 PM
vlorentz triaged T3449: pypi / package loader: handle dangling branches as Normal priority.
Thu, Jul 29, 4:03 PM · Package Loader, Easy hack, PyPI loader
vlorentz moved T2431: Document how to export the graph edge dataset from sys-admin (docs/sysadm) to developers (docs/devel/) on the Documentation board.
Thu, Jul 29, 3:54 PM · Documentation, Graph service, Datasets
vlorentz moved T2431: Document how to export the graph edge dataset from archive-users (docs/user-guides/) to sys-admin (docs/sysadm) on the Documentation board.
Thu, Jul 29, 3:54 PM · Documentation, Graph service, Datasets
vlorentz added a comment to T2431: Document how to export the graph edge dataset.

It is now somewhat documented here: https://forge.softwareheritage.org/source/swh-environment/browse/master/docker/services/swh-graph/entrypoint.sh

Thu, Jul 29, 3:54 PM · Documentation, Graph service, Datasets
vlorentz closed T1936: integrate swh-graph into the docker environment as Resolved.
Thu, Jul 29, 3:51 PM · Docker environment, Graph service
vlorentz accepted D6044: scanner: translate CoreSWHID instances before the request to the backend.
Thu, Jul 29, 3:50 PM
vlorentz added a comment to D6027: swh-scanner: add 'auto' option as default policy.

Thanks!

Thu, Jul 29, 1:42 PM
vlorentz added inline comments to D6025: translator.py: Translate search query language to ES DSL.
Thu, Jul 29, 12:49 PM
vlorentz added inline comments to D6025: translator.py: Translate search query language to ES DSL.
Thu, Jul 29, 12:48 PM
vlorentz closed D6030: server: Define make_app_from_configfile so it can be ran by gunicorn.
Thu, Jul 29, 11:53 AM
vlorentz committed rDGRPH90d1109c5103: server: Define make_app_from_configfile so it can be ran by gunicorn (authored by vlorentz).
server: Define make_app_from_configfile so it can be ran by gunicorn
Thu, Jul 29, 11:53 AM
vlorentz accepted D6036: Use versioned ExtIDs in loader mercurial implementation.
Thu, Jul 29, 11:26 AM
vlorentz created P1106 hg extid version test working.
Thu, Jul 29, 11:11 AM
vlorentz added a comment to D6027: swh-scanner: add 'auto' option as default policy.

docstring for each policy

Thu, Jul 29, 10:52 AM
vlorentz created P1105 hg extid version test.
Thu, Jul 29, 10:48 AM
vlorentz closed D6042: graph: don't run export+compression unless needed, and add documentation.
Thu, Jul 29, 10:15 AM
vlorentz committed rDENV122eb0d250e6: graph: don't run export+compression unless needed, and add documentation (authored by vlorentz).
graph: don't run export+compression unless needed, and add documentation
Thu, Jul 29, 10:15 AM
vlorentz committed rDSEAc34011b7c762: setup.py: Install swh_ql.so alongside .py files, so it can be seen by… (authored by vlorentz).
setup.py: Install swh_ql.so alongside .py files, so it can be seen by…
Thu, Jul 29, 9:47 AM
vlorentz closed D6039: setup.py: Install swh_ql.so alongside .py files, so it can be seen by pkg_resources.
Thu, Jul 29, 9:47 AM

Wed, Jul 28

vlorentz added inline comments to D6040: test: Make the child process wait longer so it gets actually killed.
Wed, Jul 28, 5:56 PM
vlorentz added inline comments to D6040: test: Make the child process wait longer so it gets actually killed.
Wed, Jul 28, 5:55 PM
vlorentz accepted D6040: test: Make the child process wait longer so it gets actually killed.
Wed, Jul 28, 5:44 PM
vlorentz added inline comments to D6042: graph: don't run export+compression unless needed, and add documentation.
Wed, Jul 28, 4:53 PM
vlorentz added inline comments to D6040: test: Make the child process wait longer so it gets actually killed.
Wed, Jul 28, 4:50 PM
vlorentz added a task to D6042: graph: don't run export+compression unless needed, and add documentation: T1936: integrate swh-graph into the docker environment.
Wed, Jul 28, 4:44 PM
vlorentz added a task to D6038: journalprocessor: Fix deserialize_message raising EOFError on the last message of each assignment: T1936: integrate swh-graph into the docker environment.
Wed, Jul 28, 4:44 PM
vlorentz added revisions to T1936: integrate swh-graph into the docker environment: D6042: graph: don't run export+compression unless needed, and add documentation, D6038: journalprocessor: Fix deserialize_message raising EOFError on the last message of each assignment.
Wed, Jul 28, 4:44 PM · Docker environment, Graph service
vlorentz retitled D6029: Add swh-graph to the docker environment from Start adding swh-graph to the docker environment to Add swh-graph to the docker environment.
Wed, Jul 28, 4:41 PM
vlorentz requested review of D6042: graph: don't run export+compression unless needed, and add documentation.
Wed, Jul 28, 4:41 PM
vlorentz added inline comments to D6029: Add swh-graph to the docker environment.
Wed, Jul 28, 4:19 PM
vlorentz closed D6029: Add swh-graph to the docker environment.
Wed, Jul 28, 4:19 PM
vlorentz committed rDENVf4e9375c7a72: Add swh-graph to the docker environment (authored by vlorentz).
Add swh-graph to the docker environment
Wed, Jul 28, 4:19 PM
vlorentz updated the diff for D6029: Add swh-graph to the docker environment.

it works, finally

Wed, Jul 28, 4:18 PM
vlorentz closed D6038: journalprocessor: Fix deserialize_message raising EOFError on the last message of each assignment.
Wed, Jul 28, 3:15 PM
vlorentz committed rDDATASET202bcdf29b90: journalprocessor: Fix deserialize_message raising EOFError on the last message… (authored by vlorentz).
journalprocessor: Fix deserialize_message raising EOFError on the last message…
Wed, Jul 28, 3:15 PM
vlorentz requested review of D6039: setup.py: Install swh_ql.so alongside .py files, so it can be seen by pkg_resources.
Wed, Jul 28, 3:14 PM
vlorentz committed rDSEA5f786a5c5ed7: tox.ini: pass YARN env variable, needed if YARN isn't in the PATH. (authored by vlorentz).
tox.ini: pass YARN env variable, needed if YARN isn't in the PATH.
Wed, Jul 28, 3:04 PM
vlorentz requested review of D6038: journalprocessor: Fix deserialize_message raising EOFError on the last message of each assignment.
Wed, Jul 28, 12:32 PM
vlorentz added a comment to D6025: translator.py: Translate search query language to ES DSL.

_traverse should raise errors instead of silently ignoring unexpected nodes.

Wed, Jul 28, 10:59 AM
vlorentz closed D6035: Add CMAK container.
Wed, Jul 28, 10:25 AM
vlorentz closed D6034: Expose zookeeper and kafka ports so they are accessible outside docker.
Wed, Jul 28, 10:25 AM
vlorentz committed rDENV25276a5c429d: Expose zookeeper and kafka ports so they are accessible outside docker (authored by vlorentz).
Expose zookeeper and kafka ports so they are accessible outside docker
Wed, Jul 28, 10:25 AM
vlorentz committed rDENV49b90da2ef7e: Add CMAK container (authored by vlorentz).
Add CMAK container
Wed, Jul 28, 10:25 AM
vlorentz updated the diff for D6035: Add CMAK container.

rebase

Wed, Jul 28, 10:25 AM
vlorentz updated the diff for D6034: Expose zookeeper and kafka ports so they are accessible outside docker.

rebase

Wed, Jul 28, 10:25 AM
vlorentz added inline comments to D6030: server: Define make_app_from_configfile so it can be ran by gunicorn.
Wed, Jul 28, 10:24 AM
vlorentz added a comment to D6029: Add swh-graph to the docker environment.

Do you restart regularly the graph service (through some docker-compose command) so the commands gets executed to fill it?

Wed, Jul 28, 10:21 AM
vlorentz committed rDENV5d2bc5477e4c: Update scheduler documentation to match the new scheduler workflow (w/ origins) (authored by vlorentz).
Update scheduler documentation to match the new scheduler workflow (w/ origins)
Wed, Jul 28, 10:21 AM
vlorentz closed D6033: Update scheduler documentation to match the new scheduler workflow (w/ origins).
Wed, Jul 28, 10:21 AM
vlorentz updated the diff for D6035: Add CMAK container.

rewrite forms as well

Wed, Jul 28, 10:16 AM
vlorentz requested review of D6035: Add CMAK container.
Wed, Jul 28, 10:07 AM
vlorentz requested review of D6034: Expose zookeeper and kafka ports so they are accessible outside docker.
Wed, Jul 28, 9:38 AM
vlorentz requested review of D6033: Update scheduler documentation to match the new scheduler workflow (w/ origins).
Wed, Jul 28, 9:38 AM
vlorentz abandoned D6032: [wip] Start adding swh-graph to the docker environment.

oops, wrong commits

Wed, Jul 28, 9:37 AM
vlorentz requested review of D6032: [wip] Start adding swh-graph to the docker environment.
Wed, Jul 28, 9:33 AM
vlorentz updated the diff for D6029: Add swh-graph to the docker environment.

partition all the topics

Wed, Jul 28, 9:32 AM

Tue, Jul 27

vlorentz added inline comments to D6031: Add a quick start section in the documentation and simplify the configuration file loading mechanism in the cli.
Tue, Jul 27, 6:28 PM
vlorentz added a comment to D6031: Add a quick start section in the documentation and simplify the configuration file loading mechanism in the cli.

thanks

Tue, Jul 27, 6:26 PM
vlorentz added a comment to T1936: integrate swh-graph into the docker environment.

I decided to make the swh-graph container create the compressed graph itself before starting. That's the easiest way to use it AND to implement it IMO.

Tue, Jul 27, 5:39 PM · Docker environment, Graph service
vlorentz requested review of D6030: server: Define make_app_from_configfile so it can be ran by gunicorn.
Tue, Jul 27, 5:38 PM
vlorentz claimed T1936: integrate swh-graph into the docker environment.
Tue, Jul 27, 5:37 PM · Docker environment, Graph service
vlorentz added a task to D6028: journalprocessor: Fix freeze on empty offset ranges.: T1936: integrate swh-graph into the docker environment.
Tue, Jul 27, 5:37 PM