Page MenuHomeSoftware Heritage

vlorentz (Valentin Lorentz)
User

User Details

User Since
Oct 1 2018, 11:23 AM (133 w, 3 d)

Recent Activity

Today

vlorentz added a comment to D5550: tox: Add a sphinx environment to check sane doc build.

Another random idea: what about -e ../swh-docs#egg=swh.docs (or a variant that works, I didn't try it) instead of `-e git+https://forge.softwareheritage.org/source/swh-docs#egg=swh.docs ?

Fri, Apr 23, 12:58 AM
vlorentz accepted D5580: tests: Fix failure after recent changes in swh-auth.
Fri, Apr 23, 12:56 AM
vlorentz accepted D5579: tests: Fix failures after recent changes in swh-auth.
Fri, Apr 23, 12:56 AM

Yesterday

vlorentz accepted D5571: sphinx/conf.py: Add standalone package doc configuration for tox build.
Thu, Apr 22, 8:47 PM
vlorentz committed rDDOCa8ca608e4f53: Include class docstrings in the autodocumentation. (authored by vlorentz).
Include class docstrings in the autodocumentation.
Thu, Apr 22, 8:37 PM
vlorentz closed D5576: Remove 'journal_type' argument from the CLI.
Thu, Apr 22, 8:21 PM
vlorentz committed rDCNT85cac362ca67: Remove 'journal_type' argument from the CLI (authored by vlorentz).
Remove 'journal_type' argument from the CLI
Thu, Apr 22, 8:21 PM
vlorentz updated the diff for D5576: Remove 'journal_type' argument from the CLI.

typo

Thu, Apr 22, 8:21 PM
vlorentz added a comment to D5576: Remove 'journal_type' argument from the CLI.

I suppose the message.value() is returning a copy of the content

Thu, Apr 22, 8:20 PM
vlorentz added a comment to D5576: Remove 'journal_type' argument from the CLI.

It also makes the module aware of how the objects are serialized on kafka, which looks quite low level.

Thu, Apr 22, 8:18 PM
vlorentz added inline comments to D5577: adds a filter by node type as a query argument.
Thu, Apr 22, 5:57 PM
vlorentz requested changes to D5577: adds a filter by node type as a query argument.
Thu, Apr 22, 5:52 PM
vlorentz added a comment to D5577: adds a filter by node type as a query argument.

Looks good overall, but I'm not familiar with this piece of code.

Thu, Apr 22, 5:52 PM
vlorentz closed D5574: cassandra: Deduplicate table names.
Thu, Apr 22, 5:22 PM
vlorentz committed rDSTOeb8c147a542f: cassandra: Deduplicate table names (authored by vlorentz).
cassandra: Deduplicate table names
Thu, Apr 22, 5:22 PM
vlorentz updated the diff for D5574: cassandra: Deduplicate table names.

fix typo

Thu, Apr 22, 5:22 PM
vlorentz updated the diff for D5576: Remove 'journal_type' argument from the CLI.

found the bug, it was variable shadowing in handle_messages.

Thu, Apr 22, 4:46 PM
vlorentz requested review of D5576: Remove 'journal_type' argument from the CLI.
Thu, Apr 22, 4:37 PM
vlorentz added a comment to D5572: Implement the jounal client counting an internal property of an object.

D5576

Thu, Apr 22, 4:32 PM
vlorentz accepted D5572: Implement the jounal client counting an internal property of an object.

I'll send a diff with my idea, it will be clearer

Thu, Apr 22, 3:36 PM
vlorentz added a comment to D5572: Implement the jounal client counting an internal property of an object.

It's for performance considerations only, for most of the counters, counting the keys is enough as it's the unique identifier in kafka.
The KeyOrientedJournalClient[1] is bypassing the object deserialization when a message is received, so a more classical client is needed for this specific Person case.

Thu, Apr 22, 3:29 PM
vlorentz added a comment to D5572: Implement the jounal client counting an internal property of an object.

So we will need two journal clients, one that reads only keys, and one that reads only messages? Why not a single one to do both?

Thu, Apr 22, 2:43 PM
vlorentz triaged T3282: Add support for "uninterpreted upstream object" in SWH model and storage as Low priority.
Thu, Apr 22, 2:33 PM
vlorentz updated subscribers of T3281: Create a list of known test/buggy repos and use them in loader/storage tests.
Thu, Apr 22, 2:27 PM
vlorentz requested changes to D5572: Implement the jounal client counting an internal property of an object.

Still missing documentation of the journal_type CLI argument AFAICT

Thu, Apr 22, 1:37 PM
vlorentz added inline comments to D5572: Implement the jounal client counting an internal property of an object.
Thu, Apr 22, 12:48 PM
vlorentz added inline comments to D5572: Implement the jounal client counting an internal property of an object.
Thu, Apr 22, 12:27 PM
vlorentz requested review of D5574: cassandra: Deduplicate table names.
Thu, Apr 22, 12:27 PM
vlorentz added inline comments to D5572: Implement the jounal client counting an internal property of an object.
Thu, Apr 22, 11:28 AM
vlorentz requested changes to D5571: sphinx/conf.py: Add standalone package doc configuration for tox build.

(requesting changes, so you don't accidentally merge it if you missed the notif, but I'm fine on principle)

Thu, Apr 22, 11:21 AM
vlorentz added a comment to D5571: sphinx/conf.py: Add standalone package doc configuration for tox build.

thx!

Thu, Apr 22, 11:21 AM
vlorentz requested changes to D5564: common/identifiers: Fix content SWHID with anchor revision browse URL.
Thu, Apr 22, 11:12 AM

Tue, Apr 20

vlorentz added inline comments to D5565: sphinx/conf.py: Merge swh-web sphinx config into root conf.py file.
Tue, Apr 20, 9:00 PM
vlorentz added inline comments to D5565: sphinx/conf.py: Merge swh-web sphinx config into root conf.py file.
Tue, Apr 20, 6:34 PM
vlorentz added a comment to D5564: common/identifiers: Fix content SWHID with anchor revision browse URL.

Why use hypothesis for this? According to Jenkins, one of the two important branches are not covered.

Tue, Apr 20, 6:31 PM
vlorentz accepted D5563: common/highlightjs: Turn some global variables into functions.
Tue, Apr 20, 6:29 PM
vlorentz accepted D5566: docs: Remove doc_config module and its use.
Tue, Apr 20, 6:28 PM
vlorentz accepted D5567: sphin/conf.py: Improve django settings handling.
Tue, Apr 20, 6:27 PM
vlorentz closed D5561: Simplify priority computation in tests + improve exhaustivity.
Tue, Apr 20, 1:01 PM
vlorentz committed rDSCHf33f74346f33: Simplify priority computation in tests + improve exhaustivity (authored by vlorentz).
Simplify priority computation in tests + improve exhaustivity
Tue, Apr 20, 1:01 PM
vlorentz updated the diff for D5561: Simplify priority computation in tests + improve exhaustivity.

rebase

Tue, Apr 20, 1:01 PM
vlorentz requested review of D5561: Simplify priority computation in tests + improve exhaustivity.
Tue, Apr 20, 12:06 PM
vlorentz accepted D5552: scheduler: Clean up priority/ratio task dead code.
Tue, Apr 20, 11:22 AM
vlorentz closed D5560: Parse task_ids before calling set_status_tasks..
Tue, Apr 20, 11:20 AM
vlorentz committed rDSCH4e06bcd71e37: Parse task_ids before calling set_status_tasks. (authored by vlorentz).
Parse task_ids before calling set_status_tasks.
Tue, Apr 20, 11:20 AM
vlorentz updated the diff for D5560: Parse task_ids before calling set_status_tasks..

update copyright

Tue, Apr 20, 11:20 AM
vlorentz accepted D5558: Update the general architecture diagram.

Looks good, thanks!

Tue, Apr 20, 11:04 AM
vlorentz requested review of D5560: Parse task_ids before calling set_status_tasks..
Tue, Apr 20, 11:02 AM
vlorentz added a comment to D5558: Update the general architecture diagram.

Tue, Apr 20, 11:02 AM
vlorentz added a comment to T2823: Write tests for swh/journal/writer/inmemory.py.

If we replaced the code with just this:

Tue, Apr 20, 10:12 AM · Easy hack, Journal
vlorentz added a comment to D5554: origin_save: Add docstring and some test scenarios.

Can't you use exhaustive comparisons instead of len(sors) == 1 + comparing items?

Tue, Apr 20, 10:05 AM
vlorentz requested changes to D5501: add an anti-Dos limit for edges traversed as a query parameter.

In case you missed my message on IRC/Matrix:

Tue, Apr 20, 9:53 AM

Mon, Apr 19

vlorentz added a comment to T2265: Building the documentation should not show any warning..

We need more loud warnings of doc build issues (eg. broken references) first.

Mon, Apr 19, 4:17 PM · Easy hack, Documentation
vlorentz added a comment to T2602: Investigate how to upgrade the schema of the Cassandra storage.

I just discussed the multiplexer-based migration process I described above with ardumont/olasd/vsellier.

Mon, Apr 19, 3:22 PM · Storage manager
vlorentz added a comment to T2602: Investigate how to upgrade the schema of the Cassandra storage.

Doesn't this deserve a state-of-the-art kind of thing?

Mon, Apr 19, 3:22 PM · Storage manager
vlorentz added inline comments to D5552: scheduler: Clean up priority/ratio task dead code.
Mon, Apr 19, 1:52 PM
vlorentz lowered the priority of T3273: Use "fork" relationships to speed-up initial load of large repositories from Normal to Low.
Mon, Apr 19, 1:50 PM · Extrinsic metadata, Core Loader
vlorentz triaged T3273: Use "fork" relationships to speed-up initial load of large repositories as Normal priority.
Mon, Apr 19, 1:49 PM · Extrinsic metadata, Core Loader
vlorentz renamed T2724: fuse: populate history/ incrementally to avoid temporary hangs on huge repos from populate history/ incrementally to avoid temporary hangs on huge repos to fuse: populate history/ incrementally to avoid temporary hangs on huge repos.
Mon, Apr 19, 1:43 PM · Software Heritage filesystem
vlorentz accepted D5484: assets/webapp-utils: Add SWHID validation in search inputs.
Mon, Apr 19, 1:00 PM
vlorentz committed rDDOCcbe30affa057: Fix missing double-colon (authored by vlorentz).
Fix missing double-colon
Mon, Apr 19, 12:48 PM
vlorentz requested changes to D5548: exporters/edges: Fix sphinx warning.

it should be:

Mon, Apr 19, 12:47 PM
vlorentz claimed T2471: NPM package angular-ts-manage fails to be properly loaded.
Mon, Apr 19, 12:43 PM · Core Loader
vlorentz removed a parent task for T3089: Remove the 'metadata' column of the 'revision' table: T2471: NPM package angular-ts-manage fails to be properly loaded.
Mon, Apr 19, 12:43 PM · Storage manager, Archive content
vlorentz added a parent task for T3138: Make loaders stop writing in the 'metadata' column of the 'revision' table: T2471: NPM package angular-ts-manage fails to be properly loaded.
Mon, Apr 19, 12:43 PM · Core Loader
vlorentz edited subtasks for T2471: NPM package angular-ts-manage fails to be properly loaded, added: T3138: Make loaders stop writing in the 'metadata' column of the 'revision' table; removed: T3089: Remove the 'metadata' column of the 'revision' table.
Mon, Apr 19, 12:43 PM · Core Loader
vlorentz requested changes to D5551: Fix swh-scanner for python > 3.7.
Mon, Apr 19, 11:00 AM
vlorentz added a comment to D5551: Fix swh-scanner for python > 3.7.

From what I understand, the addition of importlib-metadata to requirements.txt is needed to fix python 3.7, not python 3.8.

Mon, Apr 19, 11:00 AM
vlorentz added a comment to D5550: tox: Add a sphinx environment to check sane doc build.

that's a lot of code to add to every package :/

Mon, Apr 19, 10:20 AM

Fri, Apr 16

vlorentz added a revision to T3258: Build documentation on every diff: D5550: tox: Add a sphinx environment to check sane doc build.
Fri, Apr 16, 7:27 PM · Continuous Integration, Development environment, Documentation
vlorentz added a task to D5550: tox: Add a sphinx environment to check sane doc build: T3258: Build documentation on every diff.
Fri, Apr 16, 7:27 PM
vlorentz triaged T3258: Build documentation on every diff as Normal priority.
Fri, Apr 16, 5:33 PM · Continuous Integration, Development environment, Documentation
vlorentz closed T2265: Building the documentation should not show any warning. as Resolved.
Fri, Apr 16, 5:26 PM · Easy hack, Documentation
vlorentz closed D5545: phabricator: Emphasize the 'arc diff --update' message should not be the default one..
Fri, Apr 16, 5:24 PM
vlorentz committed rDDOCbfeba3d7dee9: phabricator: Emphasize the 'arc diff --update' message should not be the… (authored by vlorentz).
phabricator: Emphasize the 'arc diff --update' message should not be the…
Fri, Apr 16, 5:24 PM
vlorentz updated the diff for D5545: phabricator: Emphasize the 'arc diff --update' message should not be the default one..

rebase

Fri, Apr 16, 5:24 PM
vlorentz updated the diff for D5545: phabricator: Emphasize the 'arc diff --update' message should not be the default one..

reword in a clearer way

Fri, Apr 16, 2:15 PM
vlorentz requested review of D5545: phabricator: Emphasize the 'arc diff --update' message should not be the default one..
Fri, Apr 16, 2:14 PM
vlorentz added a comment to T2602: Investigate how to upgrade the schema of the Cassandra storage.

What we can do, however:

Fri, Apr 16, 1:45 PM · Storage manager
vlorentz added a subtask for T1892: Cassandra as a storage backend: T2602: Investigate how to upgrade the schema of the Cassandra storage.
Fri, Apr 16, 1:36 PM · meta-task, Storage manager
vlorentz added a parent task for T2602: Investigate how to upgrade the schema of the Cassandra storage: T1892: Cassandra as a storage backend.
Fri, Apr 16, 1:36 PM · Storage manager
vlorentz added a comment to D5501: add an anti-Dos limit for edges traversed as a query parameter.

@Hakimb You rebase is still incorrect. If you look at the content of the diff, there is still some of your initial work in red.

Fri, Apr 16, 1:30 PM
vlorentz added a comment to D5544: docs/cli: Fix sphinx warning.

Uh? That wasn't an issue for me

Fri, Apr 16, 12:34 PM
vlorentz accepted D5543: multiplexer_objstorage: Fix sphinx warning and napoleon style.

Oh, looks like I forgot to submit my commit for this...

Fri, Apr 16, 12:09 PM
vlorentz accepted D5529: Add test to ensure that an exception is raised if unique_key isn't implemented.
Fri, Apr 16, 9:05 AM

Thu, Apr 15

vlorentz closed D5538: cassandra: Use prepared statements in extid_index_*.
Thu, Apr 15, 4:23 PM
vlorentz committed rDSTOa1fc5fb7143d: cassandra: Use prepared statements in extid_index_* (authored by vlorentz).
cassandra: Use prepared statements in extid_index_*
Thu, Apr 15, 4:23 PM
vlorentz added a comment to T3251: Count authors from revisions and releases.

don't forget to count committers too

Thu, Apr 15, 4:23 PM · Monitoring, Web app
vlorentz requested review of D5538: cassandra: Use prepared statements in extid_index_*.
Thu, Apr 15, 4:02 PM
vlorentz placed T3018: Allow querying raw_extrinsic_metadata by hash in swh-storage up for grabs.
Thu, Apr 15, 3:17 PM · Storage manager, Extrinsic metadata
vlorentz added a subtask for T3090: Make loaders not rely on the 'metadata' column of the 'revision' table: T2564: migrate existing revisions metadata extra_headers to actual extra_headers field.
Thu, Apr 15, 3:15 PM · Core Loader
vlorentz added a parent task for T2564: migrate existing revisions metadata extra_headers to actual extra_headers field: T3090: Make loaders not rely on the 'metadata' column of the 'revision' table.
Thu, Apr 15, 3:15 PM · Storage manager
vlorentz closed T3090: Make loaders not rely on the 'metadata' column of the 'revision' table as Resolved.
Thu, Apr 15, 3:15 PM · Core Loader
vlorentz closed T3090: Make loaders not rely on the 'metadata' column of the 'revision' table, a subtask of T3089: Remove the 'metadata' column of the 'revision' table, as Resolved.
Thu, Apr 15, 3:15 PM · Storage manager, Archive content
vlorentz closed T3138: Make loaders stop writing in the 'metadata' column of the 'revision' table, a subtask of T3090: Make loaders not rely on the 'metadata' column of the 'revision' table, as Resolved.
Thu, Apr 15, 3:15 PM · Core Loader
vlorentz closed T3138: Make loaders stop writing in the 'metadata' column of the 'revision' table as Resolved.
Thu, Apr 15, 3:15 PM · Core Loader
vlorentz closed T3139: Make loaders stop reading from the 'metadata' column of the 'revision' table, a subtask of T3138: Make loaders stop writing in the 'metadata' column of the 'revision' table, as Resolved.
Thu, Apr 15, 3:15 PM · Core Loader
vlorentz closed T3139: Make loaders stop reading from the 'metadata' column of the 'revision' table as Resolved.
Thu, Apr 15, 3:15 PM · Core Loader
vlorentz closed T3142: Make loaders write to the ExtId storage, a subtask of T3143: Migrate revision metadata to extid in the storage, as Resolved.
Thu, Apr 15, 3:15 PM · System administration, Storage manager, Core Loader
vlorentz closed T3142: Make loaders write to the ExtId storage as Resolved.
Thu, Apr 15, 3:15 PM · Core Loader