Page MenuHomeSoftware Heritage

marmoute (Pierre-Yves David)
User

Projects

User does not belong to any projects.

User Details

User Since
Feb 11 2020, 6:04 PM (219 w, 2 d)

Recent Activity

Sep 30 2022

marmoute added a comment to T4590: swh-scanner - improve distribution, documentation and usability.

D8572 (patterns: adds a list of common patterns to exclude) should be attached to this task instead.

Sep 30 2022, 11:20 AM · Code scanner
marmoute added a comment to T4592: swh-scanner - generic bug fixing, robustness and performance improvements.

https://forge.softwareheritage.org/D8571 is also relevant for this Task

Sep 30 2022, 11:19 AM · Code scanner
marmoute closed D8571: from_disks: fix some of the pattern checking logic.
Sep 30 2022, 11:18 AM
marmoute committed rDMOD6a38c4ad047d: from_disks: fix some of the pattern checking logic (authored by marmoute).
from_disks: fix some of the pattern checking logic
Sep 30 2022, 11:18 AM

Sep 29 2022

marmoute updated the diff for D8571: from_disks: fix some of the pattern checking logic.
fixed typos
Sep 29 2022, 6:12 PM
marmoute added a comment to D8571: from_disks: fix some of the pattern checking logic.

Thanks!

This lgtm overall.

Given it's a well covered module though, that'd be neat to add the necessary tests to
cover at least the new introduced path (line 311). That'd clarify the current code
change (for the reviewer).

Sep 29 2022, 6:11 PM
marmoute committed rDTSCN9a8258a7719a: login: automatically fetch a token through the API (authored by marmoute).
login: automatically fetch a token through the API
Sep 29 2022, 5:40 PM
marmoute committed rDTSCN4aad6c17e8b3: scanner: add a `swh scanner login` command (authored by Alphare).
scanner: add a `swh scanner login` command
Sep 29 2022, 5:40 PM
marmoute closed D8555: Add a warning and a command to help user to login.
Sep 29 2022, 5:40 PM
marmoute committed rDTSCNee788b24bcd0: scanner: issue a warning when no auth token exists (authored by marmoute).
scanner: issue a warning when no auth token exists
Sep 29 2022, 5:40 PM
marmoute updated the diff for D8555: Add a warning and a command to help user to login.
rebase on the new master
Sep 29 2022, 5:36 PM
marmoute committed rDTSCN5f3140ab75da: client: add an option to display http debug information (authored by marmoute).
client: add an option to display http debug information
Sep 29 2022, 5:35 PM
marmoute committed rDTSCN11b13ce99119: client: add some basic error detection and rate limiting (authored by marmoute).
client: add some basic error detection and rate limiting
Sep 29 2022, 5:35 PM
marmoute committed rDTSCN1b05e7f3ee5d: client: prepare for retry (authored by marmoute).
client: prepare for retry
Sep 29 2022, 5:35 PM
marmoute committed rDTSCNa17846cda5b5: client: transforme the swhid to string earlier (authored by marmoute).
client: transforme the swhid to string earlier
Sep 29 2022, 5:35 PM
marmoute committed rDTSCN038e7b9454e7: client: compute the client endpoin once and ealier (authored by marmoute).
client: compute the client endpoin once and ealier
Sep 29 2022, 5:35 PM
marmoute committed rDTSCNc44df1c8ecfb: client: move `make_request` to be a method (authored by marmoute).
client: move `make_request` to be a method
Sep 29 2022, 5:35 PM
marmoute committed rDTSCN549faee3fcbf: client: reorder some conditionnal (authored by marmoute).
client: reorder some conditionnal
Sep 29 2022, 5:35 PM
marmoute closed D8541: Make HTTP requests more resilient.
Sep 29 2022, 5:35 PM
marmoute committed rDTSCN5293bbe477f7: client: extract the `get_chunk` closure (authored by marmoute).
client: extract the `get_chunk` closure
Sep 29 2022, 5:35 PM
marmoute updated the diff for D8541: Make HTTP requests more resilient.
rebase on master
Sep 29 2022, 5:28 PM
marmoute closed D8572: patterns: adds a list of common patterns to exclude.
Sep 29 2022, 5:27 PM
marmoute committed rDTSCN4363c9a9e307: patterns: adds a list of common patterns to exclude (authored by marmoute).
patterns: adds a list of common patterns to exclude
Sep 29 2022, 5:27 PM
marmoute updated the diff for D8572: patterns: adds a list of common patterns to exclude.
added the TODO comment suggested by ardumont
Sep 29 2022, 5:21 PM
marmoute added a comment to D8572: patterns: adds a list of common patterns to exclude.

lgtm

"As usual", i don't know how well that module is covered, it'd be good to
have a test around the scan function to check it behaves as we want.

Another suggestion inline.

Sep 29 2022, 5:17 PM

Sep 28 2022

marmoute requested review of D8571: from_disks: fix some of the pattern checking logic.
Sep 28 2022, 7:49 PM
marmoute requested review of D8572: patterns: adds a list of common patterns to exclude.
Sep 28 2022, 7:40 PM
marmoute updated the diff for D8541: Make HTTP requests more resilient.
use the logging module instead of a print
Sep 28 2022, 6:18 PM
marmoute updated the diff for D8555: Add a warning and a command to help user to login.
rebased on the new master.
Sep 28 2022, 5:29 PM
marmoute committed rDTSCNf4f091776951: output: pick "summary" as the default (authored by marmoute).
output: pick "summary" as the default
Sep 28 2022, 5:26 PM
marmoute committed rDTSCN6e3a0bbee973: output: add a "summary" output (authored by marmoute).
output: add a "summary" output
Sep 28 2022, 5:26 PM
marmoute committed rDTSCN821873a03541: scan: start documenting the --output-format option better (authored by marmoute).
scan: start documenting the --output-format option better
Sep 28 2022, 5:26 PM
marmoute closed D8540: Various user Experience improvement:.
Sep 28 2022, 5:26 PM
marmoute committed rDTSCNb3b8ca5f0f57: scan: use the current directory as the default the target directory (authored by marmoute).
scan: use the current directory as the default the target directory
Sep 28 2022, 5:26 PM
marmoute updated the diff for D8555: Add a warning and a command to help user to login.
  • login: automatically fetch a token through the API
Sep 28 2022, 5:21 PM
marmoute requested review of D8555: Add a warning and a command to help user to login.
Sep 28 2022, 4:36 PM
marmoute updated the diff for D8541: Make HTTP requests more resilient.
  • client: add an option to display http debug information
Sep 28 2022, 3:12 PM
marmoute updated the diff for D8540: Various user Experience improvement:.
  • ran black
Sep 28 2022, 3:10 PM
marmoute updated the diff for D8540: Various user Experience improvement:.

.

Sep 28 2022, 3:08 PM
marmoute updated the diff for D8540: Various user Experience improvement:.
gathered relevant string as poitned by valentin
Sep 28 2022, 3:07 PM

Sep 27 2022

marmoute added a comment to D8541: Make HTTP requests more resilient.

I feel all this retry logic should be in swh.web.client, because all clients need it, not just swh.scanner

Sep 27 2022, 7:09 PM
marmoute added inline comments to D8540: Various user Experience improvement:.
Sep 27 2022, 6:59 PM
marmoute removed a reviewer for D8541: Make HTTP requests more resilient: bchauvet.
Sep 27 2022, 5:42 PM
marmoute removed a reviewer for D8540: Various user Experience improvement:: bchauvet.
Sep 27 2022, 5:42 PM
marmoute requested review of D8541: Make HTTP requests more resilient.
Sep 27 2022, 12:45 AM
marmoute updated the diff for D8540: Various user Experience improvement:.

fix some commit

Sep 27 2022, 12:41 AM
marmoute retitled D8540: Various user Experience improvement: from Various user Experience improvement: - using a default directory to be scanned `$PWD` - document the --output-format value and meaning - add a "summary" output and use it the default. Check individual commit messag for default: --- scan: use... to Various user Experience improvement:.
Sep 27 2022, 12:40 AM
marmoute requested review of D8540: Various user Experience improvement:.
Sep 27 2022, 12:24 AM

Sep 26 2022

marmoute committed rDMOD2d65a24a5f41: model: inline the call to `_check_swhid` (authored by marmoute).
model: inline the call to `_check_swhid`
Sep 26 2022, 2:29 PM
marmoute committed rDMOD3608271a0aaa: model: optimization pass on custom validator (authored by marmoute).
model: optimization pass on custom validator
Sep 26 2022, 2:29 PM
marmoute committed rDMODb7267a890888: model: remove the try/except (authored by marmoute).
model: remove the try/except
Sep 26 2022, 2:29 PM
marmoute committed rDMOD3796e5ba30ff: model: delete unused validator code (authored by marmoute).
model: delete unused validator code
Sep 26 2022, 2:29 PM
marmoute committed rDMODcf529cd1622e: model: also optimize combined validator (authored by marmoute).
model: also optimize combined validator
Sep 26 2022, 2:29 PM
marmoute committed rDMODedb57fb15359: model: implement specialized attribute-validator functions (authored by marmoute).
model: implement specialized attribute-validator functions
Sep 26 2022, 2:29 PM
marmoute committed rDMOD6ababdebb749: model: drop the `type_validator()` indirection (authored by marmoute).
model: drop the `type_validator()` indirection
Sep 26 2022, 2:29 PM
marmoute closed D8512: various optimisation to the model validation logic..
Sep 26 2022, 2:29 PM
marmoute closed D8527: model: avoid another extra creation of Model object.
Sep 26 2022, 2:29 PM
marmoute closed D8525: from_disk: skip intermediate dictionnary creation when building model.
Sep 26 2022, 2:29 PM
marmoute committed rDMOD1dfea3247702: model: prepare the filtering of type_validator into something faster (authored by marmoute).
model: prepare the filtering of type_validator into something faster
Sep 26 2022, 2:28 PM
marmoute committed rDMODa2e8f18c2a68: from_disk: skip intermediate dictionnary creation when building model (authored by marmoute).
from_disk: skip intermediate dictionnary creation when building model
Sep 26 2022, 2:28 PM
marmoute committed rDMODad3ecac9beae: model: avoid another extra creation of Model object (authored by marmoute).
model: avoid another extra creation of Model object
Sep 26 2022, 2:28 PM
marmoute closed D8510: from_disk: only build a model object once.
Sep 26 2022, 2:28 PM
marmoute committed rDMOD814a6c8416d5: from_disk: only build a model object once (authored by marmoute).
from_disk: only build a model object once
Sep 26 2022, 2:28 PM

Sep 23 2022

marmoute updated the diff for D8512: various optimisation to the model validation logic..

batch-update

Sep 23 2022, 7:49 PM
marmoute updated the diff for D8525: from_disk: skip intermediate dictionnary creation when building model.

batch-update

Sep 23 2022, 7:49 PM
marmoute updated the diff for D8527: model: avoid another extra creation of Model object.

batch-update

Sep 23 2022, 7:49 PM
marmoute updated the diff for D8510: from_disk: only build a model object once.

batch-update

Sep 23 2022, 7:49 PM
marmoute added inline comments to D8512: various optimisation to the model validation logic..
Sep 23 2022, 7:45 PM
marmoute added a comment to D8525: from_disk: skip intermediate dictionnary creation when building model.

You are planning to merge this new code with child_to_directory_entry's in a future diff, right?

Sep 23 2022, 7:35 PM
marmoute added a reviewer for D8512: various optimisation to the model validation logic.: Reviewers.
Sep 23 2022, 4:55 PM
marmoute retitled D8512: various optimisation to the model validation logic. from model: prepare the filtering of type_validator into something faster to various optimisation to the model validation logic..
Sep 23 2022, 4:54 PM
marmoute retitled D8525: from_disk: skip intermediate dictionnary creation when building model from model: skip intermediate dictionnary creation when building model to from_disk: skip intermediate dictionnary creation when building model.
Sep 23 2022, 4:53 PM
marmoute updated the summary of D8527: model: avoid another extra creation of Model object.
Sep 23 2022, 4:52 PM
marmoute retitled D8527: model: avoid another extra creation of Model object from avoid another extra creation of Model object to model: avoid another extra creation of Model object.
Sep 23 2022, 4:51 PM
marmoute updated the summary of D8510: from_disk: only build a model object once.
Sep 23 2022, 4:50 PM
marmoute updated the diff for D8512: various optimisation to the model validation logic..

batch-update

Sep 23 2022, 4:47 PM
marmoute updated the diff for D8525: from_disk: skip intermediate dictionnary creation when building model.

batch-update

Sep 23 2022, 4:47 PM
marmoute updated the diff for D8527: model: avoid another extra creation of Model object.

batch-update

Sep 23 2022, 4:47 PM
marmoute updated the diff for D8510: from_disk: only build a model object once.

batch-update

Sep 23 2022, 4:47 PM
marmoute planned changes to D8512: various optimisation to the model validation logic..

I am in the process to adding more timing/profile data to the commit message (then to fight to get phabricator to display them)

Sep 23 2022, 12:35 PM
marmoute planned changes to D8525: from_disk: skip intermediate dictionnary creation when building model.

I am in the process to adding more timing/profile data to the commit message (then to fight to get phabricator to display them)

Sep 23 2022, 12:35 PM
marmoute planned changes to D8527: model: avoid another extra creation of Model object.

I am in the process to adding more timing/profile data to the commit message (then to fight to get phabricator to display them)

Sep 23 2022, 12:35 PM
marmoute added a comment to D8510: from_disk: only build a model object once.

I have updated the commit message with more timing data, but I am not sure how to get phabricator to reflect that.

Sep 23 2022, 12:34 PM
marmoute updated the diff for D8512: various optimisation to the model validation logic..

batch-update

Sep 23 2022, 11:42 AM
marmoute updated the diff for D8525: from_disk: skip intermediate dictionnary creation when building model.

batch-update

Sep 23 2022, 11:42 AM
marmoute updated the diff for D8527: model: avoid another extra creation of Model object.

batch-update

Sep 23 2022, 11:42 AM
marmoute updated the diff for D8510: from_disk: only build a model object once.

batch-update

Sep 23 2022, 11:42 AM
marmoute updated the diff for D8512: various optimisation to the model validation logic..
  • model: prepare the filtering of type_validator into something faster
  • model: implement specialized attribute-validator functions
  • model: drop the type_validator() indirection
  • model: also optimize combined validator
  • model: remove the try/except
  • model: delete unused code dead code
  • model: optimization pass on custom validator
  • model: inline _check_swhid
Sep 23 2022, 11:40 AM
marmoute updated the diff for D8525: from_disk: skip intermediate dictionnary creation when building model.

testing update of diff

Sep 23 2022, 11:40 AM
marmoute updated the diff for D8527: model: avoid another extra creation of Model object.

testing update of diff

Sep 23 2022, 11:40 AM
marmoute updated the diff for D8510: from_disk: only build a model object once.

testing update of diff

Sep 23 2022, 11:39 AM
marmoute requested review of D8527: model: avoid another extra creation of Model object.
Sep 23 2022, 11:34 AM
marmoute requested review of D8525: from_disk: skip intermediate dictionnary creation when building model.
Sep 23 2022, 11:27 AM

Sep 22 2022

marmoute updated the diff for D8512: various optimisation to the model validation logic..
  • model: inline _check_swhid
Sep 22 2022, 2:56 PM
marmoute updated the diff for D8512: various optimisation to the model validation logic..
  • model: optimization pass on custom validator
Sep 22 2022, 2:49 PM
marmoute added a comment to D8512: various optimisation to the model validation logic..

The individual commit should be fairly clean. I'll provides some timing or profile soon™

Sep 22 2022, 12:03 PM
marmoute updated the diff for D8512: various optimisation to the model validation logic..
  • model: remove the try/except
  • model: delete unused code dead code
Sep 22 2022, 12:01 PM

Sep 20 2022

marmoute requested review of D8512: various optimisation to the model validation logic..
Sep 20 2022, 6:56 PM
marmoute requested review of D8510: from_disk: only build a model object once.
Sep 20 2022, 2:40 PM

May 13 2022

marmoute added a comment to T4239: Unstuck bitbucket incremental lister.

Our initial contact (Erik von Zijst) is no longer there, but you can try pinging other who were part of the discussion :

  • TJ Kells <tkells@atlassian.com>
  • Daniel Tao <dtao@atlassian.com>
  • Ming Gong <mgong@atlassian.com>
  • Tom Kane <tkane@atlassian.com>
May 13 2022, 12:46 PM · Bitbucket lister