Page MenuHomeSoftware Heritage

seirl (Antoine Pietri)
User

User Details

User Since
Feb 2 2017, 11:38 AM (258 w, 5 d)

Recent Activity

Yesterday

seirl closed T2981: Graph API: add a (node type) result filters as Resolved by committing rDGRPH294128e0f96e: Use AllowedNodesTest to implement return type filtering.
Tue, Jan 18, 1:26 PM · Graph service
seirl closed D6954: Use AllowedNodesTest to implement return type filtering.
Tue, Jan 18, 1:26 PM
seirl committed rDGRPH294128e0f96e: Use AllowedNodesTest to implement return type filtering (authored by seirl).
Use AllowedNodesTest to implement return type filtering
Tue, Jan 18, 1:26 PM

Mon, Jan 17

seirl updated the diff for D6954: Use AllowedNodesTest to implement return type filtering.

Rebase

Mon, Jan 17, 4:35 PM
seirl closed T2983: graph service: allow loading in memory only one direction of the graph as Resolved.
Mon, Jan 17, 4:33 PM · Graph service
seirl closed D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.
Mon, Jan 17, 4:32 PM
seirl committed rDGRPH83fcf6bb8156: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph (authored by seirl).
Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph
Mon, Jan 17, 4:32 PM
seirl committed rDGRPHbb9b5fc59d8d: SwhBidirectionalGraph: add UML diagram of class hierarchy (authored by seirl).
SwhBidirectionalGraph: add UML diagram of class hierarchy
Mon, Jan 17, 4:31 PM
seirl closed T3302: Write docstrings for each method in swh/graph/backend.py as Resolved.

Since D6676 the specialized methods of swh/graph/backend.py have all been removed and replaced by a generic proxy layer that calls all the methods in a completely transparent fashion, so this specific issue appears to be obsolete now.

Mon, Jan 17, 4:09 PM · Graph service
seirl added a comment to D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.

Right, this is what T3855 will cover, but there is also a dependency on T1971 to figure out where this will go exactly (package summary vs sphinx)

Mon, Jan 17, 3:58 PM
seirl added a comment to D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.

I filed T3855 to document the architecture of the java code of swh-graph so that it stays on our radar.

Mon, Jan 17, 3:41 PM
seirl triaged T3855: Document the architecture of the Java code in swh-graph as Normal priority.
Mon, Jan 17, 3:40 PM · Graph service
seirl added a comment to D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.

I agree in general, but I disagree for the specifics of this diff. This is a description of the design of a specific component, which could be copied in an architecture presentation if it makes sense, but is certainly not an architecture presentation in itself.

Mon, Jan 17, 3:39 PM
seirl updated the diff for D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.

SwhBidirectionalGraph: add UML diagram of class hierarchy

Mon, Jan 17, 1:46 PM
seirl added a comment to D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.

@vlorentz this is all written in the docstrings of the classes, which will be added to swh-docs once https://forge.softwareheritage.org/T1971 is implemented (which is going to be easier to do thanks to this diff :-))

Mon, Jan 17, 1:42 PM

Sat, Jan 15

seirl requested review of D6954: Use AllowedNodesTest to implement return type filtering.
Sat, Jan 15, 12:05 AM
seirl changed the status of T2981: Graph API: add a (node type) result filters from Open to Work in Progress.
Sat, Jan 15, 12:04 AM · Graph service
seirl added a revision to T2981: Graph API: add a (node type) result filters: D6954: Use AllowedNodesTest to implement return type filtering.
Sat, Jan 15, 12:03 AM · Graph service

Fri, Jan 14

seirl renamed T3832: Investigate Luigi as an ETL framework for the compression pipeline from Investigate Luigi as an ETR framework for the compression pipeline to Investigate Luigi as an ETL framework for the compression pipeline.
Fri, Jan 14, 11:33 PM · Graph service
seirl added a revision to T2983: graph service: allow loading in memory only one direction of the graph: D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.
Fri, Jan 14, 11:29 PM · Graph service
seirl updated the summary of D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.
Fri, Jan 14, 11:29 PM
seirl changed the status of T2983: graph service: allow loading in memory only one direction of the graph from Open to Work in Progress.
Fri, Jan 14, 11:29 PM · Graph service
seirl updated the diff for D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.

Fix more buggy Graph -> SwhBidirectionalGraph

Fri, Jan 14, 11:28 PM
seirl updated the diff for D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.

Remove Graph -> SwhBidirectionalGraph strings

Fri, Jan 14, 11:24 PM
seirl requested review of D6953: Refactor Graph class in SwhUnidirectionalGraph and SwhBidirectionalGraph.
Fri, Jan 14, 11:20 PM

Thu, Jan 13

seirl committed rDGRPH64f35108dddd: Make random_walk tests deterministic (authored by seirl).
Make random_walk tests deterministic
Thu, Jan 13, 4:08 PM
seirl closed T3831: Flaky test in swh-graph as Resolved by committing rDGRPHc2074adc3d2c: Increase retries for random walks from 5 to 10.
Thu, Jan 13, 4:08 PM · Graph service
seirl closed D6893: Increase retries for random walks from 5 to 10.
Thu, Jan 13, 4:08 PM
seirl committed rDGRPHc2074adc3d2c: Increase retries for random walks from 5 to 10 (authored by seirl).
Increase retries for random walks from 5 to 10
Thu, Jan 13, 4:08 PM
seirl added a comment to D6893: Increase retries for random walks from 5 to 10.

Here, I changed it so that all the paths go to the correct node.

Thu, Jan 13, 3:52 PM
seirl updated the diff for D6893: Increase retries for random walks from 5 to 10.

Make random_walk tests deterministic

Thu, Jan 13, 3:52 PM

Wed, Jan 12

seirl closed T3161: graph service: add anti-DoS limit on the number of edges traversed, a subtask of T2220: swh-graph in production, as Resolved.
Wed, Jan 12, 5:01 PM · meta-task, Roadmap 2021, Graph service
seirl closed T3161: graph service: add anti-DoS limit on the number of edges traversed as Resolved.
Wed, Jan 12, 5:01 PM · Graph service
seirl closed D5675: update the graph rpc api doc.
Wed, Jan 12, 2:52 PM
seirl committed rDGRPH1a1bb59c1b8c: Document max_edges and return_types query parameters in RPC API (authored by seirl).
Document max_edges and return_types query parameters in RPC API
Wed, Jan 12, 2:52 PM
seirl accepted D6919: api/graph: Handle query parameters that might be passed in graph_query.
Wed, Jan 12, 2:50 PM
seirl updated the diff for D5675: update the graph rpc api doc.

Rebase

Wed, Jan 12, 2:45 PM
seirl requested changes to D6919: api/graph: Handle query parameters that might be passed in graph_query.

Minor comment to avoid doing url parsing manually, otherwise LGTM

Wed, Jan 12, 2:17 PM
seirl accepted D6914: api/graph: Implement anti-DoS policies for graph visits.
Wed, Jan 12, 2:08 PM
seirl closed D6892: Add max_edges argument to all the endpoints.
Wed, Jan 12, 2:06 PM
seirl committed rDGRPH32d6b0ccf3b1: Add max_edges argument to all the endpoints (authored by seirl).
Add max_edges argument to all the endpoints
Wed, Jan 12, 2:06 PM

Tue, Jan 11

seirl requested changes to D6914: api/graph: Implement anti-DoS policies for graph visits.

I have just one security consideration, otherwise LGTM. Thanks!

Tue, Jan 11, 3:15 PM

Mon, Jan 10

seirl added a comment to D6892: Add max_edges argument to all the endpoints.

The coverage report does not seem to agree though ;)

Mon, Jan 10, 7:50 PM
seirl added a comment to T3831: Flaky test in swh-graph.

No, we want to check that random_walk can reach its actual destination.

Mon, Jan 10, 2:54 PM · Graph service

Fri, Jan 7

seirl triaged T3836: Define and implement an anti-DoS policy for graph visits using the max_edges parameter as High priority.
Fri, Jan 7, 5:12 PM · Web app
seirl changed the status of T3831: Flaky test in swh-graph from Open to Work in Progress.
Fri, Jan 7, 4:37 PM · Graph service
seirl added a comment to T3831: Flaky test in swh-graph.

I made a temporary fix in D6893, it doesn't solve the underlying issue but greatly decreases the probability of it happening. I'm not quite sure what would be a proper test for this endpoint, but this is at least enough to fix this issue in particular.

Fri, Jan 7, 4:36 PM · Graph service
seirl requested review of D5675: update the graph rpc api doc.
Fri, Jan 7, 4:35 PM
seirl requested review of D6893: Increase retries for random walks from 5 to 10.
Fri, Jan 7, 4:28 PM
seirl added a revision to T3831: Flaky test in swh-graph: D6893: Increase retries for random walks from 5 to 10.
Fri, Jan 7, 4:26 PM · Graph service
seirl requested review of D6892: Add max_edges argument to all the endpoints.
Fri, Jan 7, 4:02 PM

Tue, Jan 4

seirl triaged T3832: Investigate Luigi as an ETL framework for the compression pipeline as Normal priority.
Tue, Jan 4, 2:17 PM · Graph service

Dec 14 2021

seirl claimed T3161: graph service: add anti-DoS limit on the number of edges traversed.
Dec 14 2021, 1:32 PM · Graph service

Dec 10 2021

seirl closed T2647: add LLP support to graph compression pipeline as Resolved by committing rDGRPH00112952614e: Add LLP compression to the WebGraph pipeline.
Dec 10 2021, 3:00 PM · Graph service
seirl closed D4821: Add LLP compression to the WebGraph pipeline.
Dec 10 2021, 3:00 PM
seirl committed rDGRPH00112952614e: Add LLP compression to the WebGraph pipeline (authored by seirl).
Add LLP compression to the WebGraph pipeline
Dec 10 2021, 3:00 PM
seirl triaged T3794: Document swh-graph compression pipeline options as Normal priority.
Dec 10 2021, 3:00 PM · Graph service
seirl triaged T3793: Add copyright notices to all swh-graph Java files as Low priority.
Dec 10 2021, 3:00 PM · Graph service
seirl added a comment to D4821: Add LLP compression to the WebGraph pipeline.

Yes, this is implicitly all tested under the compression pipeline test.

Dec 10 2021, 2:59 PM

Dec 6 2021

seirl triaged T3768: Read compression input from ORC instead of the edges file as High priority.
Dec 6 2021, 11:05 AM · Graph service
seirl created T3768: Read compression input from ORC instead of the edges file.
Dec 6 2021, 11:05 AM · Graph service

Dec 4 2021

seirl updated the diff for D4821: Add LLP compression to the WebGraph pipeline.

Add simplify step, fix various review comments

Dec 4 2021, 2:07 AM
seirl committed rDGRPH58de681bd729: Update Maven dependencies (authored by seirl).
Update Maven dependencies
Dec 4 2021, 2:05 AM
seirl closed D6699: Stop writing swhid2node.bin maps.
Dec 4 2021, 1:30 AM
seirl committed rDGRPHd0dbfda9a775: Stop writing swhid2node.bin maps (authored by seirl).
Stop writing swhid2node.bin maps
Dec 4 2021, 1:30 AM

Dec 1 2021

seirl closed T3739: swh-graph: Remove SWHID -> Node ID mapping, use MPH instead as Resolved.
Dec 1 2021, 4:24 PM · Graph service
seirl added a comment to D6699: Stop writing swhid2node.bin maps.

why the need for SortOutputHandler attributes to become final?

Dec 1 2021, 3:00 PM

Nov 26 2021

seirl requested review of D6699: Stop writing swhid2node.bin maps.
Nov 26 2021, 5:36 PM
seirl closed T3740: swh-graph: Translate node IDs on the Java side, not Python side, a subtask of T3739: swh-graph: Remove SWHID -> Node ID mapping, use MPH instead, as Resolved.
Nov 26 2021, 5:33 PM · Graph service
seirl closed T3740: swh-graph: Translate node IDs on the Java side, not Python side as Resolved.
Nov 26 2021, 5:33 PM · Graph service
seirl closed D6676: Move SWHID<->node ID conversion in the Java backend.
Nov 26 2021, 5:05 PM
seirl committed rDGRPH0b33cff0d228: Move SWHID<->node ID conversion in the Java backend (authored by seirl).
Move SWHID<->node ID conversion in the Java backend
Nov 26 2021, 5:05 PM
seirl updated the diff for D6676: Move SWHID<->node ID conversion in the Java backend.

Fix src/dst inversion, add regression test

Nov 26 2021, 4:33 PM
seirl added inline comments to D6676: Move SWHID<->node ID conversion in the Java backend.
Nov 26 2021, 1:47 PM

Nov 25 2021

seirl committed rDGRPH32bab89d4448: BidirectionalImmutableGraph: implement outdegrees and predecessorBigArray… (authored by seirl).
BidirectionalImmutableGraph: implement outdegrees and predecessorBigArray…
Nov 25 2021, 5:00 PM
seirl committed rDGRPH5f5ae5dcc104: Add mvn/jvm.config to fix spotless not working with OpenJDK 16+ (authored by seirl).
Add mvn/jvm.config to fix spotless not working with OpenJDK 16+
Nov 25 2021, 4:04 PM
seirl committed rDGRPHbe6c986a5238: Move bidirectional graph logic into a separate ImmutableBidirectionalGraph class (authored by seirl).
Move bidirectional graph logic into a separate ImmutableBidirectionalGraph class
Nov 25 2021, 4:04 PM
seirl committed rDGRPH3cbcf625aa24: SubdatasetSizeFunction: collect more statistics (authored by seirl).
SubdatasetSizeFunction: collect more statistics
Nov 25 2021, 4:03 PM
seirl updated the task description for T3579: Meta-task: upgrade infrastructure to Debian Bullseye.
Nov 25 2021, 12:11 PM · System administration (Component upgrades)
seirl updated the task description for T3579: Meta-task: upgrade infrastructure to Debian Bullseye.
Nov 25 2021, 12:10 PM · System administration (Component upgrades)

Nov 23 2021

seirl requested review of D6676: Move SWHID<->node ID conversion in the Java backend.
Nov 23 2021, 5:35 PM
seirl added a revision to T3740: swh-graph: Translate node IDs on the Java side, not Python side: D6676: Move SWHID<->node ID conversion in the Java backend.
Nov 23 2021, 5:29 PM · Graph service

Nov 19 2021

seirl committed rDSEA150cbbca19bb: setup.py: use yarnpkg instead of yarn if present in PATH (authored by seirl).
setup.py: use yarnpkg instead of yarn if present in PATH
Nov 19 2021, 5:53 PM
seirl closed T3742: yarn called in swh-search setup.py but not present in developer setup docs as Resolved.

Fixed in rDDOC55cdfd9ee957f57cf91b0f6932cc941d2887d933

Nov 19 2021, 5:29 PM · Archive search
seirl committed rDDOC55cdfd9ee957: Add yarnpkg dependency to developer-setup (authored by seirl).
Add yarnpkg dependency to developer-setup
Nov 19 2021, 5:28 PM
seirl triaged T3742: yarn called in swh-search setup.py but not present in developer setup docs as Normal priority.
Nov 19 2021, 5:17 PM · Archive search
seirl added a subtask for T3739: swh-graph: Remove SWHID -> Node ID mapping, use MPH instead: T3740: swh-graph: Translate node IDs on the Java side, not Python side.
Nov 19 2021, 4:44 PM · Graph service
seirl added a parent task for T3740: swh-graph: Translate node IDs on the Java side, not Python side: T3739: swh-graph: Remove SWHID -> Node ID mapping, use MPH instead.
Nov 19 2021, 4:44 PM · Graph service
seirl triaged T3740: swh-graph: Translate node IDs on the Java side, not Python side as High priority.
Nov 19 2021, 4:42 PM · Graph service
seirl triaged T3739: swh-graph: Remove SWHID -> Node ID mapping, use MPH instead as High priority.
Nov 19 2021, 4:40 PM · Graph service

Nov 2 2021

seirl added a comment to T2983: graph service: allow loading in memory only one direction of the graph.

Copying my comment from a linked diff:

Nov 2 2021, 3:19 PM · Graph service
seirl added a comment to D6594: Add parameter to load a single graph direction in memory..

Hey! Thanks for this initial diff.

Nov 2 2021, 3:18 PM

Oct 18 2021

seirl closed T3624: Update swh-graph from 0.3.0 to 0.5.0 on granet, a subtask of T3623: Run swh-graph with gunicorn to support multiple/parallel requests, as Resolved.
Oct 18 2021, 3:01 PM · Graph service, System administration
seirl closed T3624: Update swh-graph from 0.3.0 to 0.5.0 on granet as Resolved.

Done

Oct 18 2021, 3:01 PM · Graph service, System administration

Oct 14 2021

seirl updated the task description for T3639: prepare quote for "granet2", next gen swh-graph compression server.
Oct 14 2021, 1:58 PM · System administration

Aug 12 2021

seirl accepted D6072: StreamingGraphView: Buffer lines before writing.
Aug 12 2021, 1:44 AM

Jul 28 2021

seirl accepted D6038: journalprocessor: Fix deserialize_message raising EOFError on the last message of each assignment.

LGTM too

Jul 28 2021, 1:26 PM

Jul 27 2021

seirl accepted D6028: journalprocessor: Fix freeze on empty offset ranges..
Jul 27 2021, 4:26 PM

Jul 26 2021

seirl created P1101 Weird repos.
Jul 26 2021, 6:40 PM

Jul 9 2021

seirl committed rDGRPHbb1ac27436bd: LabelMapBuilder: mmap order file, use less RAM (authored by seirl).
LabelMapBuilder: mmap order file, use less RAM
Jul 9 2021, 5:19 PM