Page MenuHomeSoftware Heritage

seirl (Antoine Pietri)
User

User Details

User Since
Feb 2 2017, 11:38 AM (282 w, 3 d)

Recent Activity

Thu, Jun 30

seirl committed rDGRPH555e3b6d3e11: rpc_server: use shlex.quote() to print command (authored by seirl).
rpc_server: use shlex.quote() to print command
Thu, Jun 30, 8:49 PM
seirl committed rDGRPHa55edc1f88d5: Remove shlex.join() call (3.8 only) (authored by seirl).
Remove shlex.join() call (3.8 only)
Thu, Jun 30, 8:33 PM
seirl committed rDGRPH94418d4944a8: GRPC: use port 50091 by default (authored by seirl).
GRPC: use port 50091 by default
Thu, Jun 30, 8:07 PM
seirl committed rDGRPH441aacc6aeeb: Update sux4j to 5.4.0 (authored by seirl).
Update sux4j to 5.4.0
Thu, Jun 30, 8:07 PM
seirl requested review of D8064: swh-graph: rename services (now production-ready, no longer dev).
Thu, Jun 30, 7:15 PM
seirl committed rDGRPH26e52b389fb1: rpc_server.py: terminate, then kill server on timeout (authored by seirl).
rpc_server.py: terminate, then kill server on timeout
Thu, Jun 30, 6:59 PM
seirl added a comment to T3259: Gracefully handle a client closing the connection in the middle of a response being streamed.

GRPC update: cancelling a GRPC stream works fine, but it doesn't seem like aiohttp is doing that when the HTTP stream is closed.

Thu, Jun 30, 6:41 PM · Compressed graph service
seirl closed T4316: Push of swh-graph to pypi is broken as Resolved.

We requested a larger quota here: https://github.com/pypa/pypi-support/issues/1998

Thu, Jun 30, 5:37 PM · System administration, Compressed graph service
seirl committed rDGRPHf2f9e8ba69ef: DumpProperties: fix output for tag names (authored by seirl).
DumpProperties: fix output for tag names
Thu, Jun 30, 5:18 PM
seirl closed D8061: Re-add swh/graph/client.py for backward-compatibility.
Thu, Jun 30, 4:48 PM
seirl committed rDGRPH3b9685923021: Re-add swh/graph/client.py for backward-compatibility (authored by seirl).
Re-add swh/graph/client.py for backward-compatibility
Thu, Jun 30, 4:48 PM
seirl requested review of D8061: Re-add swh/graph/client.py for backward-compatibility.
Thu, Jun 30, 4:44 PM
seirl closed T2100: Bootstrap Debian packaging for swh.graph, a subtask of T3168: Proper deployment of swh-graph with debian package, as Wontfix.
Thu, Jun 30, 4:21 PM · Compressed graph service, Puppet recipes
seirl closed T2100: Bootstrap Debian packaging for swh.graph as Wontfix.

We are migrating away from Debian packages as a deployment tool, closing this as WONTFIX.

Thu, Jun 30, 4:21 PM · Compressed graph service
seirl closed T3168: Proper deployment of swh-graph with debian package as Wontfix.

We are migrating away from Debian packages as a deployment tool, closing this as WONTFIX.

Thu, Jun 30, 4:20 PM · Compressed graph service, Puppet recipes
seirl closed T2081: swh-graph: "Cannot open client FIFO" when answering HEAD requests as Resolved.

No longer happens after the GRPC migration:

Thu, Jun 30, 4:19 PM · Compressed graph service
seirl closed T2103: (Debian) package py4j, a subtask of T2100: Bootstrap Debian packaging for swh.graph, as Wontfix.
Thu, Jun 30, 4:16 PM · Compressed graph service
seirl closed T2103: (Debian) package py4j as Wontfix.

We removed the Py4J dependency by migrating to GRPC.

Thu, Jun 30, 4:16 PM · Compressed graph service
seirl closed T3301: graph: add test for the "algo" parameter of walk() as Wontfix.

We no longer support multiple algorithms for shortest path requests.

Thu, Jun 30, 4:16 PM · Easy hack, Compressed graph service
seirl closed T3623: Run swh-graph with gunicorn to support multiple/parallel requests as Resolved.

Obsoleted by the migration to GRPC. Now we use GRPC's threading model, with a threadpool configurable by passing --threads to the Java service. By default, nproc is used.

Thu, Jun 30, 4:12 PM · Compressed graph service, System administration
seirl closed D8059: Generate .labelobl files to speed-up labelled graph loading.
Thu, Jun 30, 4:11 PM
seirl committed rDGRPH32f2626a7d0a: Generate .labelobl files to speed-up labelled graph loading (authored by seirl).
Generate .labelobl files to speed-up labelled graph loading
Thu, Jun 30, 4:11 PM
seirl closed T4113: Review border case of empty response for `visit_nodes` as Resolved.

Obsoleted by the migration to GRPC. We no longer create iterators on the decoded stream of a UNIX pipeline, we directly use GRPC stream iterators.

Thu, Jun 30, 4:11 PM · Compressed graph service
seirl closed T4115: Some unknown SWHID errors crash the graph server as Resolved.

Fixed by the migration to GRPC.

Thu, Jun 30, 4:08 PM · Compressed graph service
seirl updated the diff for D8059: Generate .labelobl files to speed-up labelled graph loading.

rebase

Thu, Jun 30, 4:02 PM
seirl updated the diff for D8059: Generate .labelobl files to speed-up labelled graph loading.
  • Generate .labelobl files to speed-up labelled graph loading
Thu, Jun 30, 4:02 PM
seirl committed rDGRPH795fa182f212: docs: remove relative include of swhgraph.proto (authored by seirl).
docs: remove relative include of swhgraph.proto
Thu, Jun 30, 3:59 PM
seirl created P1397 (An Untitled Masterwork).
Thu, Jun 30, 3:48 PM
seirl requested review of D8059: Generate .labelobl files to speed-up labelled graph loading.
Thu, Jun 30, 3:37 PM
seirl committed rDGRPH5c3f59ecfe52: Rename Node.Type enum to SwhType (authored by seirl).
Rename Node.Type enum to SwhType
Thu, Jun 30, 3:28 PM
seirl closed T3793: Add copyright notices to all swh-graph Java files as Resolved.

Fixed in D8050

Thu, Jun 30, 2:28 PM · Compressed graph service

Wed, Jun 29

seirl closed D8050: Add missing copyright notices to the entire Java codebase.
Wed, Jun 29, 9:46 PM
seirl committed rDGRPHd5a07973b1be: Add missing copyright notices to the entire Java codebase (authored by seirl).
Add missing copyright notices to the entire Java codebase
Wed, Jun 29, 9:46 PM
seirl updated the diff for D8050: Add missing copyright notices to the entire Java codebase.

Fix ranges

Wed, Jun 29, 3:24 PM
seirl added a revision to T3793: Add copyright notices to all swh-graph Java files: D8050: Add missing copyright notices to the entire Java codebase.
Wed, Jun 29, 3:07 PM · Compressed graph service
seirl committed rDGRPH066edad7cc36: SwhGraphProperties: remove useless IOExceptions (authored by seirl).
SwhGraphProperties: remove useless IOExceptions
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHb61ca2d1ef44: SwhUnidirectionalGraph: make constructor with properties public (authored by seirl).
SwhUnidirectionalGraph: make constructor with properties public
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHfd61a31ef0c4: Add CheckSwhid() and Stats() RPC methods (authored by seirl).
Add CheckSwhid() and Stats() RPC methods
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH0b4fd29e3b14: GRPC: initial commit with protobuf + java server (authored by seirl).
GRPC: initial commit with protobuf + java server
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH208f5dc60128: Java: add proto/ dir symlink (authored by seirl).
Java: add proto/ dir symlink
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHf098232f4282: Python: migrate HTTP API to GRPC (authored by seirl).
Python: migrate HTTP API to GRPC
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH431d98de6ae4: RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work… (authored by seirl).
RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work…
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH822f03de2161: Java tests: migrate to GRPC (authored by seirl).
Java tests: migrate to GRPC
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH5c08b117f336: Remove old Java HTTP server, replaced by GRPC (authored by seirl).
Remove old Java HTTP server, replaced by GRPC
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH9eaccde2cadb: NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile (authored by seirl).
NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH6d63a39642ba: Remove now useless dot.py (authored by seirl).
Remove now useless dot.py
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHa4d1eb10793f: Remove now useless backend.py (authored by seirl).
Remove now useless backend.py
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH4e592de704c7: Reorganize Python files (RPC/HTTP server distinction) (authored by seirl).
Reorganize Python files (RPC/HTTP server distinction)
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH5d36ca6eaffa: flake: exclude swh/graph/rpc dir (authored by seirl).
flake: exclude swh/graph/rpc dir
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH131e7165ff72: mypy.ini: ignore grpc generated files (authored by seirl).
mypy.ini: ignore grpc generated files
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHccc6b295a387: requirements-test: add grpc-stubs (authored by seirl).
requirements-test: add grpc-stubs
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHe7fb41e4cd99: Add rpc.StatsTest (authored by seirl).
Add rpc.StatsTest
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH3649bd634e61: proto: migrate to FieldMask to filter out fields (authored by seirl).
proto: migrate to FieldMask to filter out fields
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHeefdd910c43c: Remove GraphDirection.BOTH (labelled iteration not supported yet) (authored by seirl).
Remove GraphDirection.BOTH (labelled iteration not supported yet)
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH383ed27418f9: Add FindPathTo and FindPathBetween endpoints (authored by seirl).
Add FindPathTo and FindPathBetween endpoints
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH70fc951eac99: Traversal: test impossible paths (authored by seirl).
Traversal: test impossible paths
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHa2c35364b57a: Traversal: check for invalid arguments (authored by seirl).
Traversal: check for invalid arguments
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH83be48e06673: Traversal: test traversals from multiple sources/to multiple dests (authored by seirl).
Traversal: test traversals from multiple sources/to multiple dests
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH60e93f13390c: Remove CheckSwhid, use GetNode instead (authored by seirl).
Remove CheckSwhid, use GetNode instead
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH6e16ec9d7579: Add tests for CountNodes/CountEdges (authored by seirl).
Add tests for CountNodes/CountEdges
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHc117623b3c26: Traversal: simplify the StopTraversal logic (authored by seirl).
Traversal: simplify the StopTraversal logic
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH9f83ed730669: Traversal: add max depth and common ancestors tests (authored by seirl).
Traversal: add max depth and common ancestors tests
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH328f26e31ccf: More protobuf/grpc documentation, better field names (authored by seirl).
More protobuf/grpc documentation, better field names
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHc241e13548f2: Traversal: add max edge tests (authored by seirl).
Traversal: add max edge tests
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHc7559ac55f7c: Document protobuf/grpc services and fields (authored by seirl).
Document protobuf/grpc services and fields
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH2f149363d21f: swhgraph.proto: small documentation fixes (authored by seirl).
swhgraph.proto: small documentation fixes
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH98fce8be99f5: Add inline docstrings to Java GRPC server (authored by seirl).
Add inline docstrings to Java GRPC server
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHb58cb841b6a1: doc: First full draft of GRPC API documentation (authored by seirl).
doc: First full draft of GRPC API documentation
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHe6e6866279da: doc: add GRPC page skeleton (authored by seirl).
doc: add GRPC page skeleton
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHab895e3a0b53: Remove old useless classes, including Traversal (authored by seirl).
Remove old useless classes, including Traversal
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH3ce74dfd2a01: Regenerate python protobuf documentation (authored by seirl).
Regenerate python protobuf documentation
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHa6e02f726926: style: typos and indent fixes (authored by seirl).
style: typos and indent fixes
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHd89f35478012: SwhBidirectionalGraph: fix copy() not actually copying subgraphs (authored by seirl).
SwhBidirectionalGraph: fix copy() not actually copying subgraphs
Wed, Jun 29, 12:08 PM
seirl committed rDGRPHf528aa80db5b: GetNode: make endpoint thread-safe with a lightweight copy of the graph (authored by seirl).
GetNode: make endpoint thread-safe with a lightweight copy of the graph
Wed, Jun 29, 12:08 PM
seirl closed D7890: Migrate low-level RPC API from Py4J to GRPC.
Wed, Jun 29, 12:08 PM
seirl committed rDGRPH43a3e3302abe: Update webgraph-big to 3.7.0 (authored by seirl).
Update webgraph-big to 3.7.0
Wed, Jun 29, 12:07 PM
seirl committed rDGRPH53b7d553ba06: GraphServer: Use nproc threads in threadpool by default (authored by seirl).
GraphServer: Use nproc threads in threadpool by default
Wed, Jun 29, 12:07 PM
seirl updated the diff for D7890: Migrate low-level RPC API from Py4J to GRPC.
  • SwhBidirectionalGraph: fix copy() not actually copying subgraphs
  • GetNode: make endpoint thread-safe with a lightweight copy of the graph
  • GraphServer: Use nproc threads in threadpool by default
  • Update webgraph-big to 3.7.0
Wed, Jun 29, 12:07 PM

Tue, Jun 28

seirl closed D8037: swhgraphshm: do not hardcode the graph path, use /latest symlink.
Tue, Jun 28, 6:27 PM
seirl committed rSPSITEf64bcc100756: swhgraphshm: do not hardcode the graph path, use /latest symlink (authored by seirl).
swhgraphshm: do not hardcode the graph path, use /latest symlink
Tue, Jun 28, 6:26 PM
seirl updated the diff for D8037: swhgraphshm: do not hardcode the graph path, use /latest symlink.

Rebase

Tue, Jun 28, 6:26 PM

Fri, Jun 24

seirl requested review of D8037: swhgraphshm: do not hardcode the graph path, use /latest symlink.
Fri, Jun 24, 8:30 PM
seirl updated the diff for D7890: Migrate low-level RPC API from Py4J to GRPC.
  • Remove old useless classes, including Traversal
  • Regenerate python protobuf documentation
  • style: typos and indent fixes
Fri, Jun 24, 12:28 PM
seirl added inline comments to D7890: Migrate low-level RPC API from Py4J to GRPC.
Fri, Jun 24, 12:03 PM
seirl added inline comments to D7890: Migrate low-level RPC API from Py4J to GRPC.
Fri, Jun 24, 12:02 PM
seirl added inline comments to D7890: Migrate low-level RPC API from Py4J to GRPC.
Fri, Jun 24, 11:38 AM

Thu, Jun 23

seirl added a revision to T3259: Gracefully handle a client closing the connection in the middle of a response being streamed: D7890: Migrate low-level RPC API from Py4J to GRPC.
Thu, Jun 23, 7:00 PM · Compressed graph service
seirl added a revision to T2103: (Debian) package py4j: D7890: Migrate low-level RPC API from Py4J to GRPC.
Thu, Jun 23, 7:00 PM · Compressed graph service
seirl added a revision to T4340: swh-graph timeouts: D7890: Migrate low-level RPC API from Py4J to GRPC.
Thu, Jun 23, 7:00 PM · Compressed graph service
seirl added tasks to D7890: Migrate low-level RPC API from Py4J to GRPC: Unknown Object (Maniphest Task), T4340: swh-graph timeouts, T2103: (Debian) package py4j, T3259: Gracefully handle a client closing the connection in the middle of a response being streamed.
Thu, Jun 23, 7:00 PM
seirl added a revision to T4115: Some unknown SWHID errors crash the graph server: D7890: Migrate low-level RPC API from Py4J to GRPC.
Thu, Jun 23, 6:54 PM · Compressed graph service
seirl added a task to D7890: Migrate low-level RPC API from Py4J to GRPC: T4115: Some unknown SWHID errors crash the graph server.
Thu, Jun 23, 6:54 PM
seirl updated the diff for D7890: Migrate low-level RPC API from Py4J to GRPC.
  • Add inline docstrings to Java GRPC server
Thu, Jun 23, 6:17 PM
seirl abandoned D2349: java/Traversal: add findCommonDescendant.

Will be replaced by the GRPC API

Thu, Jun 23, 6:16 PM
seirl updated the diff for D7890: Migrate low-level RPC API from Py4J to GRPC.
  • doc: First full draft of GRPC API documentation
Thu, Jun 23, 5:37 PM
seirl added inline comments to D7890: Migrate low-level RPC API from Py4J to GRPC.
Thu, Jun 23, 12:41 PM
seirl updated the diff for D7890: Migrate low-level RPC API from Py4J to GRPC.
  • SwhGraphProperties: remove useless IOExceptions
  • SwhUnidirectionalGraph: make constructor with properties public
  • GRPC: initial commit with protobuf + java server
  • Add CheckSwhid() and Stats() RPC methods
  • RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work yet
  • Java: add proto/ dir symlink
  • Python: migrate HTTP API to GRPC
  • Java tests: migrate to GRPC
  • Remove old Java HTTP server, replaced by GRPC
  • NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile
  • Remove now useless backend.py
  • Remove now useless dot.py
  • Reorganize Python files (RPC/HTTP server distinction)
  • flake: exclude swh/graph/rpc dir
  • requirements-test: add grpc-stubs
  • mypy.ini: ignore grpc generated files
  • Add rpc.StatsTest
  • proto: migrate to FieldMask to filter out fields
  • Add FindPathTo and FindPathBetween endpoints
  • Remove GraphDirection.BOTH (labelled iteration not supported yet)
  • Traversal: test traversals from multiple sources/to multiple dests
  • Traversal: test impossible paths
  • Traversal: check for invalid arguments
  • Traversal: simplify the StopTraversal logic
  • Remove CheckSwhid, use GetNode instead
  • Add tests for CountNodes/CountEdges
  • Traversal: add max depth and common ancestors tests
  • Traversal: add max edge tests
  • Document protobuf/grpc services and fields
  • More protobuf/grpc documentation, better field names
  • swhgraph.proto: small documentation fixes
  • doc: add GRPC page skeleton
Thu, Jun 23, 12:40 PM

Tue, Jun 14

seirl planned changes to D7890: Migrate low-level RPC API from Py4J to GRPC.

Still needs some work/documentation

Tue, Jun 14, 7:30 PM
seirl requested review of D7890: Migrate low-level RPC API from Py4J to GRPC.
Tue, Jun 14, 7:30 PM
seirl closed D7935: ORC: handle nullable columns/empty tables properly.

Merged

Tue, Jun 14, 5:38 PM