Page MenuHomeSoftware Heritage

Migrate low-level RPC API from Py4J to GRPC
ClosedPublic

Authored by seirl on May 24 2022, 9:13 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T3259: Gracefully handle a client closing the connection in the middle of a response being streamed
T2103: (Debian) package py4j
T4340: swh-graph timeouts
Restricted Maniphest Task
T4115: Some unknown SWHID errors crash the graph server
Commits
rDGRPH066edad7cc36: SwhGraphProperties: remove useless IOExceptions
rDGRPHb61ca2d1ef44: SwhUnidirectionalGraph: make constructor with properties public
rDGRPH0b4fd29e3b14: GRPC: initial commit with protobuf + java server
rDGRPHfd61a31ef0c4: Add CheckSwhid() and Stats() RPC methods
rDGRPH822f03de2161: Java tests: migrate to GRPC
rDGRPH431d98de6ae4: RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work…
rDGRPH208f5dc60128: Java: add proto/ dir symlink
rDGRPHf098232f4282: Python: migrate HTTP API to GRPC
rDGRPH5c08b117f336: Remove old Java HTTP server, replaced by GRPC
rDGRPH6d63a39642ba: Remove now useless dot.py
rDGRPHa4d1eb10793f: Remove now useless backend.py
rDGRPH9eaccde2cadb: NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile
rDGRPH4e592de704c7: Reorganize Python files (RPC/HTTP server distinction)
rDGRPH5d36ca6eaffa: flake: exclude swh/graph/rpc dir
rDGRPHccc6b295a387: requirements-test: add grpc-stubs
rDGRPH131e7165ff72: mypy.ini: ignore grpc generated files
rDGRPHe7fb41e4cd99: Add rpc.StatsTest
rDGRPH3649bd634e61: proto: migrate to FieldMask to filter out fields
rDGRPHeefdd910c43c: Remove GraphDirection.BOTH (labelled iteration not supported yet)
rDGRPH383ed27418f9: Add FindPathTo and FindPathBetween endpoints
rDGRPHa2c35364b57a: Traversal: check for invalid arguments
rDGRPH83be48e06673: Traversal: test traversals from multiple sources/to multiple dests
rDGRPH70fc951eac99: Traversal: test impossible paths
rDGRPH60e93f13390c: Remove CheckSwhid, use GetNode instead
rDGRPH6e16ec9d7579: Add tests for CountNodes/CountEdges
rDGRPHc117623b3c26: Traversal: simplify the StopTraversal logic
rDGRPHc241e13548f2: Traversal: add max edge tests
rDGRPHc7559ac55f7c: Document protobuf/grpc services and fields
rDGRPH9f83ed730669: Traversal: add max depth and common ancestors tests
rDGRPH328f26e31ccf: More protobuf/grpc documentation, better field names
rDGRPH2f149363d21f: swhgraph.proto: small documentation fixes
rDGRPHe6e6866279da: doc: add GRPC page skeleton
rDGRPHb58cb841b6a1: doc: First full draft of GRPC API documentation
rDGRPH98fce8be99f5: Add inline docstrings to Java GRPC server
rDGRPHab895e3a0b53: Remove old useless classes, including Traversal
rDGRPH3ce74dfd2a01: Regenerate python protobuf documentation
rDGRPHa6e02f726926: style: typos and indent fixes
rDGRPHd89f35478012: SwhBidirectionalGraph: fix copy() not actually copying subgraphs
rDGRPHf528aa80db5b: GetNode: make endpoint thread-safe with a lightweight copy of the graph
rDGRPH43a3e3302abe: Update webgraph-big to 3.7.0
rDGRPH53b7d553ba06: GraphServer: Use nproc threads in threadpool by default
Summary
  • SwhGraphProperties: remove useless IOExceptions
  • SwhUnidirectionalGraph: make constructor with properties public
  • GRPC: initial commit with protobuf + java server
  • Add CheckSwhid() and Stats() RPC methods
  • RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work yet
  • Java: add proto/ dir symlink
  • Python: migrate HTTP API to GRPC
  • Java tests: migrate to GRPC
  • Remove old Java HTTP server, replaced by GRPC
  • NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile
  • Remove now useless backend.py
  • Remove now useless dot.py
  • Reorganize Python files (RPC/HTTP server distinction)

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
java/README.md
22

No, I put GRPC on port 50091 since 5009 is the HTTP API. If we deprecate the HTTP API we can move GRPC to 5009.

Build is green

Patch application report for D7890 (id=28920)

Rebasing onto 89975d2372...

Current branch diff-target is up to date.
Changes applied before test
commit a487585886553aa05658e91f9a3b43e1f071d7bc
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:39:56 2022 +0200

    doc: add GRPC page skeleton

commit 0b3d75d9bb1116e6161b636a5b01d398202ac7a2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:38:00 2022 +0200

    swhgraph.proto: small documentation fixes

commit cbb52ff80b97d02d1541a7a39d5e5f85c3c7da3e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 01:44:01 2022 +0200

    More protobuf/grpc documentation, better field names

commit c7559ac55f7cd0021c0818097bbfc9b39528d08a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 20:11:13 2022 +0200

    Document protobuf/grpc services and fields

commit c241e13548f21269592a144883e673388c6fc2fd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:54:13 2022 +0200

    Traversal: add max edge tests

commit 9f83ed730669d2e7b9dbe2bac678db4699ab941a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:30:06 2022 +0200

    Traversal: add max depth and common ancestors tests

commit 6e16ec9d757973c4c7bcee9830f80ff7473cf4ac
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:46:21 2022 +0200

    Add tests for CountNodes/CountEdges

commit 60e93f13390ce6ca51e109476a7959da1bb7e0a2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:04:21 2022 +0200

    Remove CheckSwhid, use GetNode instead

commit c117623b3c26525e9ca51d96fa41a69f8843e85c
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:38:51 2022 +0200

    Traversal: simplify the StopTraversal logic

commit a2c35364b57a0de75397d4b09c310a553fb7036f
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:35:32 2022 +0200

    Traversal: check for invalid arguments

commit 70fc951eac997c4a64eff090578d8f1632e384a7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:16:50 2022 +0200

    Traversal: test impossible paths

commit 83be48e06673ccdb46bdeacf60c086fa8eda2797
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:30:11 2022 +0200

    Traversal: test traversals from multiple sources/to multiple dests

commit eefdd910c43cd36af0e0190d03ef65eb81426064
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:28:42 2022 +0200

    Remove GraphDirection.BOTH (labelled iteration not supported yet)

commit 383ed27418f9d4750d28438a1c1a8882a12fc0bd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon Jun 20 12:13:17 2022 +0200

    Add FindPathTo and FindPathBetween endpoints
    
    - FindPathTo takes a list of sources and a criteria for a destination.
    - FindPathBetween takes a list of sources and a list of destinations.

commit 3649bd634e61452a0d171f3397cb7d3340f64dfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 16 00:29:10 2022 +0200

    proto: migrate to FieldMask to filter out fields

commit e7fb41e4cd99a1ec22e3ffc35c8c2d50dc4bfbb0
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 15 15:05:25 2022 +0200

    Add rpc.StatsTest

commit 131e7165ff726791e1fe275230793494e53fd86e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:35 2022 +0200

    mypy.ini: ignore grpc generated files

commit ccc6b295a387b3ce75a9dcd527eeb4e9ae1b1bfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:17 2022 +0200

    requirements-test: add grpc-stubs

commit 5d36ca6eaffa0215951acda4073cf7b1293c3d0d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:31:59 2022 +0200

    flake: exclude swh/graph/rpc dir

commit 4e592de704c77d0b90fdeaabbf2cdfa134f8cfde
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:08:09 2022 +0200

    Reorganize Python files (RPC/HTTP server distinction)

commit 6d63a39642bad3ae3984a59af0fa7126e72f82ef
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:41:45 2022 +0200

    Remove now useless dot.py

commit a4d1eb10793f3f2c061ba48afd8334ca8562e27a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:23:30 2022 +0200

    Remove now useless backend.py

commit 9eaccde2cadb42a1373630413c6ce4b55758db2b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:37:17 2022 +0200

    NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile

commit 5c08b117f3369cab2b62260249257fc5c3098874
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:05:47 2022 +0200

    Remove old Java HTTP server, replaced by GRPC

commit 822f03de2161057d3b29375d0ec7b20a39608eb4
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 18:37:44 2022 +0200

    Java tests: migrate to GRPC

commit f098232f4282e05606f433ccf10d9c0c66fd384b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 15:53:14 2022 +0200

    Python: migrate HTTP API to GRPC

commit 208f5dc60128d0e6bdc23f0277a59a80fe39c219
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:42:53 2022 +0200

    Java: add proto/ dir symlink

commit 431d98de6ae4b79cc9691d0c70f7bc503a529168
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:40:55 2022 +0200

    RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work yet

commit fd61a31ef0c4b5afd29545c751b9a7543f5a7de6
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:31:44 2022 +0200

    Add CheckSwhid() and Stats() RPC methods

commit 0b4fd29e3b14fd398271f4a52fb2ebd620044ab7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:06:33 2022 +0200

    GRPC: initial commit with protobuf + java server

commit b61ca2d1ef44fe5591f01499bdfeccf36c744e3d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:51 2022 +0200

    SwhUnidirectionalGraph: make constructor with properties public

commit 066edad7cc361721a83ca33b92479a119357ef43
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:21 2022 +0200

    SwhGraphProperties: remove useless IOExceptions

See https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/202/ for more details.

  • doc: First full draft of GRPC API documentation

Build is green

Patch application report for D7890 (id=28928)

Rebasing onto 89975d2372...

Current branch diff-target is up to date.
Changes applied before test
commit 83866a0441f9fe3402d2ed1e3ac527efcd37e00a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 17:36:52 2022 +0200

    doc: First full draft of GRPC API documentation

commit a487585886553aa05658e91f9a3b43e1f071d7bc
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:39:56 2022 +0200

    doc: add GRPC page skeleton

commit 0b3d75d9bb1116e6161b636a5b01d398202ac7a2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:38:00 2022 +0200

    swhgraph.proto: small documentation fixes

commit cbb52ff80b97d02d1541a7a39d5e5f85c3c7da3e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 01:44:01 2022 +0200

    More protobuf/grpc documentation, better field names

commit c7559ac55f7cd0021c0818097bbfc9b39528d08a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 20:11:13 2022 +0200

    Document protobuf/grpc services and fields

commit c241e13548f21269592a144883e673388c6fc2fd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:54:13 2022 +0200

    Traversal: add max edge tests

commit 9f83ed730669d2e7b9dbe2bac678db4699ab941a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:30:06 2022 +0200

    Traversal: add max depth and common ancestors tests

commit 6e16ec9d757973c4c7bcee9830f80ff7473cf4ac
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:46:21 2022 +0200

    Add tests for CountNodes/CountEdges

commit 60e93f13390ce6ca51e109476a7959da1bb7e0a2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:04:21 2022 +0200

    Remove CheckSwhid, use GetNode instead

commit c117623b3c26525e9ca51d96fa41a69f8843e85c
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:38:51 2022 +0200

    Traversal: simplify the StopTraversal logic

commit a2c35364b57a0de75397d4b09c310a553fb7036f
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:35:32 2022 +0200

    Traversal: check for invalid arguments

commit 70fc951eac997c4a64eff090578d8f1632e384a7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:16:50 2022 +0200

    Traversal: test impossible paths

commit 83be48e06673ccdb46bdeacf60c086fa8eda2797
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:30:11 2022 +0200

    Traversal: test traversals from multiple sources/to multiple dests

commit eefdd910c43cd36af0e0190d03ef65eb81426064
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:28:42 2022 +0200

    Remove GraphDirection.BOTH (labelled iteration not supported yet)

commit 383ed27418f9d4750d28438a1c1a8882a12fc0bd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon Jun 20 12:13:17 2022 +0200

    Add FindPathTo and FindPathBetween endpoints
    
    - FindPathTo takes a list of sources and a criteria for a destination.
    - FindPathBetween takes a list of sources and a list of destinations.

commit 3649bd634e61452a0d171f3397cb7d3340f64dfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 16 00:29:10 2022 +0200

    proto: migrate to FieldMask to filter out fields

commit e7fb41e4cd99a1ec22e3ffc35c8c2d50dc4bfbb0
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 15 15:05:25 2022 +0200

    Add rpc.StatsTest

commit 131e7165ff726791e1fe275230793494e53fd86e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:35 2022 +0200

    mypy.ini: ignore grpc generated files

commit ccc6b295a387b3ce75a9dcd527eeb4e9ae1b1bfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:17 2022 +0200

    requirements-test: add grpc-stubs

commit 5d36ca6eaffa0215951acda4073cf7b1293c3d0d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:31:59 2022 +0200

    flake: exclude swh/graph/rpc dir

commit 4e592de704c77d0b90fdeaabbf2cdfa134f8cfde
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:08:09 2022 +0200

    Reorganize Python files (RPC/HTTP server distinction)

commit 6d63a39642bad3ae3984a59af0fa7126e72f82ef
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:41:45 2022 +0200

    Remove now useless dot.py

commit a4d1eb10793f3f2c061ba48afd8334ca8562e27a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:23:30 2022 +0200

    Remove now useless backend.py

commit 9eaccde2cadb42a1373630413c6ce4b55758db2b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:37:17 2022 +0200

    NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile

commit 5c08b117f3369cab2b62260249257fc5c3098874
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:05:47 2022 +0200

    Remove old Java HTTP server, replaced by GRPC

commit 822f03de2161057d3b29375d0ec7b20a39608eb4
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 18:37:44 2022 +0200

    Java tests: migrate to GRPC

commit f098232f4282e05606f433ccf10d9c0c66fd384b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 15:53:14 2022 +0200

    Python: migrate HTTP API to GRPC

commit 208f5dc60128d0e6bdc23f0277a59a80fe39c219
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:42:53 2022 +0200

    Java: add proto/ dir symlink

commit 431d98de6ae4b79cc9691d0c70f7bc503a529168
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:40:55 2022 +0200

    RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work yet

commit fd61a31ef0c4b5afd29545c751b9a7543f5a7de6
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:31:44 2022 +0200

    Add CheckSwhid() and Stats() RPC methods

commit 0b4fd29e3b14fd398271f4a52fb2ebd620044ab7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:06:33 2022 +0200

    GRPC: initial commit with protobuf + java server

commit b61ca2d1ef44fe5591f01499bdfeccf36c744e3d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:51 2022 +0200

    SwhUnidirectionalGraph: make constructor with properties public

commit 066edad7cc361721a83ca33b92479a119357ef43
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:21 2022 +0200

    SwhGraphProperties: remove useless IOExceptions

See https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/203/ for more details.

  • Add inline docstrings to Java GRPC server

Build is green

Patch application report for D7890 (id=28929)

Rebasing onto 89975d2372...

Current branch diff-target is up to date.
Changes applied before test
commit 98fce8be99f5ca8bce784208d6074acd5607667e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 18:15:53 2022 +0200

    Add inline docstrings to Java GRPC server

commit b58cb841b6a1cacbd290800b659774099c843a17
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 17:36:52 2022 +0200

    doc: First full draft of GRPC API documentation

commit e6e6866279da76fbadbc41bead968ad357138fe4
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:39:56 2022 +0200

    doc: add GRPC page skeleton

commit 2f149363d21f097abac13bb384cbed7abfaf0b62
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:38:00 2022 +0200

    swhgraph.proto: small documentation fixes

commit 328f26e31ccf207d0894a045bef79a5b3f13c7d2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 01:44:01 2022 +0200

    More protobuf/grpc documentation, better field names

commit c7559ac55f7cd0021c0818097bbfc9b39528d08a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 20:11:13 2022 +0200

    Document protobuf/grpc services and fields

commit c241e13548f21269592a144883e673388c6fc2fd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:54:13 2022 +0200

    Traversal: add max edge tests

commit 9f83ed730669d2e7b9dbe2bac678db4699ab941a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:30:06 2022 +0200

    Traversal: add max depth and common ancestors tests

commit 6e16ec9d757973c4c7bcee9830f80ff7473cf4ac
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:46:21 2022 +0200

    Add tests for CountNodes/CountEdges

commit 60e93f13390ce6ca51e109476a7959da1bb7e0a2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:04:21 2022 +0200

    Remove CheckSwhid, use GetNode instead

commit c117623b3c26525e9ca51d96fa41a69f8843e85c
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:38:51 2022 +0200

    Traversal: simplify the StopTraversal logic

commit a2c35364b57a0de75397d4b09c310a553fb7036f
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:35:32 2022 +0200

    Traversal: check for invalid arguments

commit 70fc951eac997c4a64eff090578d8f1632e384a7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:16:50 2022 +0200

    Traversal: test impossible paths

commit 83be48e06673ccdb46bdeacf60c086fa8eda2797
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:30:11 2022 +0200

    Traversal: test traversals from multiple sources/to multiple dests

commit eefdd910c43cd36af0e0190d03ef65eb81426064
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:28:42 2022 +0200

    Remove GraphDirection.BOTH (labelled iteration not supported yet)

commit 383ed27418f9d4750d28438a1c1a8882a12fc0bd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon Jun 20 12:13:17 2022 +0200

    Add FindPathTo and FindPathBetween endpoints
    
    - FindPathTo takes a list of sources and a criteria for a destination.
    - FindPathBetween takes a list of sources and a list of destinations.

commit 3649bd634e61452a0d171f3397cb7d3340f64dfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 16 00:29:10 2022 +0200

    proto: migrate to FieldMask to filter out fields

commit e7fb41e4cd99a1ec22e3ffc35c8c2d50dc4bfbb0
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 15 15:05:25 2022 +0200

    Add rpc.StatsTest

commit 131e7165ff726791e1fe275230793494e53fd86e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:35 2022 +0200

    mypy.ini: ignore grpc generated files

commit ccc6b295a387b3ce75a9dcd527eeb4e9ae1b1bfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:17 2022 +0200

    requirements-test: add grpc-stubs

commit 5d36ca6eaffa0215951acda4073cf7b1293c3d0d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:31:59 2022 +0200

    flake: exclude swh/graph/rpc dir

commit 4e592de704c77d0b90fdeaabbf2cdfa134f8cfde
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:08:09 2022 +0200

    Reorganize Python files (RPC/HTTP server distinction)

commit 6d63a39642bad3ae3984a59af0fa7126e72f82ef
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:41:45 2022 +0200

    Remove now useless dot.py

commit a4d1eb10793f3f2c061ba48afd8334ca8562e27a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:23:30 2022 +0200

    Remove now useless backend.py

commit 9eaccde2cadb42a1373630413c6ce4b55758db2b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:37:17 2022 +0200

    NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile

commit 5c08b117f3369cab2b62260249257fc5c3098874
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:05:47 2022 +0200

    Remove old Java HTTP server, replaced by GRPC

commit 822f03de2161057d3b29375d0ec7b20a39608eb4
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 18:37:44 2022 +0200

    Java tests: migrate to GRPC

commit f098232f4282e05606f433ccf10d9c0c66fd384b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 15:53:14 2022 +0200

    Python: migrate HTTP API to GRPC

commit 208f5dc60128d0e6bdc23f0277a59a80fe39c219
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:42:53 2022 +0200

    Java: add proto/ dir symlink

commit 431d98de6ae4b79cc9691d0c70f7bc503a529168
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:40:55 2022 +0200

    RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work yet

commit fd61a31ef0c4b5afd29545c751b9a7543f5a7de6
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:31:44 2022 +0200

    Add CheckSwhid() and Stats() RPC methods

commit 0b4fd29e3b14fd398271f4a52fb2ebd620044ab7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:06:33 2022 +0200

    GRPC: initial commit with protobuf + java server

commit b61ca2d1ef44fe5591f01499bdfeccf36c744e3d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:51 2022 +0200

    SwhUnidirectionalGraph: make constructor with properties public

commit 066edad7cc361721a83ca33b92479a119357ef43
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:21 2022 +0200

    SwhGraphProperties: remove useless IOExceptions

See https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/204/ for more details.

I've read through the documentation so far and it's quite good, thanks.
A couple of non-blocking questions and remarks for that.

I'm a tad out of my depth regarding the java part for now though (which i've not read entirely yet).

docs/grpc-api.rst
313 ↗(On Diff #28929)

why not condense the label and the content itself when streaming the successors?
And let client use fieldmask to filter if they don't want it?

337 ↗(On Diff #28929)

Keep the same order you gave in the previous paragraph (or even drop it since you already mentionedi it).

358 ↗(On Diff #28929)
391 ↗(On Diff #28929)

And to continue streaming results, can we do a query starting from the last result streamed by the first query?

If so, it may be worth mentioning it?

412 ↗(On Diff #28929)

awesome, some part of provenance in one query!

553 ↗(On Diff #28929)

cool stuff (^ overall document), thanks!

java/pom.xml
120

indentation seems off in regards to previous deps.

383

lol, maven triggers the makefile ;)

(we are always getting back to a makefile)

java/src/main/java/org/softwareheritage/graph/rpc/GraphServer.java
1

copyright headers ¯\_(ツ)_/¯.

58

Could be worth adding statsd metrics around the time of each step?

75

Seems unused in the rest of the class.

docs/grpc-api.rst
313 ↗(On Diff #28929)

to clarify "condense" i meant to provide the content of the cnt {} block at the same time of the successor response.

To which seirl replied:

11:16 <+seirl> ardumont: thanks! could you explain what you mean by "condense the label and the content"?
11:16 <+ardumont> i meant i see 3 successive messages
11:16 <+ardumont> but in the first one there is already the cnt with its label
11:16 <+seirl> ah, okay
11:16 <+ardumont> can't you add the other properties as well in the first response
11:16 <+ardumont> ?
11:17 <+ardumont> in another block {}
11:17 <+seirl> so, forget about the successors field for a second
11:17 <+ardumont> (so the fieldmask can still be used)
11:17 <+seirl> the idea is to perform a graph traversal and stream all the nodes as i'm seeing them
11:18 <+ardumont> yeah, i got that
11:18 <+seirl> the only problem is that if i just stream the source swhid, i lose the information of what the edges are
11:18 <+seirl> so i also need to give the src -> dst information somewhere (potentially with a label)
11:19 <+seirl> that's what the succesor field is for
11:19 <+seirl> but when i'm in the successor field, i'm still processing the *previous* node
11:20 <+seirl> so i don't want to add any properties there, because i know the node will get streamed later
11:20 <+ardumont> ack
11:20 <+ardumont> thx
seirl added inline comments.
docs/grpc-api.rst
391 ↗(On Diff #28929)

There's no special mechanism to do that, but you can just set src to the previous results if you want to.

java/src/main/java/org/softwareheritage/graph/rpc/GraphServer.java
1

There's another task for copyright headers, and I already have a branch that adds them everywhere at once, but since it's irrelevant for this PR in particular I wanted to wait for it to be merge to add the headers everywhere.

75

start is used in the main function and stop is used in the start function.

well, this overall looks good to me.

I've only schemed through the deep internal java stuff but the overall surface (including protobuf) looks well in sync with the very well written documentation.

So, i think it's fine to land this.

proto/swhgraph.proto
1

copyright headers again ;)

201

can't there be dir as well?

This revision is now accepted and ready to land.Jun 24 2022, 11:51 AM
seirl added inline comments.
proto/swhgraph.proto
1

(See my other comment on copyright headers)

201

There's no data for directories and releases.

seirl added inline comments.
java/src/main/java/org/softwareheritage/graph/rpc/GraphServer.java
58

Good idea but adding the statsd machinery to Java is outside the scope of the diff

seirl marked 3 inline comments as done.
  • Remove old useless classes, including Traversal
  • Regenerate python protobuf documentation
  • style: typos and indent fixes

Build is green

Patch application report for D7890 (id=28932)

Rebasing onto 89975d2372...

Current branch diff-target is up to date.
Changes applied before test
commit a6e02f726926503b5bc36f92b49992e00dd46c45
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Fri Jun 24 12:28:31 2022 +0200

    style: typos and indent fixes

commit 3ce74dfd2a013e9332b8624d406c617c8fb8882d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 19:19:28 2022 +0200

    Regenerate python protobuf documentation

commit ab895e3a0b532bb1bcdd03c905c2d357e9d00570
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 18:37:56 2022 +0200

    Remove old useless classes, including Traversal

commit 98fce8be99f5ca8bce784208d6074acd5607667e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 18:15:53 2022 +0200

    Add inline docstrings to Java GRPC server

commit b58cb841b6a1cacbd290800b659774099c843a17
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 17:36:52 2022 +0200

    doc: First full draft of GRPC API documentation

commit e6e6866279da76fbadbc41bead968ad357138fe4
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:39:56 2022 +0200

    doc: add GRPC page skeleton

commit 2f149363d21f097abac13bb384cbed7abfaf0b62
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:38:00 2022 +0200

    swhgraph.proto: small documentation fixes

commit 328f26e31ccf207d0894a045bef79a5b3f13c7d2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 01:44:01 2022 +0200

    More protobuf/grpc documentation, better field names

commit c7559ac55f7cd0021c0818097bbfc9b39528d08a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 20:11:13 2022 +0200

    Document protobuf/grpc services and fields

commit c241e13548f21269592a144883e673388c6fc2fd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:54:13 2022 +0200

    Traversal: add max edge tests

commit 9f83ed730669d2e7b9dbe2bac678db4699ab941a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:30:06 2022 +0200

    Traversal: add max depth and common ancestors tests

commit 6e16ec9d757973c4c7bcee9830f80ff7473cf4ac
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:46:21 2022 +0200

    Add tests for CountNodes/CountEdges

commit 60e93f13390ce6ca51e109476a7959da1bb7e0a2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:04:21 2022 +0200

    Remove CheckSwhid, use GetNode instead

commit c117623b3c26525e9ca51d96fa41a69f8843e85c
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:38:51 2022 +0200

    Traversal: simplify the StopTraversal logic

commit a2c35364b57a0de75397d4b09c310a553fb7036f
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:35:32 2022 +0200

    Traversal: check for invalid arguments

commit 70fc951eac997c4a64eff090578d8f1632e384a7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:16:50 2022 +0200

    Traversal: test impossible paths

commit 83be48e06673ccdb46bdeacf60c086fa8eda2797
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:30:11 2022 +0200

    Traversal: test traversals from multiple sources/to multiple dests

commit eefdd910c43cd36af0e0190d03ef65eb81426064
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:28:42 2022 +0200

    Remove GraphDirection.BOTH (labelled iteration not supported yet)

commit 383ed27418f9d4750d28438a1c1a8882a12fc0bd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon Jun 20 12:13:17 2022 +0200

    Add FindPathTo and FindPathBetween endpoints
    
    - FindPathTo takes a list of sources and a criteria for a destination.
    - FindPathBetween takes a list of sources and a list of destinations.

commit 3649bd634e61452a0d171f3397cb7d3340f64dfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 16 00:29:10 2022 +0200

    proto: migrate to FieldMask to filter out fields

commit e7fb41e4cd99a1ec22e3ffc35c8c2d50dc4bfbb0
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 15 15:05:25 2022 +0200

    Add rpc.StatsTest

commit 131e7165ff726791e1fe275230793494e53fd86e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:35 2022 +0200

    mypy.ini: ignore grpc generated files

commit ccc6b295a387b3ce75a9dcd527eeb4e9ae1b1bfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:17 2022 +0200

    requirements-test: add grpc-stubs

commit 5d36ca6eaffa0215951acda4073cf7b1293c3d0d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:31:59 2022 +0200

    flake: exclude swh/graph/rpc dir

commit 4e592de704c77d0b90fdeaabbf2cdfa134f8cfde
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:08:09 2022 +0200

    Reorganize Python files (RPC/HTTP server distinction)

commit 6d63a39642bad3ae3984a59af0fa7126e72f82ef
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:41:45 2022 +0200

    Remove now useless dot.py

commit a4d1eb10793f3f2c061ba48afd8334ca8562e27a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:23:30 2022 +0200

    Remove now useless backend.py

commit 9eaccde2cadb42a1373630413c6ce4b55758db2b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:37:17 2022 +0200

    NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile

commit 5c08b117f3369cab2b62260249257fc5c3098874
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:05:47 2022 +0200

    Remove old Java HTTP server, replaced by GRPC

commit 822f03de2161057d3b29375d0ec7b20a39608eb4
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 18:37:44 2022 +0200

    Java tests: migrate to GRPC

commit f098232f4282e05606f433ccf10d9c0c66fd384b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 15:53:14 2022 +0200

    Python: migrate HTTP API to GRPC

commit 208f5dc60128d0e6bdc23f0277a59a80fe39c219
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:42:53 2022 +0200

    Java: add proto/ dir symlink

commit 431d98de6ae4b79cc9691d0c70f7bc503a529168
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:40:55 2022 +0200

    RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work yet

commit fd61a31ef0c4b5afd29545c751b9a7543f5a7de6
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:31:44 2022 +0200

    Add CheckSwhid() and Stats() RPC methods

commit 0b4fd29e3b14fd398271f4a52fb2ebd620044ab7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:06:33 2022 +0200

    GRPC: initial commit with protobuf + java server

commit b61ca2d1ef44fe5591f01499bdfeccf36c744e3d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:51 2022 +0200

    SwhUnidirectionalGraph: make constructor with properties public

commit 066edad7cc361721a83ca33b92479a119357ef43
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:21 2022 +0200

    SwhGraphProperties: remove useless IOExceptions

See https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/205/ for more details.

java/src/main/java/org/softwareheritage/graph/rpc/GraphServer.java
58

yeah, totally for later.

I realized i did not explicit the 'for later' part.

75

d'oh i recall having looked for main and not start, nvm then.

java/src/main/java/org/softwareheritage/graph/rpc/GraphServer.java
1

ack

  • SwhBidirectionalGraph: fix copy() not actually copying subgraphs
  • GetNode: make endpoint thread-safe with a lightweight copy of the graph
  • GraphServer: Use nproc threads in threadpool by default
  • Update webgraph-big to 3.7.0
This revision was landed with ongoing or failed builds.Jun 29 2022, 12:07 PM
This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.

Build is green

Patch application report for D7890 (id=28990)

Rebasing onto 89975d2372...

Current branch diff-target is up to date.
Changes applied before test
commit 43a3e3302abe827923ce9cdb1509ba35893f423b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 29 12:01:49 2022 +0200

    Update webgraph-big to 3.7.0

commit 53b7d553ba062442e02890d81fe3b30e5e75d969
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Fri Jun 24 19:43:37 2022 +0200

    GraphServer: Use nproc threads in threadpool by default

commit f528aa80db5b9676f54ffb78422e3b34b02a9a09
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Fri Jun 24 18:29:34 2022 +0200

    GetNode: make endpoint thread-safe with a lightweight copy of the graph

commit d89f35478012c3ccfc8ef4f6d72161d9acf84e0e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Fri Jun 24 18:28:05 2022 +0200

    SwhBidirectionalGraph: fix copy() not actually copying subgraphs

commit a6e02f726926503b5bc36f92b49992e00dd46c45
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Fri Jun 24 12:28:31 2022 +0200

    style: typos and indent fixes

commit 3ce74dfd2a013e9332b8624d406c617c8fb8882d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 19:19:28 2022 +0200

    Regenerate python protobuf documentation

commit ab895e3a0b532bb1bcdd03c905c2d357e9d00570
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 18:37:56 2022 +0200

    Remove old useless classes, including Traversal

commit 98fce8be99f5ca8bce784208d6074acd5607667e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 18:15:53 2022 +0200

    Add inline docstrings to Java GRPC server

commit b58cb841b6a1cacbd290800b659774099c843a17
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 17:36:52 2022 +0200

    doc: First full draft of GRPC API documentation

commit e6e6866279da76fbadbc41bead968ad357138fe4
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:39:56 2022 +0200

    doc: add GRPC page skeleton

commit 2f149363d21f097abac13bb384cbed7abfaf0b62
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 12:38:00 2022 +0200

    swhgraph.proto: small documentation fixes

commit 328f26e31ccf207d0894a045bef79a5b3f13c7d2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 23 01:44:01 2022 +0200

    More protobuf/grpc documentation, better field names

commit c7559ac55f7cd0021c0818097bbfc9b39528d08a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 20:11:13 2022 +0200

    Document protobuf/grpc services and fields

commit c241e13548f21269592a144883e673388c6fc2fd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:54:13 2022 +0200

    Traversal: add max edge tests

commit 9f83ed730669d2e7b9dbe2bac678db4699ab941a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 18:30:06 2022 +0200

    Traversal: add max depth and common ancestors tests

commit 6e16ec9d757973c4c7bcee9830f80ff7473cf4ac
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:46:21 2022 +0200

    Add tests for CountNodes/CountEdges

commit 60e93f13390ce6ca51e109476a7959da1bb7e0a2
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 17:04:21 2022 +0200

    Remove CheckSwhid, use GetNode instead

commit c117623b3c26525e9ca51d96fa41a69f8843e85c
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:38:51 2022 +0200

    Traversal: simplify the StopTraversal logic

commit a2c35364b57a0de75397d4b09c310a553fb7036f
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:35:32 2022 +0200

    Traversal: check for invalid arguments

commit 70fc951eac997c4a64eff090578d8f1632e384a7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 16:16:50 2022 +0200

    Traversal: test impossible paths

commit 83be48e06673ccdb46bdeacf60c086fa8eda2797
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:30:11 2022 +0200

    Traversal: test traversals from multiple sources/to multiple dests

commit eefdd910c43cd36af0e0190d03ef65eb81426064
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 22 15:28:42 2022 +0200

    Remove GraphDirection.BOTH (labelled iteration not supported yet)

commit 383ed27418f9d4750d28438a1c1a8882a12fc0bd
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon Jun 20 12:13:17 2022 +0200

    Add FindPathTo and FindPathBetween endpoints
    
    - FindPathTo takes a list of sources and a criteria for a destination.
    - FindPathBetween takes a list of sources and a list of destinations.

commit 3649bd634e61452a0d171f3397cb7d3340f64dfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Thu Jun 16 00:29:10 2022 +0200

    proto: migrate to FieldMask to filter out fields

commit e7fb41e4cd99a1ec22e3ffc35c8c2d50dc4bfbb0
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Jun 15 15:05:25 2022 +0200

    Add rpc.StatsTest

commit 131e7165ff726791e1fe275230793494e53fd86e
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:35 2022 +0200

    mypy.ini: ignore grpc generated files

commit ccc6b295a387b3ce75a9dcd527eeb4e9ae1b1bfb
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 30 14:05:17 2022 +0200

    requirements-test: add grpc-stubs

commit 5d36ca6eaffa0215951acda4073cf7b1293c3d0d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:31:59 2022 +0200

    flake: exclude swh/graph/rpc dir

commit 4e592de704c77d0b90fdeaabbf2cdfa134f8cfde
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 21:08:09 2022 +0200

    Reorganize Python files (RPC/HTTP server distinction)

commit 6d63a39642bad3ae3984a59af0fa7126e72f82ef
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:41:45 2022 +0200

    Remove now useless dot.py

commit a4d1eb10793f3f2c061ba48afd8334ca8562e27a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 20:23:30 2022 +0200

    Remove now useless backend.py

commit 9eaccde2cadb42a1373630413c6ce4b55758db2b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:37:17 2022 +0200

    NodeIdMap: use new *MappedBigList classes, remove deprecated classes and MapFile

commit 5c08b117f3369cab2b62260249257fc5c3098874
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 19:05:47 2022 +0200

    Remove old Java HTTP server, replaced by GRPC

commit 822f03de2161057d3b29375d0ec7b20a39608eb4
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 18:37:44 2022 +0200

    Java tests: migrate to GRPC

commit f098232f4282e05606f433ccf10d9c0c66fd384b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 15:53:14 2022 +0200

    Python: migrate HTTP API to GRPC

commit 208f5dc60128d0e6bdc23f0277a59a80fe39c219
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:42:53 2022 +0200

    Java: add proto/ dir symlink

commit 431d98de6ae4b79cc9691d0c70f7bc503a529168
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:40:55 2022 +0200

    RPC: temporarily use loadLabelled() because loadLabelledMapped() doesn't work yet

commit fd61a31ef0c4b5afd29545c751b9a7543f5a7de6
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Tue May 24 01:31:44 2022 +0200

    Add CheckSwhid() and Stats() RPC methods

commit 0b4fd29e3b14fd398271f4a52fb2ebd620044ab7
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:06:33 2022 +0200

    GRPC: initial commit with protobuf + java server

commit b61ca2d1ef44fe5591f01499bdfeccf36c744e3d
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:51 2022 +0200

    SwhUnidirectionalGraph: make constructor with properties public

commit 066edad7cc361721a83ca33b92479a119357ef43
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Mon May 23 23:05:21 2022 +0200

    SwhGraphProperties: remove useless IOExceptions

See https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/206/ for more details.