Changeset View
Standalone View
docs/grpc-api.rst
Show First 20 Lines • Show All 79 Lines • ▼ Show 20 Lines | .. code-block:: console | ||||
{ | { | ||||
"numNodes": "21", | "numNodes": "21", | ||||
"numEdges": "23", | "numEdges": "23", | ||||
[...] | [...] | ||||
} | } | ||||
Rpc succeeded with OK status | Rpc succeeded with OK status | ||||
Or, in Python: | |||||
.. code-block:: python | |||||
import grpc | |||||
import swh.graph.grpc.swhgraph_pb2 as swhgraph | |||||
import swh.graph.grpc.swhgraph_pb2_grpc as swhgraph_grpc | |||||
GRAPH_GRPC_SERVER = "granet.internal.softwareheritage.org:50091" | |||||
with grpc.insecure_channel(GRAPH_GRPC_SERVER) as channel: | |||||
stub = swhgraph_grpc.TraversalServiceStub(channel) | |||||
response = stub.Stats(swhgraph.StatsRequest()) | |||||
print(response) | |||||
anlambert: add spaces around `*` operator | |||||
print("Compression ratio:", response.compression_ratio * 100, "%") | |||||
which prints: | |||||
.. code-block:: | |||||
num_nodes: 25340003875 | |||||
num_edges: 359467940510 | |||||
compression_ratio: 0.096 | |||||
bits_per_node: 43.993 | |||||
bits_per_edge: 3.101 | |||||
avg_locality: 1030367242.935 | |||||
indegree_max: 381552037 | |||||
indegree_avg: 14.185788695346046 | |||||
outdegree_max: 1033207 | |||||
outdegree_avg: 14.185788695346046 | |||||
Compression ratio: 9.6 % | |||||
**Note**: grpc_cli's outputs in this document are slightly modified for | **Note**: grpc_cli's outputs in this document are slightly modified for | ||||
readability's sake. | readability's sake. | ||||
Simple queries | Simple queries | ||||
============== | ============== | ||||
For a full documentation of all the endpoints, as well as the request and | For a full documentation of all the endpoints, as well as the request and | ||||
response messages, see :ref:`swh-graph-grpc-api-protobuf`. | response messages, see :ref:`swh-graph-grpc-api-protobuf`. | ||||
All Python examples below assume they are run in the following context: | |||||
.. code-block:: python | |||||
import grpc | |||||
from google.protobuf.field_mask_pb2 import FieldMask | |||||
import swh.graph.grpc.swhgraph_pb2 as swhgraph | |||||
import swh.graph.grpc.swhgraph_pb2_grpc as swhgraph_grpc | |||||
GRAPH_GRPC_SERVER = "granet.internal.softwareheritage.org:50091" | |||||
with grpc.insecure_channel(GRAPH_GRPC_SERVER) as channel: | |||||
stub = swhgraph_grpc.TraversalServiceStub(channel) | |||||
pass # <insert snippet here> | |||||
Querying a single node | Querying a single node | ||||
---------------------- | ---------------------- | ||||
The **GetNode** endpoint can be used to return information on a single | The **GetNode** endpoint can be used to return information on a single | ||||
node of the graph, including all its node properties, from its SWHID. Here | node of the graph, including all its node properties, from its SWHID. Here | ||||
are a few examples from the test graph: | are a few examples from the test graph: | ||||
Content | Content | ||||
~~~~~~~ | ~~~~~~~ | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | ||||
'swhid: "swh:1:cnt:0000000000000000000000000000000000000001"' | 'swhid: "swh:1:cnt:0000000000000000000000000000000000000001"' | ||||
.. code-block:: python | |||||
swhid = "swh:1:cnt:0000000000000000000000000000000000000001" | |||||
response = stub.GetNode(swhgraph.GetNodeRequest(swhid=swhid)) | |||||
print(response) | |||||
Not Done Inline Actions"maybe" we could suggest a breakpoint (with pdbpp) or ipython in the code (as a note below) so people could investigate responseby themselves too. response.<tab><tab>... ardumont: "maybe" we could suggest a `breakpoint` (with `pdbpp`) or `ipython` in the code (as a note… | |||||
Done Inline Actionsmeh, this is going to bloat the examples vlorentz: meh, this is going to bloat the examples | |||||
Not Done Inline ActionsI meant as a one-off note in the main part, with the context at the beginning of the sampled python (not for all python examples...). ardumont: I meant as a one-off note in the main part, with the context at the beginning of the sampled… | |||||
# results will be in response.cnt.length and response.cnt.is_skipped | |||||
.. code-block:: javascript | .. code-block:: javascript | ||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000001" | swhid: "swh:1:cnt:0000000000000000000000000000000000000001" | ||||
cnt { | cnt { | ||||
length: 42 | length: 42 | ||||
is_skipped: false | is_skipped: false | ||||
} | } | ||||
Revision | Revision | ||||
~~~~~~~~ | ~~~~~~~~ | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | ||||
'swhid: "swh:1:rev:0000000000000000000000000000000000000009"' | 'swhid: "swh:1:rev:0000000000000000000000000000000000000009"' | ||||
.. code-block:: python | |||||
swhid = "swh:1:rev:0000000000000000000000000000000000000009" | |||||
response = stub.GetNode(swhgraph.GetNodeRequest(swhid=swhid)) | |||||
print(response) | |||||
# results will be in response.rev.author, response.rev.author_date, ... | |||||
.. code-block:: javascript | .. code-block:: javascript | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000009" | swhid: "swh:1:rev:0000000000000000000000000000000000000009" | ||||
rev { | rev { | ||||
author: 2 | author: 2 | ||||
author_date: 1111140840 | author_date: 1111140840 | ||||
author_date_offset: 120 | author_date_offset: 120 | ||||
committer: 2 | committer: 2 | ||||
committer_date: 1111151950 | committer_date: 1111151950 | ||||
committer_date_offset: 120 | committer_date_offset: 120 | ||||
message: "Add parser" | message: "Add parser" | ||||
} | } | ||||
Note that author and committer names are not available in the compressed graph, | |||||
so you must use either the :swh_web:`public API <1/revision/>` or swh-storage | |||||
directly to access them. | |||||
Release | Release | ||||
~~~~~~~ | ~~~~~~~ | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | ||||
'swhid: "swh:1:rel:0000000000000000000000000000000000000010"' | 'swhid: "swh:1:rel:0000000000000000000000000000000000000010"' | ||||
.. code-block:: python | |||||
swhid = "swh:1:rel:0000000000000000000000000000000000000010" | |||||
response = stub.GetNode(swhgraph.GetNodeRequest(swhid=swhid)) | |||||
print(response) | |||||
# results will be in response.rel.author, response.rel.author_date, ... | |||||
.. code-block:: javascript | .. code-block:: javascript | ||||
swhid: "swh:1:rel:0000000000000000000000000000000000000010" | swhid: "swh:1:rel:0000000000000000000000000000000000000010" | ||||
rel { | rel { | ||||
author: 0 | author: 0 | ||||
author_date: 1234564290 | author_date: 1234564290 | ||||
author_date_offset: 120 | author_date_offset: 120 | ||||
message: "Version 1.0" | message: "Version 1.0" | ||||
} | } | ||||
Origin | Origin | ||||
~~~~~~ | ~~~~~~ | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | ||||
'swhid: "swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054"' | 'swhid: "swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054"' | ||||
.. code-block:: python | |||||
swhid = "swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054" | |||||
response = stub.GetNode(swhgraph.GetNodeRequest(swhid=swhid)) | |||||
print(response) | |||||
# results will be in response.ori.url | |||||
.. code-block:: javascript | .. code-block:: javascript | ||||
swhid: "swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054" | swhid: "swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054" | ||||
ori { | ori { | ||||
url: "https://example.com/swh/graph" | url: "https://example.com/swh/graph" | ||||
} | } | ||||
Checking the presence of a node | Checking the presence of a node | ||||
------------------------------- | ------------------------------- | ||||
The **GetNode** endpoint can also be used to check if a node exists in the | The **GetNode** endpoint can also be used to check if a node exists in the | ||||
graph. The RPC will return the ``INVALID_ARGUMENT`` code, and a detailed error | graph. The RPC will return the ``INVALID_ARGUMENT`` code, and a detailed error | ||||
message. | message. | ||||
With ``grpc_cli``: | |||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | ||||
'swhid: "swh:1:ori:ffffffffffffffffffffffffffffffffffffffff"' | 'swhid: "swh:1:ori:ffffffffffffffffffffffffffffffffffffffff"' | ||||
Rpc failed with status code 3, error message: Unknown SWHID: swh:1:ori:ffffffffffffffffffffffffffffffffffffffff | Rpc failed with status code 3, error message: Unknown SWHID: swh:1:ori:ffffffffffffffffffffffffffffffffffffffff | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | ||||
'swhid: "invalidswhid"' | 'swhid: "invalidswhid"' | ||||
Rpc failed with status code 3, error message: malformed SWHID: invalidswhid | Rpc failed with status code 3, error message: malformed SWHID: invalidswhid | ||||
With Python: | |||||
.. code-block:: | |||||
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: | |||||
status = StatusCode.INVALID_ARGUMENT | |||||
details = "Unknown SWHID: swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054" | |||||
debug_error_string = "{"created":"@1666018913.304633417","description":"Error received from peer ipv4:192.168.100.51:50091","file":"src/core/lib/surface/call.cc","file_line":966,"grpc_message":"Unknown SWHID: swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054","grpc_status":3}" | |||||
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: | |||||
status = StatusCode.INVALID_ARGUMENT | |||||
details = "malformed SWHID: malformedswhid" | |||||
debug_error_string = "{"created":"@1666019057.270929623","description":"Error received from peer ipv4:192.168.100.51:50091","file":"src/core/lib/surface/call.cc","file_line":966,"grpc_message":"malformed SWHID: malformedswhid","grpc_status":3}" | |||||
Selecting returned fields with FieldMask | Selecting returned fields with FieldMask | ||||
---------------------------------------- | ---------------------------------------- | ||||
Many endpoints, including **GetNode**, contain a ``mask`` field of type | Many endpoints, including **GetNode**, contain a ``mask`` field of type | ||||
`FieldMask | `FieldMask | ||||
<https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/FieldMask.html>`_, | <https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/FieldMask.html>`_, | ||||
which can be used to select which fields should be returned in the response. | which can be used to select which fields should be returned in the response. | ||||
This is particularly interesting for traversal queries that return a large | This is particularly interesting for traversal queries that return a large | ||||
number of nodes, because property access is quite costly from the compressed | number of nodes, because property access is quite costly from the compressed | ||||
graph (at least compared to regular node access). It is therefore recommended | graph (at least compared to regular node access). It is therefore recommended | ||||
that clients systematically use FieldMasks to only request the properties that | that clients systematically use FieldMasks to only request the properties that | ||||
they will consume. | they will consume. | ||||
Not Done Inline ActionsThis makes me wonder whether the masks should be made mandatory (or default to an empty value, instead of the current "maximalist" default). olasd: This makes me wonder whether the masks should be made mandatory (or default to an empty value… | |||||
Done Inline ActionsOmitting masks is very convenient while debugging (eg. on a CLI) so we shouldn't do it on the server side IMO. And to do it on the client side, we would need a wrapper class in each language, which is meh. vlorentz: Omitting masks is very convenient while debugging (eg. on a CLI) so we shouldn't do it on the… | |||||
A FieldMask is represented as a set of "field paths" in dotted notation. For | A FieldMask is represented as a set of "field paths" in dotted notation. For | ||||
instance, ``paths: ["swhid", "rev.message"]`` will only request the swhid and | instance, ``paths: ["swhid", "rev.message"]`` will only request the swhid and | ||||
the message of a given node. An empty mask will return an empty object. | the message of a given node. An empty mask will return an empty object. | ||||
Example: | Examples: | ||||
**Just the SWHID**: | |||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | ||||
'swhid: "swh:1:rev:0000000000000000000000000000000000000009", mask: {paths: ["swhid"]}' | 'swhid: "swh:1:rev:0000000000000000000000000000000000000009", mask: {paths: ["swhid"]}' | ||||
.. code-block:: python | |||||
response = stub.GetNode(swhgraph.GetNodeRequest( | |||||
swhid="swh:1:rev:0000000000000000000000000000000000000009", | |||||
mask=FieldMask(paths=["swhid"]) | |||||
)) | |||||
print(response) | |||||
# Result is in response.swhid; other fields are omitted from the response as | |||||
# they are not part of the FieldMask. | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:rev:0000000000000000000000000000000000000009" | swhid: "swh:1:rev:0000000000000000000000000000000000000009" | ||||
**Multiple fields**: | |||||
.. code-block:: console | |||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.GetNode \ | ||||
'swhid: "swh:1:rev:0000000000000000000000000000000000000009", mask: {paths: ["swhid", "rev.message", "rev.author"]}' | 'swhid: "swh:1:rev:0000000000000000000000000000000000000009", mask: {paths: ["swhid", "rev.message", "rev.author"]}' | ||||
.. code-block:: python | |||||
response = stub.GetNode(swhgraph.GetNodeRequest( | |||||
swhid="swh:1:rev:0000000000000000000000000000000000000009", | |||||
mask=FieldMask(paths=["swhid", "rev.message", "rev.author"]) | |||||
)) | |||||
print(response) | |||||
Done Inline ActionsYou may want to mention that the "paths" values (in the request) are actually what's going to be mapped as a response (in the generic context you mentioned early). That seems to be a pattern from afar. ardumont: You may want to mention that the "paths" values (in the request) are actually what's going to… | |||||
Done Inline ActionsIsn't this already documented above? A FieldMask is represented as a set of "field paths" in dotted notation. For instance, ``paths: ["swhid", "rev.message"]`` will only request the swhid and the message of a given node. An empty mask will return an empty object. vlorentz: Isn't this already documented above?
```
A FieldMask is represented as a set of "field paths"… | |||||
Done Inline ActionsNo, i don't think that it is that explicit. ardumont: No, i don't think that it is that explicit.
But i'm fine either way. Like i said (non-blocking)… | |||||
# Results are in response.swhid, response.rev.message, and response.rev.author; | |||||
# other fields are omitted from the response as they are not part of the FieldMask. | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:rev:0000000000000000000000000000000000000009" | swhid: "swh:1:rev:0000000000000000000000000000000000000009" | ||||
rev { | rev { | ||||
author: 2 | author: 2 | ||||
message: "Add parser" | message: "Add parser" | ||||
} | } | ||||
Getting statistics on the graph | Getting statistics on the graph | ||||
------------------------------- | ------------------------------- | ||||
The **Stats** endpoint returns overall statistics on the entire compressed | The **Stats** endpoint returns overall statistics on the entire compressed | ||||
graph. Most notably, the total number of nodes and edges, as well as the | graph. Most notably, the total number of nodes and edges, as well as the | ||||
range of indegrees and outdegrees, and some compression-related statistics. | range of indegrees and outdegrees, and some compression-related statistics. | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli --json_output call localhost:50091 swh.graph.TraversalService.Stats "" | $ grpc_cli --json_output call localhost:50091 swh.graph.TraversalService.Stats "" | ||||
.. code-block:: python | |||||
response = stub.Stats(swhgraph.StatsRequest()) | |||||
print(response) | |||||
.. code-block:: json | .. code-block:: json | ||||
{ | { | ||||
"numNodes": "21", | "numNodes": "21", | ||||
"numEdges": "23", | "numEdges": "23", | ||||
"compression": 1.412, | "compression": 1.412, | ||||
"bitsPerNode": 8.524, | "bitsPerNode": 8.524, | ||||
"bitsPerEdge": 7.783, | "bitsPerEdge": 7.783, | ||||
Show All 21 Lines | |||||
For instance, here we run a traversal from a directory that contains two | For instance, here we run a traversal from a directory that contains two | ||||
contents: | contents: | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | ||||
"src: 'swh:1:dir:0000000000000000000000000000000000000006'" | "src: 'swh:1:dir:0000000000000000000000000000000000000006'" | ||||
.. code-block:: python | |||||
response = stub.Traverse(swhgraph.TraversalRequest( | |||||
src=["swh:1:dir:0000000000000000000000000000000000000006"] | |||||
)) | |||||
for item in response: | |||||
print(item) | |||||
We get the following stream of nodes: first, the source directory (including | We get the following stream of nodes: first, the source directory (including | ||||
its properties, successor list and their labels), then the contents themselves | its properties, successor list and their labels), then the contents themselves | ||||
and their respective properties. | and their respective properties. | ||||
.. code-block:: javascript | .. code-block:: javascript | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000006" | swhid: "swh:1:dir:0000000000000000000000000000000000000006" | ||||
successor { | successor { | ||||
Show All 30 Lines | |||||
Again, it is possible to use a FieldMask to restrict which fields get returned. | Again, it is possible to use a FieldMask to restrict which fields get returned. | ||||
For instance, if we only care about the SWHIDs: | For instance, if we only care about the SWHIDs: | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | ||||
"src: 'swh:1:dir:0000000000000000000000000000000000000006', mask: {paths: ['swhid']}" | "src: 'swh:1:dir:0000000000000000000000000000000000000006', mask: {paths: ['swhid']}" | ||||
.. code-block:: python | |||||
response = stub.Traverse(swhgraph.TraversalRequest( | |||||
src=["swh:1:dir:0000000000000000000000000000000000000006"], | |||||
mask=FieldMask(paths=["swhid"]) | |||||
)) | |||||
for item in response: | |||||
print(f'swhid: "{item.swhid}"') | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:dir:0000000000000000000000000000000000000006" | swhid: "swh:1:dir:0000000000000000000000000000000000000006" | ||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000005" | swhid: "swh:1:cnt:0000000000000000000000000000000000000005" | ||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000004" | swhid: "swh:1:cnt:0000000000000000000000000000000000000004" | ||||
Graph direction | Graph direction | ||||
~~~~~~~~~~~~~~~ | ~~~~~~~~~~~~~~~ | ||||
For many purposes, especially that of finding the provenance of software | For many purposes, especially that of finding the provenance of software | ||||
artifacts, it is useful to query the backward (or transposed) graph instead, | artifacts, it is useful to query the backward (or transposed) graph instead, | ||||
which is the same as the forward graph except all the edges are reversed. | which is the same as the forward graph except all the edges are reversed. | ||||
To achieve this, the ``direction`` field can be used to specify a direction | To achieve this, the ``direction`` field can be used to specify a direction | ||||
from the ``GraphDirection`` enum (either ``FORWARD`` or ``BACKWARD``). | from the ``GraphDirection`` enum (either ``FORWARD`` or ``BACKWARD``). | ||||
This query returns all the nodes reachable from a given directory in the | This query returns all the nodes reachable from a given directory in the | ||||
*backward* (or "transposed") graph: | *backward* (or "transposed") graph: | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | ||||
"src: 'swh:1:dir:0000000000000000000000000000000000000006', direction: BACKWARD, mask: {paths: ['swhid']}" | "src: 'swh:1:dir:0000000000000000000000000000000000000006', direction: BACKWARD, mask: {paths: ['swhid']}" | ||||
.. code-block:: python | |||||
response = stub.Traverse(swhgraph.TraversalRequest( | |||||
src=["swh:1:dir:0000000000000000000000000000000000000006"], | |||||
direction=swhgraph.GraphDirection.BACKWARD, | |||||
mask=FieldMask(paths=["swhid"]), | |||||
)) | |||||
for item in response: | |||||
print(f'swhid: "{item.swhid}"') | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:dir:0000000000000000000000000000000000000006" | swhid: "swh:1:dir:0000000000000000000000000000000000000006" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000008" | swhid: "swh:1:dir:0000000000000000000000000000000000000008" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000012" | swhid: "swh:1:dir:0000000000000000000000000000000000000012" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000009" | swhid: "swh:1:rev:0000000000000000000000000000000000000009" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000013" | swhid: "swh:1:rev:0000000000000000000000000000000000000013" | ||||
swhid: "swh:1:rel:0000000000000000000000000000000000000010" | swhid: "swh:1:rel:0000000000000000000000000000000000000010" | ||||
swhid: "swh:1:snp:0000000000000000000000000000000000000020" | swhid: "swh:1:snp:0000000000000000000000000000000000000020" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000018" | swhid: "swh:1:rev:0000000000000000000000000000000000000018" | ||||
Show All 13 Lines | |||||
This query traverses the parent revisions of a given revision only (i.e., it | This query traverses the parent revisions of a given revision only (i.e., it | ||||
outputs the *commit log* from a given commit): | outputs the *commit log* from a given commit): | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | ||||
"src: 'swh:1:rev:0000000000000000000000000000000000000018', edges: 'rev:rev', mask: {paths: ['swhid']}" | "src: 'swh:1:rev:0000000000000000000000000000000000000018', edges: 'rev:rev', mask: {paths: ['swhid']}" | ||||
.. code-block:: python | |||||
response = stub.Traverse(swhgraph.TraversalRequest( | |||||
src=["swh:1:rev:0000000000000000000000000000000000000018"], | |||||
edges="rev:rev", | |||||
mask=FieldMask(paths=["swhid"]), | |||||
)) | |||||
for item in response: | |||||
print(f'swhid: "{item.swhid}"') | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:rev:0000000000000000000000000000000000000018" | swhid: "swh:1:rev:0000000000000000000000000000000000000018" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000013" | swhid: "swh:1:rev:0000000000000000000000000000000000000013" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000009" | swhid: "swh:1:rev:0000000000000000000000000000000000000009" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000003" | swhid: "swh:1:rev:0000000000000000000000000000000000000003" | ||||
Limiting the traversal | Limiting the traversal | ||||
~~~~~~~~~~~~~~~~~~~~~~ | ~~~~~~~~~~~~~~~~~~~~~~ | ||||
Show All 29 Lines | |||||
node type restriction string (e.g. "dir,cnt,rev"), and defaults to "*" (all). | node type restriction string (e.g. "dir,cnt,rev"), and defaults to "*" (all). | ||||
For instance, to find the list of origins in which a given directory can be | For instance, to find the list of origins in which a given directory can be | ||||
found: | found: | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | ||||
"src: 'swh:1:dir:0000000000000000000000000000000000000006', return_nodes: {types: 'ori'}, direction: BACKWARD, mask: {paths: ['swhid']}" | "src: 'swh:1:dir:0000000000000000000000000000000000000006', return_nodes: {types: 'ori'}, direction: BACKWARD, mask: {paths: ['swhid']}" | ||||
.. code-block:: python | |||||
response = stub.Traverse(swhgraph.TraversalRequest( | |||||
src=["swh:1:dir:0000000000000000000000000000000000000006"], | |||||
return_nodes=swhgraph.NodeFilter(types="ori"), | |||||
direction=swhgraph.GraphDirection.BACKWARD, | |||||
mask=FieldMask(paths=["swhid"]), | |||||
)) | |||||
for item in response: | |||||
print(f'swhid: "{item.swhid}"') | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054" | swhid: "swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054" | ||||
Traversal from multiple sources | Traversal from multiple sources | ||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
Traversals can have multiple starting nodes, when multiple source nodes are | Traversals can have multiple starting nodes, when multiple source nodes are | ||||
present in the ``src`` field. For instance, this BFS starts from two different | present in the ``src`` field. For instance, this BFS starts from two different | ||||
directories, and explores the graph in parallel from these multiple starting | directories, and explores the graph in parallel from these multiple starting | ||||
points: | points: | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.Traverse \ | ||||
"src: ['swh:1:dir:0000000000000000000000000000000000000006', 'swh:1:dir:0000000000000000000000000000000000000017'], mask: {paths: ['swhid']}" | "src: ['swh:1:dir:0000000000000000000000000000000000000006', 'swh:1:dir:0000000000000000000000000000000000000017'], mask: {paths: ['swhid']}" | ||||
.. code-block:: python | |||||
response = stub.Traverse(swhgraph.TraversalRequest( | |||||
src=[ | |||||
"swh:1:dir:0000000000000000000000000000000000000006", | |||||
"swh:1:dir:0000000000000000000000000000000000000017", | |||||
], | |||||
mask=FieldMask(paths=["swhid"]), | |||||
)) | |||||
for item in response: | |||||
print(f'swhid: "{item.swhid}"') | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:dir:0000000000000000000000000000000000000006" | swhid: "swh:1:dir:0000000000000000000000000000000000000006" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000017" | swhid: "swh:1:dir:0000000000000000000000000000000000000017" | ||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000005" | swhid: "swh:1:cnt:0000000000000000000000000000000000000005" | ||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000004" | swhid: "swh:1:cnt:0000000000000000000000000000000000000004" | ||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000014" | swhid: "swh:1:cnt:0000000000000000000000000000000000000014" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000016" | swhid: "swh:1:dir:0000000000000000000000000000000000000016" | ||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000015" | swhid: "swh:1:cnt:0000000000000000000000000000000000000015" | ||||
Show All 14 Lines | |||||
As an example, a common use-case for content provenance is to find the shortest | As an example, a common use-case for content provenance is to find the shortest | ||||
path of a content to an origin in the transposed graph. This query can be | path of a content to an origin in the transposed graph. This query can be | ||||
run like this: | run like this: | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.FindPathTo \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.FindPathTo \ | ||||
"src: 'swh:1:cnt:0000000000000000000000000000000000000001', target: {types: 'ori'}, direction: BACKWARD, mask: {paths: ['swhid']}" | "src: 'swh:1:cnt:0000000000000000000000000000000000000001', target: {types: 'ori'}, direction: BACKWARD, mask: {paths: ['swhid']}" | ||||
.. code-block:: python | |||||
response = stub.FindPathTo(swhgraph.FindPathToRequest( | |||||
src=["swh:1:cnt:0000000000000000000000000000000000000001"], | |||||
target=swhgraph.NodeFilter(types="ori"), | |||||
direction=swhgraph.GraphDirection.BACKWARD, | |||||
mask=FieldMask(paths=["swhid"]), | |||||
)) | |||||
for item in response.node: | |||||
print(f'swhid: "{item.swhid}"') | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000001" | swhid: "swh:1:cnt:0000000000000000000000000000000000000001" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000008" | swhid: "swh:1:dir:0000000000000000000000000000000000000008" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000009" | swhid: "swh:1:rev:0000000000000000000000000000000000000009" | ||||
swhid: "swh:1:snp:0000000000000000000000000000000000000020" | swhid: "swh:1:snp:0000000000000000000000000000000000000020" | ||||
swhid: "swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054" | swhid: "swh:1:ori:83404f995118bd25774f4ac14422a8f175e7a054" | ||||
As soon as the request finds an origin, it stops and returns the path from the | As soon as the request finds an origin, it stops and returns the path from the | ||||
source set to this origin. | source set to this origin. | ||||
Show All 30 Lines | |||||
restrictions. | restrictions. | ||||
**Example 1**: shortest path from a snapshot to a content (forward graph): | **Example 1**: shortest path from a snapshot to a content (forward graph): | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.FindPathBetween \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.FindPathBetween \ | ||||
"src: 'swh:1:snp:0000000000000000000000000000000000000020', dst: 'swh:1:cnt:0000000000000000000000000000000000000004', mask: {paths: ['swhid']}" | "src: 'swh:1:snp:0000000000000000000000000000000000000020', dst: 'swh:1:cnt:0000000000000000000000000000000000000004', mask: {paths: ['swhid']}" | ||||
.. code-block:: python | |||||
response = stub.FindPathBetween(swhgraph.FindPathBetweenRequest( | |||||
src=["swh:1:snp:0000000000000000000000000000000000000020"], | |||||
dst=["swh:1:cnt:0000000000000000000000000000000000000004"], | |||||
mask=FieldMask(paths=["swhid"]), | |||||
)) | |||||
for item in response.node: | |||||
print(f'swhid: "{item.swhid}"') | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:snp:0000000000000000000000000000000000000020" | swhid: "swh:1:snp:0000000000000000000000000000000000000020" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000009" | swhid: "swh:1:rev:0000000000000000000000000000000000000009" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000008" | swhid: "swh:1:dir:0000000000000000000000000000000000000008" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000006" | swhid: "swh:1:dir:0000000000000000000000000000000000000006" | ||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000004" | swhid: "swh:1:cnt:0000000000000000000000000000000000000004" | ||||
**Example 2**: shortest path from a directory to a snapshot (backward graph): | **Example 2**: shortest path from a directory to a snapshot (backward graph): | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.FindPathBetween \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.FindPathBetween \ | ||||
"src: 'swh:1:dir:0000000000000000000000000000000000000006', dst: 'swh:1:rel:0000000000000000000000000000000000000019', direction: BACKWARD, mask: {paths: ['swhid']}" | "src: 'swh:1:dir:0000000000000000000000000000000000000006', dst: 'swh:1:rel:0000000000000000000000000000000000000019', direction: BACKWARD, mask: {paths: ['swhid']}" | ||||
.. code-block:: python | |||||
response = stub.FindPathBetween(swhgraph.FindPathBetweenRequest( | |||||
src=["swh:1:dir:0000000000000000000000000000000000000006"], | |||||
dst=["swh:1:rel:0000000000000000000000000000000000000019"], | |||||
direction=swhgraph.GraphDirection.BACKWARD, | |||||
mask=FieldMask(paths=["swhid"]), | |||||
)) | |||||
for item in response.node: | |||||
print(f'swhid: "{item.swhid}"') | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:dir:0000000000000000000000000000000000000006" | swhid: "swh:1:dir:0000000000000000000000000000000000000006" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000008" | swhid: "swh:1:dir:0000000000000000000000000000000000000008" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000012" | swhid: "swh:1:dir:0000000000000000000000000000000000000012" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000013" | swhid: "swh:1:rev:0000000000000000000000000000000000000013" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000018" | swhid: "swh:1:rev:0000000000000000000000000000000000000018" | ||||
swhid: "swh:1:rel:0000000000000000000000000000000000000019" | swhid: "swh:1:rel:0000000000000000000000000000000000000019" | ||||
**Example 3**: common ancestor of two contents: | **Example 3**: common ancestor of two contents: | ||||
.. code-block:: console | .. code-block:: console | ||||
$ grpc_cli call localhost:50091 swh.graph.TraversalService.FindPathBetween \ | $ grpc_cli call localhost:50091 swh.graph.TraversalService.FindPathBetween \ | ||||
"src: 'swh:1:cnt:0000000000000000000000000000000000000004', dst: 'swh:1:cnt:0000000000000000000000000000000000000015', direction: BACKWARD, direction_reverse: BACKWARD, mask: {paths: ['swhid']}" | "src: 'swh:1:cnt:0000000000000000000000000000000000000004', dst: 'swh:1:cnt:0000000000000000000000000000000000000015', direction: BACKWARD, direction_reverse: BACKWARD, mask: {paths: ['swhid']}" | ||||
.. code-block:: python | |||||
response = stub.FindPathBetween(swhgraph.FindPathBetweenRequest( | |||||
src=["swh:1:cnt:0000000000000000000000000000000000000004"], | |||||
dst=["swh:1:cnt:0000000000000000000000000000000000000015"], | |||||
direction=swhgraph.GraphDirection.BACKWARD, | |||||
direction_reverse=swhgraph.GraphDirection.BACKWARD, | |||||
mask=FieldMask(paths=["swhid"]), | |||||
)) | |||||
for item in response.node: | |||||
print(f'swhid: "{item.swhid}"') | |||||
.. code-block:: javascript | |||||
swhid: "swh:1:cnt:0000000000000000000000000000000000000004" | swhid: "swh:1:cnt:0000000000000000000000000000000000000004" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000006" | swhid: "swh:1:dir:0000000000000000000000000000000000000006" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000008" | swhid: "swh:1:dir:0000000000000000000000000000000000000008" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000012" | swhid: "swh:1:dir:0000000000000000000000000000000000000012" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000013" | swhid: "swh:1:rev:0000000000000000000000000000000000000013" | ||||
swhid: "swh:1:rev:0000000000000000000000000000000000000018" | swhid: "swh:1:rev:0000000000000000000000000000000000000018" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000017" | swhid: "swh:1:dir:0000000000000000000000000000000000000017" | ||||
swhid: "swh:1:dir:0000000000000000000000000000000000000016" | swhid: "swh:1:dir:0000000000000000000000000000000000000016" | ||||
Show All 21 Lines |
add spaces around * operator