Differential D8362
swhgraph: use grpc API Authored by douardda on Aug 31 2022, 3:14 PM.
Details
replace the (deprecated) HTTP RCP API to access the swh-graph service, To be able to test the (now) grpc-based ArchiveGraph, compressed graph Depends on D8361
Diff Detail
Event TimelineComment Actions Build has FAILED Patch application report for D8362 (id=30178)Could not rebase; Attempt merge onto 4a241adbab... Updating 4a241ad..eb38169 Fast-forward swh/provenance/__init__.py | 4 +- swh/provenance/swhgraph/archive.py | 65 +++++-- swh/provenance/tests/conftest.py | 45 ++++- swh/provenance/tests/data/README.md | 24 ++- swh/provenance/tests/data/cmdbts2.msgpack | Bin 14470 -> 16851 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 2 + swh/provenance/tests/data/generate_repo.py | 6 + swh/provenance/tests/data/out-of-order.msgpack | Bin 6325 -> 7508 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 1 + .../cmdbts2/compressed/example-labelled.labelobl | Bin 0 -> 796 bytes .../compressed/example-labelled.labeloffsets | 1 + .../cmdbts2/compressed/example-labelled.labels | Bin 0 -> 101 bytes .../cmdbts2/compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 796 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 2 + .../example-transposed-labelled.properties | 3 + .../cmdbts2/compressed/example-transposed.graph | 1 + .../cmdbts2/compressed/example-transposed.obl | Bin 0 -> 796 bytes .../cmdbts2/compressed/example-transposed.offsets | 6 + .../compressed/example-transposed.properties | 35 ++++ .../cmdbts2/compressed/example.edges.count.txt | 1 + .../cmdbts2/compressed/example.edges.stats.txt | 8 + .../data/swhgraph/cmdbts2/compressed/example.graph | 1 + .../swhgraph/cmdbts2/compressed/example.indegree | 6 + .../cmdbts2/compressed/example.labels.count.txt | 1 + .../cmdbts2/compressed/example.labels.csv.zst | Bin 0 -> 149 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 250 bytes .../cmdbts2/compressed/example.labels.fcl.pointers | Bin 0 -> 72 bytes .../compressed/example.labels.fcl.properties | 2 + .../swhgraph/cmdbts2/compressed/example.labels.mph | Bin 0 -> 1569 bytes .../data/swhgraph/cmdbts2/compressed/example.mph | Bin 0 -> 977 bytes .../cmdbts2/compressed/example.node2swhid.bin | Bin 0 -> 1364 bytes .../cmdbts2/compressed/example.node2type.map | Bin 0 -> 369 bytes .../cmdbts2/compressed/example.nodes.count.txt | 1 + .../cmdbts2/compressed/example.nodes.csv.zst | Bin 0 -> 1420 bytes .../cmdbts2/compressed/example.nodes.stats.txt | 6 + .../data/swhgraph/cmdbts2/compressed/example.obl | Bin 0 -> 796 bytes .../swhgraph/cmdbts2/compressed/example.offsets | 2 + .../data/swhgraph/cmdbts2/compressed/example.order | Bin 0 -> 496 bytes .../swhgraph/cmdbts2/compressed/example.outdegree | 17 ++ .../cmdbts2/compressed/example.persons.count.txt | 1 + .../cmdbts2/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../cmdbts2/compressed/example.persons.mph | Bin 0 -> 961 bytes .../swhgraph/cmdbts2/compressed/example.properties | 35 ++++ .../compressed/example.property.author_id.bin | Bin 0 -> 5456 bytes .../example.property.author_timestamp.bin | Bin 0 -> 10912 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 2728 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 5456 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 10912 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 2728 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 253 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 10912 bytes .../compressed/example.property.message.bin | 17 ++ .../compressed/example.property.message.offset.bin | Bin 0 -> 10912 bytes .../compressed/example.property.tag_name.bin | 2 + .../example.property.tag_name.offset.bin | Bin 0 -> 10912 bytes .../data/swhgraph/cmdbts2/compressed/example.stats | 20 +++ .../logs/example-1661940497526-extract_nodes.log | 29 ++++ .../compressed/logs/example-1661940498184-mph.log | 15 ++ .../compressed/logs/example-1661940498602-bv.log | 33 ++++ .../compressed/logs/example-1661940508827-bfs.log | 7 + .../logs/example-1661940509085-permute_bfs.log | 23 +++ .../logs/example-1661940509454-transpose_bfs.log | 19 ++ .../logs/example-1661940509807-simplify.log | 22 +++ .../compressed/logs/example-1661940510154-llp.log | 191 +++++++++++++++++++++ .../logs/example-1661940510450-permute_llp.log | 23 +++ .../compressed/logs/example-1661940510780-obl.log | 4 + .../logs/example-1661940511036-compose_orders.log | 4 + .../logs/example-1661940511154-stats.log | 7 + .../logs/example-1661940511404-transpose.log | 19 ++ .../logs/example-1661940511747-transpose_obl.log | 4 + .../compressed/logs/example-1661940512005-maps.log | 18 ++ .../logs/example-1661940512337-extract_persons.log | 11 ++ .../logs/example-1661940512985-mph_persons.log | 15 ++ .../logs/example-1661940513432-node_properties.log | 32 ++++ .../logs/example-1661940514220-mph_labels.log | 26 +++ .../logs/example-1661940514677-fcl_labels.log | 6 + .../logs/example-1661940514950-edge_labels.log | 37 ++++ .../logs/example-1661940525586-edge_labels_obl.log | 4 + ...ple-1661940525855-edge_labels_transpose_obl.log | 4 + .../logs/example-1661940526129-clean_tmp.log | 3 + .../cmdbts2/edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../cmdbts2/edges/content/graph-all.nodes.csv.zst | Bin 0 -> 261 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 1329 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 832 bytes .../cmdbts2/edges/origin/graph-all.edges.csv.zst | Bin 0 -> 99 bytes .../cmdbts2/edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../cmdbts2/edges/release/graph-all.edges.csv.zst | Bin 0 -> 144 bytes .../cmdbts2/edges/release/graph-all.nodes.csv.zst | Bin 0 -> 94 bytes .../cmdbts2/edges/revision/graph-all.edges.csv.zst | Bin 0 -> 735 bytes .../cmdbts2/edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 370 bytes .../cmdbts2/edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 576 bytes .../cmdbts2/edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../swhgraph/cmdbts2/orc/content/content-all.orc | Bin 0 -> 2968 bytes .../cmdbts2/orc/directory/directory-all.orc | Bin 0 -> 1461 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 2829 bytes .../swhgraph/cmdbts2/orc/origin/origin-all.orc | Bin 0 -> 755 bytes .../cmdbts2/orc/origin_visit/origin_visit-all.orc | Bin 0 -> 837 bytes .../origin_visit_status-all.orc | Bin 0 -> 1271 bytes .../swhgraph/cmdbts2/orc/release/release-all.orc | Bin 0 -> 1876 bytes .../swhgraph/cmdbts2/orc/revision/revision-all.orc | Bin 0 -> 2805 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1470 bytes .../swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc | Bin 0 -> 578 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1655 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 772 bytes .../compressed/example-labelled.labeloffsets | 1 + .../compressed/example-labelled.labels | 1 + .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 772 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 1 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../out-of-order/compressed/example-transposed.obl | Bin 0 -> 772 bytes .../compressed/example-transposed.offsets | 1 + .../compressed/example-transposed.properties | 35 ++++ .../compressed/example.edges.count.txt | 1 + .../compressed/example.edges.stats.txt | 8 + .../swhgraph/out-of-order/compressed/example.graph | 1 + .../out-of-order/compressed/example.indegree | 4 + .../compressed/example.labels.count.txt | 1 + .../out-of-order/compressed/example.labels.csv.zst | Bin 0 -> 91 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 126 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 32 bytes .../compressed/example.labels.fcl.properties | 2 + .../out-of-order/compressed/example.labels.mph | Bin 0 -> 1537 bytes .../swhgraph/out-of-order/compressed/example.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.node2swhid.bin | Bin 0 -> 550 bytes .../out-of-order/compressed/example.node2type.map | Bin 0 -> 361 bytes .../compressed/example.nodes.count.txt | 1 + .../out-of-order/compressed/example.nodes.csv.zst | Bin 0 -> 644 bytes .../compressed/example.nodes.stats.txt | 6 + .../swhgraph/out-of-order/compressed/example.obl | Bin 0 -> 772 bytes .../out-of-order/compressed/example.offsets | Bin 0 -> 23 bytes .../swhgraph/out-of-order/compressed/example.order | Bin 0 -> 200 bytes .../out-of-order/compressed/example.outdegree | 9 + .../compressed/example.persons.count.txt | 1 + .../compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../out-of-order/compressed/example.persons.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.properties | 35 ++++ .../compressed/example.property.author_id.bin | Bin 0 -> 2200 bytes .../example.property.author_timestamp.bin | Bin 0 -> 4400 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1100 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 2200 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 4400 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1100 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 149 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 4400 bytes .../compressed/example.property.message.bin | 9 + .../compressed/example.property.message.offset.bin | Bin 0 -> 4400 bytes .../compressed/example.property.tag_name.bin | 1 + .../example.property.tag_name.offset.bin | Bin 0 -> 4400 bytes .../swhgraph/out-of-order/compressed/example.stats | 20 +++ .../logs/example-1661940526360-extract_nodes.log | 29 ++++ .../compressed/logs/example-1661940527077-mph.log | 15 ++ .../compressed/logs/example-1661940527519-bv.log | 33 ++++ .../compressed/logs/example-1661940539159-bfs.log | 7 + .../logs/example-1661940539426-permute_bfs.log | 23 +++ .../logs/example-1661940539810-transpose_bfs.log | 19 ++ .../logs/example-1661940540205-simplify.log | 22 +++ .../compressed/logs/example-1661940540563-llp.log | 155 +++++++++++++++++ .../logs/example-1661940540909-permute_llp.log | 23 +++ .../compressed/logs/example-1661940541270-obl.log | 4 + .../logs/example-1661940541554-compose_orders.log | 4 + .../logs/example-1661940541682-stats.log | 7 + .../logs/example-1661940541968-transpose.log | 19 ++ .../logs/example-1661940542335-transpose_obl.log | 4 + .../compressed/logs/example-1661940542614-maps.log | 18 ++ .../logs/example-1661940542947-extract_persons.log | 11 ++ .../logs/example-1661940543608-mph_persons.log | 15 ++ .../logs/example-1661940544116-node_properties.log | 32 ++++ .../logs/example-1661940544966-mph_labels.log | 26 +++ .../logs/example-1661940545479-fcl_labels.log | 6 + .../logs/example-1661940545795-edge_labels.log | 37 ++++ .../logs/example-1661940558242-edge_labels_obl.log | 4 + ...ple-1661940558536-edge_labels_transpose_obl.log | 4 + .../logs/example-1661940558799-clean_tmp.log | 3 + .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 492 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 100 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 96 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 372 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 214 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 359 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../out-of-order/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../out-of-order/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1788 bytes .../out-of-order/orc/origin/origin-all.orc | Bin 0 -> 773 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 856 bytes .../origin_visit_status-all.orc | Bin 0 -> 1296 bytes .../out-of-order/orc/release/release-all.orc | Bin 0 -> 1565 bytes .../out-of-order/orc/revision/revision-all.orc | Bin 0 -> 2335 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1221 bytes .../out-of-order/orc/snapshot/snapshot-all.orc | Bin 0 -> 576 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1441 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 780 bytes .../compressed/example-labelled.labeloffsets | 1 + .../with-merges/compressed/example-labelled.labels | 2 + .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 780 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 2 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../with-merges/compressed/example-transposed.obl | Bin 0 -> 780 bytes .../compressed/example-transposed.offsets | 2 + .../compressed/example-transposed.properties | 35 ++++ .../with-merges/compressed/example.edges.count.txt | 1 + .../with-merges/compressed/example.edges.stats.txt | 8 + .../swhgraph/with-merges/compressed/example.graph | Bin 0 -> 45 bytes .../with-merges/compressed/example.indegree | 5 + .../compressed/example.labels.count.txt | 1 + .../with-merges/compressed/example.labels.csv.zst | Bin 0 -> 105 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 161 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 40 bytes .../compressed/example.labels.fcl.properties | 2 + .../with-merges/compressed/example.labels.mph | Bin 0 -> 1545 bytes .../swhgraph/with-merges/compressed/example.mph | Bin 0 -> 961 bytes .../with-merges/compressed/example.node2swhid.bin | Bin 0 -> 638 bytes .../with-merges/compressed/example.node2type.map | Bin 0 -> 361 bytes .../with-merges/compressed/example.nodes.count.txt | 1 + .../with-merges/compressed/example.nodes.csv.zst | Bin 0 -> 725 bytes .../with-merges/compressed/example.nodes.stats.txt | 6 + .../swhgraph/with-merges/compressed/example.obl | Bin 0 -> 780 bytes .../with-merges/compressed/example.offsets | 1 + .../swhgraph/with-merges/compressed/example.order | Bin 0 -> 232 bytes .../with-merges/compressed/example.outdegree | 13 ++ .../compressed/example.persons.count.txt | 1 + .../with-merges/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../with-merges/compressed/example.persons.mph | Bin 0 -> 961 bytes .../with-merges/compressed/example.properties | 35 ++++ .../compressed/example.property.author_id.bin | Bin 0 -> 2552 bytes .../example.property.author_timestamp.bin | Bin 0 -> 5104 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1276 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 2552 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 5104 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1276 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 157 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 5104 bytes .../compressed/example.property.message.bin | 13 ++ .../compressed/example.property.message.offset.bin | Bin 0 -> 5104 bytes .../compressed/example.property.tag_name.bin | 3 + .../example.property.tag_name.offset.bin | Bin 0 -> 5104 bytes .../swhgraph/with-merges/compressed/example.stats | 20 +++ .../logs/example-1661940558942-extract_nodes.log | 29 ++++ .../compressed/logs/example-1661940559682-mph.log | 15 ++ .../compressed/logs/example-1661940560202-bv.log | 33 ++++ .../compressed/logs/example-1661940572377-bfs.log | 7 + .../logs/example-1661940572667-permute_bfs.log | 23 +++ .../logs/example-1661940573041-transpose_bfs.log | 19 ++ .../logs/example-1661940573410-simplify.log | 22 +++ .../compressed/logs/example-1661940573712-llp.log | 167 ++++++++++++++++++ .../logs/example-1661940574073-permute_llp.log | 23 +++ .../compressed/logs/example-1661940574455-obl.log | 4 + .../logs/example-1661940574735-compose_orders.log | 4 + .../logs/example-1661940574863-stats.log | 7 + .../logs/example-1661940575137-transpose.log | 19 ++ .../logs/example-1661940575528-transpose_obl.log | 4 + .../compressed/logs/example-1661940575814-maps.log | 18 ++ .../logs/example-1661940576150-extract_persons.log | 11 ++ .../logs/example-1661940576871-mph_persons.log | 15 ++ .../logs/example-1661940577366-node_properties.log | 32 ++++ .../logs/example-1661940578201-mph_labels.log | 26 +++ .../logs/example-1661940578731-fcl_labels.log | 6 + .../logs/example-1661940579029-edge_labels.log | 37 ++++ .../logs/example-1661940592752-edge_labels_obl.log | 4 + ...ple-1661940593069-edge_labels_transpose_obl.log | 4 + .../logs/example-1661940593375-clean_tmp.log | 3 + .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 118 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 499 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 100 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 203 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 118 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 445 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 260 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 475 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../with-merges/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../with-merges/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1795 bytes .../swhgraph/with-merges/orc/origin/origin-all.orc | Bin 0 -> 773 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 854 bytes .../origin_visit_status-all.orc | Bin 0 -> 1287 bytes .../with-merges/orc/release/release-all.orc | Bin 0 -> 1923 bytes .../with-merges/orc/revision/revision-all.orc | Bin 0 -> 2420 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1282 bytes .../with-merges/orc/snapshot/snapshot-all.orc | Bin 0 -> 577 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1570 bytes swh/provenance/tests/data/with-merges.msgpack | Bin 10327 -> 9480 bytes swh/provenance/tests/data/with-merges_repo.yaml | 3 + swh/provenance/tests/test_archive_interface.py | 55 ++++-- 303 files changed, 2252 insertions(+), 39 deletions(-) create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940497526-extract_nodes.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940498184-mph.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940498602-bv.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940508827-bfs.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940509085-permute_bfs.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940509454-transpose_bfs.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940509807-simplify.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940510154-llp.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940510450-permute_llp.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940510780-obl.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940511036-compose_orders.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940511154-stats.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940511404-transpose.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940511747-transpose_obl.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940512005-maps.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940512337-extract_persons.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940512985-mph_persons.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940513432-node_properties.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940514220-mph_labels.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940514677-fcl_labels.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940514950-edge_labels.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940525586-edge_labels_obl.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940525855-edge_labels_transpose_obl.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/logs/example-1661940526129-clean_tmp.log create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940526360-extract_nodes.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940527077-mph.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940527519-bv.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940539159-bfs.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940539426-permute_bfs.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940539810-transpose_bfs.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940540205-simplify.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940540563-llp.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940540909-permute_llp.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940541270-obl.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940541554-compose_orders.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940541682-stats.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940541968-transpose.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940542335-transpose_obl.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940542614-maps.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940542947-extract_persons.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940543608-mph_persons.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940544116-node_properties.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940544966-mph_labels.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940545479-fcl_labels.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940545795-edge_labels.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940558242-edge_labels_obl.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940558536-edge_labels_transpose_obl.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/logs/example-1661940558799-clean_tmp.log create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940558942-extract_nodes.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940559682-mph.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940560202-bv.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940572377-bfs.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940572667-permute_bfs.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940573041-transpose_bfs.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940573410-simplify.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940573712-llp.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940574073-permute_llp.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940574455-obl.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940574735-compose_orders.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940574863-stats.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940575137-transpose.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940575528-transpose_obl.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940575814-maps.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940576150-extract_persons.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940576871-mph_persons.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940577366-node_properties.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940578201-mph_labels.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940578731-fcl_labels.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940579029-edge_labels.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940592752-edge_labels_obl.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940593069-edge_labels_transpose_obl.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/logs/example-1661940593375-clean_tmp.log create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot_branch/snapshot_branch-all.orc Changes applied before testcommit eb38169dd3e8a722a8c1694a92b88de8366dadcd
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Aug 30 12:10:16 2022 +0200
swhgraph: use grpc API
replace the (deprecated) HTTP RCP API to access the swh-graph service,
in favor of the grpc server.
To be able to test the (now) grpc-based ArchiveGraph, compressed graph
datasets for all 3 common datasets (cmdbts2, out-of-order and with-merges)
have been generated and included in this revision.
commit 4013246bf3f8d9c532bb411c54fe08764e952727
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Aug 31 14:56:32 2022 +0200
tests: add tags (Release) in the datasets
this will be needed for testing grpc swh-graph archive backend.Link to build: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/653/ Comment Actions Build has FAILED Patch application report for D8362 (id=30179)Could not rebase; Attempt merge onto 4a241adbab... Updating 4a241ad..1f911cc Fast-forward swh/provenance/__init__.py | 4 +- swh/provenance/swhgraph/archive.py | 65 +++++++++++++++------ swh/provenance/tests/conftest.py | 45 +++++++++++++- swh/provenance/tests/data/README.md | 24 +++++++- swh/provenance/tests/data/cmdbts2.msgpack | Bin 14470 -> 16851 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 2 + swh/provenance/tests/data/generate_repo.py | 6 ++ swh/provenance/tests/data/out-of-order.msgpack | Bin 6325 -> 7508 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 1 + .../cmdbts2/compressed/example-labelled.labelobl | Bin 0 -> 796 bytes .../compressed/example-labelled.labeloffsets | 1 + .../cmdbts2/compressed/example-labelled.labels | Bin 0 -> 101 bytes .../cmdbts2/compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 796 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 2 + .../example-transposed-labelled.properties | 3 + .../cmdbts2/compressed/example-transposed.graph | 1 + .../cmdbts2/compressed/example-transposed.obl | Bin 0 -> 796 bytes .../cmdbts2/compressed/example-transposed.offsets | 6 ++ .../compressed/example-transposed.properties | 35 +++++++++++ .../cmdbts2/compressed/example.edges.count.txt | 1 + .../cmdbts2/compressed/example.edges.stats.txt | 8 +++ .../data/swhgraph/cmdbts2/compressed/example.graph | 1 + .../swhgraph/cmdbts2/compressed/example.indegree | 6 ++ .../cmdbts2/compressed/example.labels.count.txt | 1 + .../cmdbts2/compressed/example.labels.csv.zst | Bin 0 -> 149 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 250 bytes .../cmdbts2/compressed/example.labels.fcl.pointers | Bin 0 -> 72 bytes .../compressed/example.labels.fcl.properties | 2 + .../swhgraph/cmdbts2/compressed/example.labels.mph | Bin 0 -> 1569 bytes .../data/swhgraph/cmdbts2/compressed/example.mph | Bin 0 -> 977 bytes .../cmdbts2/compressed/example.node2swhid.bin | Bin 0 -> 1364 bytes .../cmdbts2/compressed/example.node2type.map | Bin 0 -> 369 bytes .../cmdbts2/compressed/example.nodes.count.txt | 1 + .../cmdbts2/compressed/example.nodes.csv.zst | Bin 0 -> 1420 bytes .../cmdbts2/compressed/example.nodes.stats.txt | 6 ++ .../data/swhgraph/cmdbts2/compressed/example.obl | Bin 0 -> 796 bytes .../swhgraph/cmdbts2/compressed/example.offsets | 2 + .../data/swhgraph/cmdbts2/compressed/example.order | Bin 0 -> 496 bytes .../swhgraph/cmdbts2/compressed/example.outdegree | 17 ++++++ .../cmdbts2/compressed/example.persons.count.txt | 1 + .../cmdbts2/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../cmdbts2/compressed/example.persons.mph | Bin 0 -> 961 bytes .../swhgraph/cmdbts2/compressed/example.properties | 35 +++++++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 5456 bytes .../example.property.author_timestamp.bin | Bin 0 -> 10912 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 2728 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 5456 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 10912 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 2728 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 253 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 10912 bytes .../compressed/example.property.message.bin | 17 ++++++ .../compressed/example.property.message.offset.bin | Bin 0 -> 10912 bytes .../compressed/example.property.tag_name.bin | 2 + .../example.property.tag_name.offset.bin | Bin 0 -> 10912 bytes .../data/swhgraph/cmdbts2/compressed/example.stats | 20 +++++++ .../cmdbts2/edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../cmdbts2/edges/content/graph-all.nodes.csv.zst | Bin 0 -> 261 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 1329 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 832 bytes .../cmdbts2/edges/origin/graph-all.edges.csv.zst | Bin 0 -> 99 bytes .../cmdbts2/edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../cmdbts2/edges/release/graph-all.edges.csv.zst | Bin 0 -> 144 bytes .../cmdbts2/edges/release/graph-all.nodes.csv.zst | Bin 0 -> 94 bytes .../cmdbts2/edges/revision/graph-all.edges.csv.zst | Bin 0 -> 735 bytes .../cmdbts2/edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 370 bytes .../cmdbts2/edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 576 bytes .../cmdbts2/edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../swhgraph/cmdbts2/orc/content/content-all.orc | Bin 0 -> 2968 bytes .../cmdbts2/orc/directory/directory-all.orc | Bin 0 -> 1461 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 2829 bytes .../swhgraph/cmdbts2/orc/origin/origin-all.orc | Bin 0 -> 755 bytes .../cmdbts2/orc/origin_visit/origin_visit-all.orc | Bin 0 -> 837 bytes .../origin_visit_status-all.orc | Bin 0 -> 1271 bytes .../swhgraph/cmdbts2/orc/release/release-all.orc | Bin 0 -> 1876 bytes .../swhgraph/cmdbts2/orc/revision/revision-all.orc | Bin 0 -> 2805 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1470 bytes .../swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc | Bin 0 -> 578 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1655 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 772 bytes .../compressed/example-labelled.labeloffsets | 1 + .../compressed/example-labelled.labels | 1 + .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 772 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 1 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../out-of-order/compressed/example-transposed.obl | Bin 0 -> 772 bytes .../compressed/example-transposed.offsets | 1 + .../compressed/example-transposed.properties | 35 +++++++++++ .../compressed/example.edges.count.txt | 1 + .../compressed/example.edges.stats.txt | 8 +++ .../swhgraph/out-of-order/compressed/example.graph | 1 + .../out-of-order/compressed/example.indegree | 4 ++ .../compressed/example.labels.count.txt | 1 + .../out-of-order/compressed/example.labels.csv.zst | Bin 0 -> 91 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 126 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 32 bytes .../compressed/example.labels.fcl.properties | 2 + .../out-of-order/compressed/example.labels.mph | Bin 0 -> 1537 bytes .../swhgraph/out-of-order/compressed/example.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.node2swhid.bin | Bin 0 -> 550 bytes .../out-of-order/compressed/example.node2type.map | Bin 0 -> 361 bytes .../compressed/example.nodes.count.txt | 1 + .../out-of-order/compressed/example.nodes.csv.zst | Bin 0 -> 644 bytes .../compressed/example.nodes.stats.txt | 6 ++ .../swhgraph/out-of-order/compressed/example.obl | Bin 0 -> 772 bytes .../out-of-order/compressed/example.offsets | Bin 0 -> 23 bytes .../swhgraph/out-of-order/compressed/example.order | Bin 0 -> 200 bytes .../out-of-order/compressed/example.outdegree | 9 +++ .../compressed/example.persons.count.txt | 1 + .../compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../out-of-order/compressed/example.persons.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.properties | 35 +++++++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 2200 bytes .../example.property.author_timestamp.bin | Bin 0 -> 4400 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1100 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 2200 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 4400 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1100 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 149 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 4400 bytes .../compressed/example.property.message.bin | 9 +++ .../compressed/example.property.message.offset.bin | Bin 0 -> 4400 bytes .../compressed/example.property.tag_name.bin | 1 + .../example.property.tag_name.offset.bin | Bin 0 -> 4400 bytes .../swhgraph/out-of-order/compressed/example.stats | 20 +++++++ .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 492 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 100 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 96 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 372 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 214 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 359 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../out-of-order/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../out-of-order/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1788 bytes .../out-of-order/orc/origin/origin-all.orc | Bin 0 -> 773 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 856 bytes .../origin_visit_status-all.orc | Bin 0 -> 1296 bytes .../out-of-order/orc/release/release-all.orc | Bin 0 -> 1565 bytes .../out-of-order/orc/revision/revision-all.orc | Bin 0 -> 2335 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1221 bytes .../out-of-order/orc/snapshot/snapshot-all.orc | Bin 0 -> 576 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1441 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 780 bytes .../compressed/example-labelled.labeloffsets | 1 + .../with-merges/compressed/example-labelled.labels | 2 + .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 780 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 2 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../with-merges/compressed/example-transposed.obl | Bin 0 -> 780 bytes .../compressed/example-transposed.offsets | 2 + .../compressed/example-transposed.properties | 35 +++++++++++ .../with-merges/compressed/example.edges.count.txt | 1 + .../with-merges/compressed/example.edges.stats.txt | 8 +++ .../swhgraph/with-merges/compressed/example.graph | Bin 0 -> 45 bytes .../with-merges/compressed/example.indegree | 5 ++ .../compressed/example.labels.count.txt | 1 + .../with-merges/compressed/example.labels.csv.zst | Bin 0 -> 105 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 161 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 40 bytes .../compressed/example.labels.fcl.properties | 2 + .../with-merges/compressed/example.labels.mph | Bin 0 -> 1545 bytes .../swhgraph/with-merges/compressed/example.mph | Bin 0 -> 961 bytes .../with-merges/compressed/example.node2swhid.bin | Bin 0 -> 638 bytes .../with-merges/compressed/example.node2type.map | Bin 0 -> 361 bytes .../with-merges/compressed/example.nodes.count.txt | 1 + .../with-merges/compressed/example.nodes.csv.zst | Bin 0 -> 725 bytes .../with-merges/compressed/example.nodes.stats.txt | 6 ++ .../swhgraph/with-merges/compressed/example.obl | Bin 0 -> 780 bytes .../with-merges/compressed/example.offsets | 1 + .../swhgraph/with-merges/compressed/example.order | Bin 0 -> 232 bytes .../with-merges/compressed/example.outdegree | 13 +++++ .../compressed/example.persons.count.txt | 1 + .../with-merges/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../with-merges/compressed/example.persons.mph | Bin 0 -> 961 bytes .../with-merges/compressed/example.properties | 35 +++++++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 2552 bytes .../example.property.author_timestamp.bin | Bin 0 -> 5104 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1276 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 2552 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 5104 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1276 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 157 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 5104 bytes .../compressed/example.property.message.bin | 13 +++++ .../compressed/example.property.message.offset.bin | Bin 0 -> 5104 bytes .../compressed/example.property.tag_name.bin | 3 + .../example.property.tag_name.offset.bin | Bin 0 -> 5104 bytes .../swhgraph/with-merges/compressed/example.stats | 20 +++++++ .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 118 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 499 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 100 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 203 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 118 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 445 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 260 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 475 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../with-merges/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../with-merges/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1795 bytes .../swhgraph/with-merges/orc/origin/origin-all.orc | Bin 0 -> 773 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 854 bytes .../origin_visit_status-all.orc | Bin 0 -> 1287 bytes .../with-merges/orc/release/release-all.orc | Bin 0 -> 1923 bytes .../with-merges/orc/revision/revision-all.orc | Bin 0 -> 2420 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1282 bytes .../with-merges/orc/snapshot/snapshot-all.orc | Bin 0 -> 577 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1570 bytes swh/provenance/tests/data/with-merges.msgpack | Bin 10327 -> 9480 bytes swh/provenance/tests/data/with-merges_repo.yaml | 3 + swh/provenance/tests/test_archive_interface.py | 55 ++++++++++++----- 231 files changed, 644 insertions(+), 39 deletions(-) create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot_branch/snapshot_branch-all.orc Changes applied before testcommit 1f911ccd4761f13fcf551530f18f6e17c9d24925
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Aug 30 12:10:16 2022 +0200
swhgraph: use grpc API
replace the (deprecated) HTTP RCP API to access the swh-graph service,
in favor of the grpc server.
To be able to test the (now) grpc-based ArchiveGraph, compressed graph
datasets for all 3 common datasets (cmdbts2, out-of-order and with-merges)
have been generated and included in this revision.
commit 4013246bf3f8d9c532bb411c54fe08764e952727
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Aug 31 14:56:32 2022 +0200
tests: add tags (Release) in the datasets
this will be needed for testing grpc swh-graph archive backend.Link to build: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/654/ Comment Actions Build is green Patch application report for D8362 (id=30203)Could not rebase; Attempt merge onto 4a241adbab... Updating 4a241ad..7c6eb04 Fast-forward requirements-test.txt | 3 +- swh/provenance/__init__.py | 4 +- swh/provenance/swhgraph/archive.py | 65 +++++++++++++++------ swh/provenance/tests/conftest.py | 56 +++++++++++++++--- swh/provenance/tests/data/README.md | 24 +++++++- swh/provenance/tests/data/cmdbts2.msgpack | Bin 14470 -> 21456 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 2 + swh/provenance/tests/data/generate_repo.py | 6 ++ .../tests/data/generate_storage_from_git.py | 6 +- swh/provenance/tests/data/origins.csv | 2 +- swh/provenance/tests/data/out-of-order.msgpack | Bin 6325 -> 7508 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 1 + .../cmdbts2/compressed/example-labelled.labelobl | Bin 0 -> 804 bytes .../compressed/example-labelled.labeloffsets | 1 + .../cmdbts2/compressed/example-labelled.labels | 4 ++ .../cmdbts2/compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 804 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | Bin 0 -> 115 bytes .../example-transposed-labelled.properties | 3 + .../cmdbts2/compressed/example-transposed.graph | Bin 0 -> 120 bytes .../cmdbts2/compressed/example-transposed.obl | Bin 0 -> 812 bytes .../cmdbts2/compressed/example-transposed.offsets | 3 + .../compressed/example-transposed.properties | 35 +++++++++++ .../cmdbts2/compressed/example.edges.count.txt | 1 + .../cmdbts2/compressed/example.edges.stats.txt | 8 +++ .../data/swhgraph/cmdbts2/compressed/example.graph | 1 + .../swhgraph/cmdbts2/compressed/example.indegree | 6 ++ .../cmdbts2/compressed/example.labels.count.txt | 1 + .../cmdbts2/compressed/example.labels.csv.zst | Bin 0 -> 149 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 250 bytes .../cmdbts2/compressed/example.labels.fcl.pointers | Bin 0 -> 72 bytes .../compressed/example.labels.fcl.properties | 2 + .../swhgraph/cmdbts2/compressed/example.labels.mph | Bin 0 -> 1569 bytes .../data/swhgraph/cmdbts2/compressed/example.mph | Bin 0 -> 977 bytes .../cmdbts2/compressed/example.node2swhid.bin | Bin 0 -> 1540 bytes .../cmdbts2/compressed/example.node2type.map | Bin 0 -> 377 bytes .../cmdbts2/compressed/example.nodes.count.txt | 1 + .../cmdbts2/compressed/example.nodes.csv.zst | Bin 0 -> 1602 bytes .../cmdbts2/compressed/example.nodes.stats.txt | 6 ++ .../data/swhgraph/cmdbts2/compressed/example.obl | Bin 0 -> 804 bytes .../swhgraph/cmdbts2/compressed/example.offsets | 4 ++ .../data/swhgraph/cmdbts2/compressed/example.order | Bin 0 -> 560 bytes .../swhgraph/cmdbts2/compressed/example.outdegree | 17 ++++++ .../cmdbts2/compressed/example.persons.count.txt | 1 + .../cmdbts2/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../cmdbts2/compressed/example.persons.mph | Bin 0 -> 961 bytes .../swhgraph/cmdbts2/compressed/example.properties | 35 +++++++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 6160 bytes .../example.property.author_timestamp.bin | Bin 0 -> 12320 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 3080 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 6160 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 12320 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 3080 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 277 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 12320 bytes .../compressed/example.property.message.bin | 19 ++++++ .../compressed/example.property.message.offset.bin | Bin 0 -> 12320 bytes .../compressed/example.property.tag_name.bin | 2 + .../example.property.tag_name.offset.bin | Bin 0 -> 12320 bytes .../data/swhgraph/cmdbts2/compressed/example.stats | 20 +++++++ .../cmdbts2/edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../cmdbts2/edges/content/graph-all.nodes.csv.zst | Bin 0 -> 260 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 1326 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 832 bytes .../cmdbts2/edges/origin/graph-all.edges.csv.zst | Bin 0 -> 302 bytes .../cmdbts2/edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../cmdbts2/edges/release/graph-all.edges.csv.zst | Bin 0 -> 144 bytes .../cmdbts2/edges/release/graph-all.nodes.csv.zst | Bin 0 -> 94 bytes .../cmdbts2/edges/revision/graph-all.edges.csv.zst | Bin 0 -> 753 bytes .../cmdbts2/edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 370 bytes .../cmdbts2/edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 764 bytes .../cmdbts2/edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 211 bytes .../swhgraph/cmdbts2/orc/content/content-all.orc | Bin 0 -> 2968 bytes .../cmdbts2/orc/directory/directory-all.orc | Bin 0 -> 1461 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 2817 bytes .../swhgraph/cmdbts2/orc/origin/origin-all.orc | Bin 0 -> 936 bytes .../cmdbts2/orc/origin_visit/origin_visit-all.orc | Bin 0 -> 931 bytes .../origin_visit_status-all.orc | Bin 0 -> 1632 bytes .../swhgraph/cmdbts2/orc/release/release-all.orc | Bin 0 -> 1876 bytes .../swhgraph/cmdbts2/orc/revision/revision-all.orc | Bin 0 -> 2806 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1470 bytes .../swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc | Bin 0 -> 792 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1915 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 772 bytes .../compressed/example-labelled.labeloffsets | 1 + .../compressed/example-labelled.labels | 1 + .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 772 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 1 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../out-of-order/compressed/example-transposed.obl | Bin 0 -> 772 bytes .../compressed/example-transposed.offsets | 1 + .../compressed/example-transposed.properties | 35 +++++++++++ .../compressed/example.edges.count.txt | 1 + .../compressed/example.edges.stats.txt | 8 +++ .../swhgraph/out-of-order/compressed/example.graph | 1 + .../out-of-order/compressed/example.indegree | 4 ++ .../compressed/example.labels.count.txt | 1 + .../out-of-order/compressed/example.labels.csv.zst | Bin 0 -> 91 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 126 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 32 bytes .../compressed/example.labels.fcl.properties | 2 + .../out-of-order/compressed/example.labels.mph | Bin 0 -> 1537 bytes .../swhgraph/out-of-order/compressed/example.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.node2swhid.bin | Bin 0 -> 550 bytes .../out-of-order/compressed/example.node2type.map | Bin 0 -> 361 bytes .../compressed/example.nodes.count.txt | 1 + .../out-of-order/compressed/example.nodes.csv.zst | Bin 0 -> 644 bytes .../compressed/example.nodes.stats.txt | 6 ++ .../swhgraph/out-of-order/compressed/example.obl | Bin 0 -> 772 bytes .../out-of-order/compressed/example.offsets | Bin 0 -> 23 bytes .../swhgraph/out-of-order/compressed/example.order | Bin 0 -> 200 bytes .../out-of-order/compressed/example.outdegree | 9 +++ .../compressed/example.persons.count.txt | 1 + .../compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../out-of-order/compressed/example.persons.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.properties | 35 +++++++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 2200 bytes .../example.property.author_timestamp.bin | Bin 0 -> 4400 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1100 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 2200 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 4400 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1100 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 149 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 4400 bytes .../compressed/example.property.message.bin | 9 +++ .../compressed/example.property.message.offset.bin | Bin 0 -> 4400 bytes .../compressed/example.property.tag_name.bin | 1 + .../example.property.tag_name.offset.bin | Bin 0 -> 4400 bytes .../swhgraph/out-of-order/compressed/example.stats | 20 +++++++ .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 492 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 100 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 96 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 372 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 214 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 359 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../out-of-order/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../out-of-order/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1788 bytes .../out-of-order/orc/origin/origin-all.orc | Bin 0 -> 773 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 856 bytes .../origin_visit_status-all.orc | Bin 0 -> 1296 bytes .../out-of-order/orc/release/release-all.orc | Bin 0 -> 1565 bytes .../out-of-order/orc/revision/revision-all.orc | Bin 0 -> 2335 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1221 bytes .../out-of-order/orc/snapshot/snapshot-all.orc | Bin 0 -> 576 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1441 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 780 bytes .../compressed/example-labelled.labeloffsets | 1 + .../with-merges/compressed/example-labelled.labels | Bin 0 -> 51 bytes .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 780 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 2 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../with-merges/compressed/example-transposed.obl | Bin 0 -> 780 bytes .../compressed/example-transposed.offsets | Bin 0 -> 35 bytes .../compressed/example-transposed.properties | 35 +++++++++++ .../with-merges/compressed/example.edges.count.txt | 1 + .../with-merges/compressed/example.edges.stats.txt | 8 +++ .../swhgraph/with-merges/compressed/example.graph | 1 + .../with-merges/compressed/example.indegree | 6 ++ .../compressed/example.labels.count.txt | 1 + .../with-merges/compressed/example.labels.csv.zst | Bin 0 -> 105 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 161 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 40 bytes .../compressed/example.labels.fcl.properties | 2 + .../with-merges/compressed/example.labels.mph | Bin 0 -> 1545 bytes .../swhgraph/with-merges/compressed/example.mph | Bin 0 -> 969 bytes .../with-merges/compressed/example.node2swhid.bin | Bin 0 -> 792 bytes .../with-merges/compressed/example.node2type.map | Bin 0 -> 361 bytes .../with-merges/compressed/example.nodes.count.txt | 1 + .../with-merges/compressed/example.nodes.csv.zst | Bin 0 -> 875 bytes .../with-merges/compressed/example.nodes.stats.txt | 6 ++ .../swhgraph/with-merges/compressed/example.obl | Bin 0 -> 780 bytes .../with-merges/compressed/example.offsets | 3 + .../swhgraph/with-merges/compressed/example.order | Bin 0 -> 288 bytes .../with-merges/compressed/example.outdegree | 13 +++++ .../compressed/example.persons.count.txt | 1 + .../with-merges/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../with-merges/compressed/example.persons.mph | Bin 0 -> 961 bytes .../with-merges/compressed/example.properties | 35 +++++++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 3168 bytes .../example.property.author_timestamp.bin | Bin 0 -> 6336 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1584 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 3168 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 6336 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1584 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 181 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 6336 bytes .../compressed/example.property.message.bin | 15 +++++ .../compressed/example.property.message.offset.bin | Bin 0 -> 6336 bytes .../compressed/example.property.tag_name.bin | 3 + .../example.property.tag_name.offset.bin | Bin 0 -> 6336 bytes .../swhgraph/with-merges/compressed/example.stats | 20 +++++++ .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 503 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 287 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 204 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 118 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 450 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 259 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 623 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 184 bytes .../with-merges/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../with-merges/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1789 bytes .../swhgraph/with-merges/orc/origin/origin-all.orc | Bin 0 -> 1001 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 986 bytes .../origin_visit_status-all.orc | Bin 0 -> 1671 bytes .../with-merges/orc/release/release-all.orc | Bin 0 -> 1923 bytes .../with-merges/orc/revision/revision-all.orc | Bin 0 -> 2421 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1277 bytes .../with-merges/orc/snapshot/snapshot-all.orc | Bin 0 -> 781 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1791 bytes swh/provenance/tests/data/with-merges.msgpack | Bin 10327 -> 13864 bytes swh/provenance/tests/data/with-merges_repo.yaml | 3 + swh/provenance/tests/test_archive_interface.py | 55 ++++++++++++----- swh/provenance/tests/test_cli.py | 4 +- swh/provenance/tests/test_journal_client.py | 4 +- swh/provenance/tests/test_origin_iterator.py | 8 ++- 237 files changed, 668 insertions(+), 57 deletions(-) create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot_branch/snapshot_branch-all.orc Changes applied before testcommit 7c6eb04a41590d813d58407cb35b9ab2de00699c
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Aug 30 12:10:16 2022 +0200
swhgraph: use grpc API
replace the (deprecated) HTTP RCP API to access the swh-graph service,
in favor of the grpc server.
To be able to test the (now) grpc-based ArchiveGraph, compressed graph
datasets for all 3 common datasets (cmdbts2, out-of-order and with-merges)
have been generated and included in this revision.
commit 3fccbc96b1ab0819f11b6d3e7bda0e71d1026d9a
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Aug 31 14:56:32 2022 +0200
tests: add tags (Release) in the datasets
this will be needed for testing grpc swh-graph archive backend.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/656/ for more details. Comment Actions Build is green Patch application report for D8362 (id=30222)Could not rebase; Attempt merge onto 4a241adbab... Updating 4a241ad..1bbe938 Fast-forward requirements-test.txt | 3 +- swh/provenance/__init__.py | 4 +- swh/provenance/swhgraph/archive.py | 65 +++++++++---- swh/provenance/tests/conftest.py | 56 +++++++++-- swh/provenance/tests/data/README.md | 24 ++++- swh/provenance/tests/data/cmdbts2.msgpack | Bin 14470 -> 21456 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 2 + swh/provenance/tests/data/generate_repo.py | 6 ++ .../tests/data/generate_storage_from_git.py | 6 +- swh/provenance/tests/data/origins.csv | 2 +- swh/provenance/tests/data/out-of-order.msgpack | Bin 6325 -> 7508 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 1 + .../cmdbts2/compressed/example-labelled.labelobl | Bin 0 -> 804 bytes .../compressed/example-labelled.labeloffsets | 1 + .../cmdbts2/compressed/example-labelled.labels | 4 + .../cmdbts2/compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 804 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | Bin 0 -> 115 bytes .../example-transposed-labelled.properties | 3 + .../cmdbts2/compressed/example-transposed.graph | Bin 0 -> 120 bytes .../cmdbts2/compressed/example-transposed.obl | Bin 0 -> 812 bytes .../cmdbts2/compressed/example-transposed.offsets | 3 + .../compressed/example-transposed.properties | 35 +++++++ .../cmdbts2/compressed/example.edges.count.txt | 1 + .../cmdbts2/compressed/example.edges.stats.txt | 8 ++ .../data/swhgraph/cmdbts2/compressed/example.graph | 1 + .../swhgraph/cmdbts2/compressed/example.indegree | 6 ++ .../cmdbts2/compressed/example.labels.count.txt | 1 + .../cmdbts2/compressed/example.labels.csv.zst | Bin 0 -> 149 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 250 bytes .../cmdbts2/compressed/example.labels.fcl.pointers | Bin 0 -> 72 bytes .../compressed/example.labels.fcl.properties | 2 + .../swhgraph/cmdbts2/compressed/example.labels.mph | Bin 0 -> 1569 bytes .../data/swhgraph/cmdbts2/compressed/example.mph | Bin 0 -> 977 bytes .../cmdbts2/compressed/example.node2swhid.bin | Bin 0 -> 1540 bytes .../cmdbts2/compressed/example.node2type.map | Bin 0 -> 377 bytes .../cmdbts2/compressed/example.nodes.count.txt | 1 + .../cmdbts2/compressed/example.nodes.csv.zst | Bin 0 -> 1602 bytes .../cmdbts2/compressed/example.nodes.stats.txt | 6 ++ .../data/swhgraph/cmdbts2/compressed/example.obl | Bin 0 -> 804 bytes .../swhgraph/cmdbts2/compressed/example.offsets | 4 + .../data/swhgraph/cmdbts2/compressed/example.order | Bin 0 -> 560 bytes .../swhgraph/cmdbts2/compressed/example.outdegree | 17 ++++ .../cmdbts2/compressed/example.persons.count.txt | 1 + .../cmdbts2/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../cmdbts2/compressed/example.persons.mph | Bin 0 -> 961 bytes .../swhgraph/cmdbts2/compressed/example.properties | 35 +++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 6160 bytes .../example.property.author_timestamp.bin | Bin 0 -> 12320 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 3080 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 6160 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 12320 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 3080 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 277 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 12320 bytes .../compressed/example.property.message.bin | 19 ++++ .../compressed/example.property.message.offset.bin | Bin 0 -> 12320 bytes .../compressed/example.property.tag_name.bin | 2 + .../example.property.tag_name.offset.bin | Bin 0 -> 12320 bytes .../data/swhgraph/cmdbts2/compressed/example.stats | 20 ++++ .../cmdbts2/edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../cmdbts2/edges/content/graph-all.nodes.csv.zst | Bin 0 -> 260 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 1326 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 832 bytes .../cmdbts2/edges/origin/graph-all.edges.csv.zst | Bin 0 -> 302 bytes .../cmdbts2/edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../cmdbts2/edges/release/graph-all.edges.csv.zst | Bin 0 -> 144 bytes .../cmdbts2/edges/release/graph-all.nodes.csv.zst | Bin 0 -> 94 bytes .../cmdbts2/edges/revision/graph-all.edges.csv.zst | Bin 0 -> 753 bytes .../cmdbts2/edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 370 bytes .../cmdbts2/edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 764 bytes .../cmdbts2/edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 211 bytes .../swhgraph/cmdbts2/orc/content/content-all.orc | Bin 0 -> 2968 bytes .../cmdbts2/orc/directory/directory-all.orc | Bin 0 -> 1461 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 2817 bytes .../swhgraph/cmdbts2/orc/origin/origin-all.orc | Bin 0 -> 936 bytes .../cmdbts2/orc/origin_visit/origin_visit-all.orc | Bin 0 -> 931 bytes .../origin_visit_status-all.orc | Bin 0 -> 1632 bytes .../swhgraph/cmdbts2/orc/release/release-all.orc | Bin 0 -> 1876 bytes .../swhgraph/cmdbts2/orc/revision/revision-all.orc | Bin 0 -> 2806 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1470 bytes .../swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc | Bin 0 -> 792 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1915 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 772 bytes .../compressed/example-labelled.labeloffsets | 1 + .../compressed/example-labelled.labels | 1 + .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 772 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 1 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../out-of-order/compressed/example-transposed.obl | Bin 0 -> 772 bytes .../compressed/example-transposed.offsets | 1 + .../compressed/example-transposed.properties | 35 +++++++ .../compressed/example.edges.count.txt | 1 + .../compressed/example.edges.stats.txt | 8 ++ .../swhgraph/out-of-order/compressed/example.graph | 1 + .../out-of-order/compressed/example.indegree | 4 + .../compressed/example.labels.count.txt | 1 + .../out-of-order/compressed/example.labels.csv.zst | Bin 0 -> 91 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 126 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 32 bytes .../compressed/example.labels.fcl.properties | 2 + .../out-of-order/compressed/example.labels.mph | Bin 0 -> 1537 bytes .../swhgraph/out-of-order/compressed/example.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.node2swhid.bin | Bin 0 -> 550 bytes .../out-of-order/compressed/example.node2type.map | Bin 0 -> 361 bytes .../compressed/example.nodes.count.txt | 1 + .../out-of-order/compressed/example.nodes.csv.zst | Bin 0 -> 644 bytes .../compressed/example.nodes.stats.txt | 6 ++ .../swhgraph/out-of-order/compressed/example.obl | Bin 0 -> 772 bytes .../out-of-order/compressed/example.offsets | Bin 0 -> 23 bytes .../swhgraph/out-of-order/compressed/example.order | Bin 0 -> 200 bytes .../out-of-order/compressed/example.outdegree | 9 ++ .../compressed/example.persons.count.txt | 1 + .../compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../out-of-order/compressed/example.persons.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.properties | 35 +++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 2200 bytes .../example.property.author_timestamp.bin | Bin 0 -> 4400 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1100 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 2200 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 4400 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1100 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 149 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 4400 bytes .../compressed/example.property.message.bin | 9 ++ .../compressed/example.property.message.offset.bin | Bin 0 -> 4400 bytes .../compressed/example.property.tag_name.bin | 1 + .../example.property.tag_name.offset.bin | Bin 0 -> 4400 bytes .../swhgraph/out-of-order/compressed/example.stats | 20 ++++ .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 492 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 100 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 96 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 372 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 214 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 359 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../out-of-order/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../out-of-order/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1788 bytes .../out-of-order/orc/origin/origin-all.orc | Bin 0 -> 773 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 856 bytes .../origin_visit_status-all.orc | Bin 0 -> 1296 bytes .../out-of-order/orc/release/release-all.orc | Bin 0 -> 1565 bytes .../out-of-order/orc/revision/revision-all.orc | Bin 0 -> 2335 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1221 bytes .../out-of-order/orc/snapshot/snapshot-all.orc | Bin 0 -> 576 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1441 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 780 bytes .../compressed/example-labelled.labeloffsets | 1 + .../with-merges/compressed/example-labelled.labels | Bin 0 -> 51 bytes .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 780 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 2 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../with-merges/compressed/example-transposed.obl | Bin 0 -> 780 bytes .../compressed/example-transposed.offsets | Bin 0 -> 35 bytes .../compressed/example-transposed.properties | 35 +++++++ .../with-merges/compressed/example.edges.count.txt | 1 + .../with-merges/compressed/example.edges.stats.txt | 8 ++ .../swhgraph/with-merges/compressed/example.graph | 1 + .../with-merges/compressed/example.indegree | 6 ++ .../compressed/example.labels.count.txt | 1 + .../with-merges/compressed/example.labels.csv.zst | Bin 0 -> 105 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 161 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 40 bytes .../compressed/example.labels.fcl.properties | 2 + .../with-merges/compressed/example.labels.mph | Bin 0 -> 1545 bytes .../swhgraph/with-merges/compressed/example.mph | Bin 0 -> 969 bytes .../with-merges/compressed/example.node2swhid.bin | Bin 0 -> 792 bytes .../with-merges/compressed/example.node2type.map | Bin 0 -> 361 bytes .../with-merges/compressed/example.nodes.count.txt | 1 + .../with-merges/compressed/example.nodes.csv.zst | Bin 0 -> 875 bytes .../with-merges/compressed/example.nodes.stats.txt | 6 ++ .../swhgraph/with-merges/compressed/example.obl | Bin 0 -> 780 bytes .../with-merges/compressed/example.offsets | 3 + .../swhgraph/with-merges/compressed/example.order | Bin 0 -> 288 bytes .../with-merges/compressed/example.outdegree | 13 +++ .../compressed/example.persons.count.txt | 1 + .../with-merges/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../with-merges/compressed/example.persons.mph | Bin 0 -> 961 bytes .../with-merges/compressed/example.properties | 35 +++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 3168 bytes .../example.property.author_timestamp.bin | Bin 0 -> 6336 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1584 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 3168 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 6336 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1584 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 181 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 6336 bytes .../compressed/example.property.message.bin | 15 +++ .../compressed/example.property.message.offset.bin | Bin 0 -> 6336 bytes .../compressed/example.property.tag_name.bin | 3 + .../example.property.tag_name.offset.bin | Bin 0 -> 6336 bytes .../swhgraph/with-merges/compressed/example.stats | 20 ++++ .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 503 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 287 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 204 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 118 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 450 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 259 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 623 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 184 bytes .../with-merges/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../with-merges/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1789 bytes .../swhgraph/with-merges/orc/origin/origin-all.orc | Bin 0 -> 1001 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 986 bytes .../origin_visit_status-all.orc | Bin 0 -> 1671 bytes .../with-merges/orc/release/release-all.orc | Bin 0 -> 1923 bytes .../with-merges/orc/revision/revision-all.orc | Bin 0 -> 2421 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1277 bytes .../with-merges/orc/snapshot/snapshot-all.orc | Bin 0 -> 781 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1791 bytes swh/provenance/tests/data/with-merges.msgpack | Bin 10327 -> 13864 bytes swh/provenance/tests/data/with-merges_repo.yaml | 3 + swh/provenance/tests/test_archive_interface.py | 103 ++++++++++++--------- swh/provenance/tests/test_cli.py | 4 +- swh/provenance/tests/test_journal_client.py | 4 +- swh/provenance/tests/test_origin_iterator.py | 8 +- 237 files changed, 690 insertions(+), 83 deletions(-) create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot_branch/snapshot_branch-all.orc Changes applied before testcommit 1bbe938bf1a9992a0fe81ce65a17a2abb016e796
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Aug 30 12:10:16 2022 +0200
swhgraph: use grpc API
replace the (deprecated) HTTP RCP API to access the swh-graph service,
in favor of the grpc server.
To be able to test the (now) grpc-based ArchiveGraph, compressed graph
datasets for all 3 common datasets (cmdbts2, out-of-order and with-merges)
have been generated and included in this revision.
commit 1d264b10e3aa6ce3a4fa7de24c44776838b3c67d
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Aug 31 14:56:32 2022 +0200
tests: add tags (Release) in the datasets
this will be needed for testing grpc swh-graph archive backend.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/658/ for more details. Comment Actions Build is green Patch application report for D8362 (id=30238)Could not rebase; Attempt merge onto 4a241adbab... Updating 4a241ad..fa15961 Fast-forward requirements-test.txt | 3 +- swh/provenance/__init__.py | 4 +- swh/provenance/swhgraph/archive.py | 65 +++++++++---- swh/provenance/tests/conftest.py | 56 +++++++++-- swh/provenance/tests/data/README.md | 24 ++++- swh/provenance/tests/data/cmdbts2.msgpack | Bin 14470 -> 21456 bytes swh/provenance/tests/data/cmdbts2_repo.yaml | 2 + swh/provenance/tests/data/cmdbts2_visits-01.yaml | 39 ++++++++ .../tests/data/generate_graph_dataset.py | 56 +++++++++++ swh/provenance/tests/data/generate_repo.py | 6 ++ .../tests/data/generate_storage_from_git.py | 6 +- swh/provenance/tests/data/origins.csv | 2 +- swh/provenance/tests/data/out-of-order.msgpack | Bin 6325 -> 7508 bytes swh/provenance/tests/data/out-of-order_repo.yaml | 1 + .../cmdbts2/compressed/example-labelled.labelobl | Bin 0 -> 804 bytes .../compressed/example-labelled.labeloffsets | 1 + .../cmdbts2/compressed/example-labelled.labels | 4 + .../cmdbts2/compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 804 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | Bin 0 -> 115 bytes .../example-transposed-labelled.properties | 3 + .../cmdbts2/compressed/example-transposed.graph | Bin 0 -> 120 bytes .../cmdbts2/compressed/example-transposed.obl | Bin 0 -> 812 bytes .../cmdbts2/compressed/example-transposed.offsets | 3 + .../compressed/example-transposed.properties | 35 +++++++ .../cmdbts2/compressed/example.edges.count.txt | 1 + .../cmdbts2/compressed/example.edges.stats.txt | 8 ++ .../data/swhgraph/cmdbts2/compressed/example.graph | 1 + .../swhgraph/cmdbts2/compressed/example.indegree | 6 ++ .../cmdbts2/compressed/example.labels.count.txt | 1 + .../cmdbts2/compressed/example.labels.csv.zst | Bin 0 -> 149 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 250 bytes .../cmdbts2/compressed/example.labels.fcl.pointers | Bin 0 -> 72 bytes .../compressed/example.labels.fcl.properties | 2 + .../swhgraph/cmdbts2/compressed/example.labels.mph | Bin 0 -> 1569 bytes .../data/swhgraph/cmdbts2/compressed/example.mph | Bin 0 -> 977 bytes .../cmdbts2/compressed/example.node2swhid.bin | Bin 0 -> 1540 bytes .../cmdbts2/compressed/example.node2type.map | Bin 0 -> 377 bytes .../cmdbts2/compressed/example.nodes.count.txt | 1 + .../cmdbts2/compressed/example.nodes.csv.zst | Bin 0 -> 1602 bytes .../cmdbts2/compressed/example.nodes.stats.txt | 6 ++ .../data/swhgraph/cmdbts2/compressed/example.obl | Bin 0 -> 804 bytes .../swhgraph/cmdbts2/compressed/example.offsets | 4 + .../data/swhgraph/cmdbts2/compressed/example.order | Bin 0 -> 560 bytes .../swhgraph/cmdbts2/compressed/example.outdegree | 17 ++++ .../cmdbts2/compressed/example.persons.count.txt | 1 + .../cmdbts2/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../cmdbts2/compressed/example.persons.mph | Bin 0 -> 961 bytes .../swhgraph/cmdbts2/compressed/example.properties | 35 +++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 6160 bytes .../example.property.author_timestamp.bin | Bin 0 -> 12320 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 3080 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 6160 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 12320 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 3080 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 277 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 12320 bytes .../compressed/example.property.message.bin | 19 ++++ .../compressed/example.property.message.offset.bin | Bin 0 -> 12320 bytes .../compressed/example.property.tag_name.bin | 2 + .../example.property.tag_name.offset.bin | Bin 0 -> 12320 bytes .../data/swhgraph/cmdbts2/compressed/example.stats | 20 ++++ .../cmdbts2/edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../cmdbts2/edges/content/graph-all.nodes.csv.zst | Bin 0 -> 260 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 1326 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 832 bytes .../cmdbts2/edges/origin/graph-all.edges.csv.zst | Bin 0 -> 302 bytes .../cmdbts2/edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../cmdbts2/edges/release/graph-all.edges.csv.zst | Bin 0 -> 144 bytes .../cmdbts2/edges/release/graph-all.nodes.csv.zst | Bin 0 -> 94 bytes .../cmdbts2/edges/revision/graph-all.edges.csv.zst | Bin 0 -> 753 bytes .../cmdbts2/edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 370 bytes .../cmdbts2/edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 764 bytes .../cmdbts2/edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 211 bytes .../swhgraph/cmdbts2/orc/content/content-all.orc | Bin 0 -> 2968 bytes .../cmdbts2/orc/directory/directory-all.orc | Bin 0 -> 1461 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 2817 bytes .../swhgraph/cmdbts2/orc/origin/origin-all.orc | Bin 0 -> 936 bytes .../cmdbts2/orc/origin_visit/origin_visit-all.orc | Bin 0 -> 931 bytes .../origin_visit_status-all.orc | Bin 0 -> 1632 bytes .../swhgraph/cmdbts2/orc/release/release-all.orc | Bin 0 -> 1876 bytes .../swhgraph/cmdbts2/orc/revision/revision-all.orc | Bin 0 -> 2806 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1470 bytes .../swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc | Bin 0 -> 792 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1915 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 772 bytes .../compressed/example-labelled.labeloffsets | 1 + .../compressed/example-labelled.labels | 1 + .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 772 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 1 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../out-of-order/compressed/example-transposed.obl | Bin 0 -> 772 bytes .../compressed/example-transposed.offsets | 1 + .../compressed/example-transposed.properties | 35 +++++++ .../compressed/example.edges.count.txt | 1 + .../compressed/example.edges.stats.txt | 8 ++ .../swhgraph/out-of-order/compressed/example.graph | 1 + .../out-of-order/compressed/example.indegree | 4 + .../compressed/example.labels.count.txt | 1 + .../out-of-order/compressed/example.labels.csv.zst | Bin 0 -> 91 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 126 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 32 bytes .../compressed/example.labels.fcl.properties | 2 + .../out-of-order/compressed/example.labels.mph | Bin 0 -> 1537 bytes .../swhgraph/out-of-order/compressed/example.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.node2swhid.bin | Bin 0 -> 550 bytes .../out-of-order/compressed/example.node2type.map | Bin 0 -> 361 bytes .../compressed/example.nodes.count.txt | 1 + .../out-of-order/compressed/example.nodes.csv.zst | Bin 0 -> 644 bytes .../compressed/example.nodes.stats.txt | 6 ++ .../swhgraph/out-of-order/compressed/example.obl | Bin 0 -> 772 bytes .../out-of-order/compressed/example.offsets | Bin 0 -> 23 bytes .../swhgraph/out-of-order/compressed/example.order | Bin 0 -> 200 bytes .../out-of-order/compressed/example.outdegree | 9 ++ .../compressed/example.persons.count.txt | 1 + .../compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../out-of-order/compressed/example.persons.mph | Bin 0 -> 961 bytes .../out-of-order/compressed/example.properties | 35 +++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 2200 bytes .../example.property.author_timestamp.bin | Bin 0 -> 4400 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1100 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 2200 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 4400 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1100 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 149 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 4400 bytes .../compressed/example.property.message.bin | 9 ++ .../compressed/example.property.message.offset.bin | Bin 0 -> 4400 bytes .../compressed/example.property.tag_name.bin | 1 + .../example.property.tag_name.offset.bin | Bin 0 -> 4400 bytes .../swhgraph/out-of-order/compressed/example.stats | 20 ++++ .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 492 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 100 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 96 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 372 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 214 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 359 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 64 bytes .../out-of-order/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../out-of-order/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1788 bytes .../out-of-order/orc/origin/origin-all.orc | Bin 0 -> 773 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 856 bytes .../origin_visit_status-all.orc | Bin 0 -> 1296 bytes .../out-of-order/orc/release/release-all.orc | Bin 0 -> 1565 bytes .../out-of-order/orc/revision/revision-all.orc | Bin 0 -> 2335 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1221 bytes .../out-of-order/orc/snapshot/snapshot-all.orc | Bin 0 -> 576 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1441 bytes .../compressed/example-labelled.labelobl | Bin 0 -> 780 bytes .../compressed/example-labelled.labeloffsets | 1 + .../with-merges/compressed/example-labelled.labels | Bin 0 -> 51 bytes .../compressed/example-labelled.properties | 3 + .../example-transposed-labelled.labelobl | Bin 0 -> 780 bytes .../example-transposed-labelled.labeloffsets | 1 + .../compressed/example-transposed-labelled.labels | 2 + .../example-transposed-labelled.properties | 3 + .../compressed/example-transposed.graph | 1 + .../with-merges/compressed/example-transposed.obl | Bin 0 -> 780 bytes .../compressed/example-transposed.offsets | Bin 0 -> 35 bytes .../compressed/example-transposed.properties | 35 +++++++ .../with-merges/compressed/example.edges.count.txt | 1 + .../with-merges/compressed/example.edges.stats.txt | 8 ++ .../swhgraph/with-merges/compressed/example.graph | 1 + .../with-merges/compressed/example.indegree | 6 ++ .../compressed/example.labels.count.txt | 1 + .../with-merges/compressed/example.labels.csv.zst | Bin 0 -> 105 bytes .../compressed/example.labels.fcl.bytearray | Bin 0 -> 161 bytes .../compressed/example.labels.fcl.pointers | Bin 0 -> 40 bytes .../compressed/example.labels.fcl.properties | 2 + .../with-merges/compressed/example.labels.mph | Bin 0 -> 1545 bytes .../swhgraph/with-merges/compressed/example.mph | Bin 0 -> 969 bytes .../with-merges/compressed/example.node2swhid.bin | Bin 0 -> 792 bytes .../with-merges/compressed/example.node2type.map | Bin 0 -> 361 bytes .../with-merges/compressed/example.nodes.count.txt | 1 + .../with-merges/compressed/example.nodes.csv.zst | Bin 0 -> 875 bytes .../with-merges/compressed/example.nodes.stats.txt | 6 ++ .../swhgraph/with-merges/compressed/example.obl | Bin 0 -> 780 bytes .../with-merges/compressed/example.offsets | 3 + .../swhgraph/with-merges/compressed/example.order | Bin 0 -> 288 bytes .../with-merges/compressed/example.outdegree | 13 +++ .../compressed/example.persons.count.txt | 1 + .../with-merges/compressed/example.persons.csv.zst | Bin 0 -> 58 bytes .../with-merges/compressed/example.persons.mph | Bin 0 -> 961 bytes .../with-merges/compressed/example.properties | 35 +++++++ .../compressed/example.property.author_id.bin | Bin 0 -> 3168 bytes .../example.property.author_timestamp.bin | Bin 0 -> 6336 bytes .../example.property.author_timestamp_offset.bin | Bin 0 -> 1584 bytes .../compressed/example.property.committer_id.bin | Bin 0 -> 3168 bytes .../example.property.committer_timestamp.bin | Bin 0 -> 6336 bytes ...example.property.committer_timestamp_offset.bin | Bin 0 -> 1584 bytes .../example.property.content.is_skipped.bin | Bin 0 -> 181 bytes .../compressed/example.property.content.length.bin | Bin 0 -> 6336 bytes .../compressed/example.property.message.bin | 15 +++ .../compressed/example.property.message.offset.bin | Bin 0 -> 6336 bytes .../compressed/example.property.tag_name.bin | 3 + .../example.property.tag_name.offset.bin | Bin 0 -> 6336 bytes .../swhgraph/with-merges/compressed/example.stats | 20 ++++ .../edges/content/graph-all.edges.csv.zst | Bin 0 -> 13 bytes .../edges/content/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/directory/graph-all.edges.csv.zst | Bin 0 -> 503 bytes .../edges/directory/graph-all.nodes.csv.zst | Bin 0 -> 326 bytes .../edges/origin/graph-all.edges.csv.zst | Bin 0 -> 287 bytes .../edges/origin/graph-all.nodes.csv.zst | Bin 0 -> 117 bytes .../edges/release/graph-all.edges.csv.zst | Bin 0 -> 204 bytes .../edges/release/graph-all.nodes.csv.zst | Bin 0 -> 118 bytes .../edges/revision/graph-all.edges.csv.zst | Bin 0 -> 450 bytes .../edges/revision/graph-all.nodes.csv.zst | Bin 0 -> 259 bytes .../edges/snapshot/graph-all.edges.csv.zst | Bin 0 -> 623 bytes .../edges/snapshot/graph-all.nodes.csv.zst | Bin 0 -> 184 bytes .../with-merges/orc/content/content-all.orc | Bin 0 -> 2297 bytes .../with-merges/orc/directory/directory-all.orc | Bin 0 -> 985 bytes .../orc/directory_entry/directory_entry-all.orc | Bin 0 -> 1789 bytes .../swhgraph/with-merges/orc/origin/origin-all.orc | Bin 0 -> 1001 bytes .../orc/origin_visit/origin_visit-all.orc | Bin 0 -> 986 bytes .../origin_visit_status-all.orc | Bin 0 -> 1671 bytes .../with-merges/orc/release/release-all.orc | Bin 0 -> 1923 bytes .../with-merges/orc/revision/revision-all.orc | Bin 0 -> 2421 bytes .../revision_extra_headers-all.orc | Bin 0 -> 245 bytes .../orc/revision_history/revision_history-all.orc | Bin 0 -> 1277 bytes .../with-merges/orc/snapshot/snapshot-all.orc | Bin 0 -> 781 bytes .../orc/snapshot_branch/snapshot_branch-all.orc | Bin 0 -> 1791 bytes swh/provenance/tests/data/with-merges.msgpack | Bin 10327 -> 13864 bytes swh/provenance/tests/data/with-merges_repo.yaml | 3 + swh/provenance/tests/test_archive_interface.py | 103 ++++++++++++--------- swh/provenance/tests/test_cli.py | 4 +- swh/provenance/tests/test_journal_client.py | 4 +- swh/provenance/tests/test_origin_iterator.py | 8 +- 239 files changed, 785 insertions(+), 83 deletions(-) create mode 100644 swh/provenance/tests/data/cmdbts2_visits-01.yaml create mode 100755 swh/provenance/tests/data/generate_graph_dataset.py create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/cmdbts2/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/out-of-order/orc/snapshot_branch/snapshot_branch-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labelobl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labeloffsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.labels create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed-labelled.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example-transposed.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.edges.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.graph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.indegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.bytearray create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.pointers create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.fcl.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.labels.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2swhid.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.node2type.map create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.nodes.stats.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.obl create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.offsets create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.order create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.outdegree create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.count.txt create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.persons.mph create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.properties create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.author_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_id.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.committer_timestamp_offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.is_skipped.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.content.length.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.message.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.property.tag_name.offset.bin create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/compressed/example.stats create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/content/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/directory/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/origin/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/release/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/revision/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.edges.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/edges/snapshot/graph-all.nodes.csv.zst create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/content/content-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory/directory-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/directory_entry/directory_entry-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin/origin-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit/origin_visit-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/origin_visit_status/origin_visit_status-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/release/release-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision/revision-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_extra_headers/revision_extra_headers-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/revision_history/revision_history-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot/snapshot-all.orc create mode 100644 swh/provenance/tests/data/swhgraph/with-merges/orc/snapshot_branch/snapshot_branch-all.orc Changes applied before testcommit fa15961caf202198c4af5b5bf71cf2a0d651453c
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Aug 30 12:10:16 2022 +0200
swhgraph: use grpc API
replace the (deprecated) HTTP RCP API to access the swh-graph service,
in favor of the grpc server.
To be able to test the (now) grpc-based ArchiveGraph, compressed graph
datasets for all 3 common datasets (cmdbts2, out-of-order and with-merges)
have been generated and included in this revision.
commit 865449ec9656afe0e05ce152698a946f005bef81
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Aug 31 14:56:32 2022 +0200
tests: add tags (Release) in the datasets
this will be needed for testing grpc swh-graph archive backend.See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/660/ for more details. Comment Actions I did the same way as seirl did for swh-graph. The thing is the compress step is quite slow (few 10s of seconds at least), so doing this each time tests are executed would be rather painful. msgpack files (dump of the storage content for a given dataset) could probably be done on the fly, but... |