Page MenuHomeSoftware Heritage
Differential D8910 Diff 32113 swh/graph/tests/dataset/compressed/logs/example-1669888204581-edge_labels.log

Changeset View

Standalone View

View Options

swh/graph/tests/dataset/compressed/logs/example-1669888204581-edge_labels.log

  • This file was added.
Starting compression step EDGE_LABELS at 2022-12-01 10:50:04.581389
Running: java org.softwareheritage.graph.compress.LabelMapBuilder --temp-dir swh/graph/tests/dataset/compressed/tmp swh/graph/tests/dataset/orc swh/graph/tests/dataset/compressed/example
Picked up JAVA_TOOL_OPTIONS: -Xmx30120409497 -XX:PretenureSizeThreshold=512M -XX:MaxNewSize=4G -XX:+UseLargePages -XX:+UseTransparentHugePages -XX:+UseNUMA -XX:+UseTLAB -XX:+ResizeTLAB -Dlogback.configurationFile=swh/graph/tests/dataset/compressed/tmp/logback.xml -Djava.io.tmpdir=swh/graph/tests/dataset/compressed/tmp
2022-12-01 10:50:04,826 166 INFO [main] o.s.g.c.LabelMapBuilder - Initializing batch arrays.
2022-12-01 10:50:09,762 5102 INFO [main] o.s.g.c.LabelMapBuilder - Reading edges and writing sorted batches.
2022-12-01 10:50:09,912 5252 WARN [ForkJoinPool-1-worker-19] o.a.h.u.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2022-12-01 10:50:10,049 5389 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/release/release-all.orc with {include: [false, true, false, false, true, true, false, false, false, false, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,051 5391 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,name:binary,message:binary,target:string,target_type:string,author:binary,date:timestamp,date_offset:smallint,date_raw_offset_bytes:binary,raw_manifest:binary>
2022-12-01 10:50:10,132 5472 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/origin/origin-all.orc with {include: [false, false, true], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,132 5472 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,url:string>
2022-12-01 10:50:10,149 5489 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/directory_entry/directory_entry-all.orc with {include: [false, true, true, true, true, true], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,150 5490 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<directory_id:string,name:binary,type:string,target:string,perms:int>
2022-12-01 10:50:10,165 5505 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/revision_history/revision_history-all.orc with {include: [false, true, true, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,165 5505 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,parent_id:string,parent_rank:int>
2022-12-01 10:50:10,176 5516 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/snapshot_branch/snapshot_branch-all.orc with {include: [false, true, true, true, true], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,176 5516 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<snapshot_id:string,name:binary,target:string,target_type:string>
2022-12-01 10:50:10,188 5528 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/origin_visit_status/origin_visit_status-all.orc with {include: [false, true, false, false, false, true, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,189 5529 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<origin:string,visit:bigint,date:timestamp,status:string,snapshot:string,type:string>
2022-12-01 10:50:10,199 5539 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/directory/directory-all.orc with {include: [false, true, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,199 5539 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,raw_manifest:binary>
2022-12-01 10:50:10,208 5548 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/skipped_content/skipped_content-all.orc with {include: [false, false, true, false, false, false, false, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,208 5548 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<sha1:string,sha1_git:string,sha256:string,blake2s256:string,length:bigint,status:string,reason:string>
2022-12-01 10:50:10,218 5558 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/snapshot/snapshot-all.orc with {include: [false, true], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,218 5558 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string>
2022-12-01 10:50:10,226 5566 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/content/content-all.orc with {include: [false, false, true, false, false, false, false, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,226 5566 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<sha1:string,sha1_git:string,sha256:string,blake2s256:string,length:bigint,status:string,data:binary>
2022-12-01 10:50:10,237 5577 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/revision/revision-all.orc with {include: [false, true, false, false, false, false, false, false, false, false, false, true, false, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2022-12-01 10:50:10,237 5577 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,message:binary,author:binary,date:timestamp,date_offset:smallint,date_raw_offset_bytes:binary,committer:binary,committer_date:timestamp,committer_offset:smallint,committer_date_raw_offset_bytes:binary,directory:string,type:string,raw_manifest:binary>
2022-12-01 10:50:10,266 5606 INFO [main] o.s.g.c.LabelMapBuilder - Created 1 forward batches and 1 backward batches.
2022-12-01 10:50:10,275 5615 INFO [main] o.s.g.c.LabelMapBuilder - Writing the labels to the label file: swh/graph/tests/dataset/compressed/example-labelled.*
2022-12-01 10:50:10,279 5619 INFO [main] o.s.g.c.LabelMapBuilder - Writing label map to file...
2022-12-01 10:50:10,281 5621 INFO [main] o.s.g.c.LabelMapBuilder - Completed.
2022-12-01 10:50:10,283 5623 INFO [main] o.s.g.c.LabelMapBuilder - Elapsed: 3ms [28 edges, 14,000.00 edges/s, 71.43 µs/edge]
2022-12-01 10:50:10,331 5671 INFO [main] o.s.g.c.LabelMapBuilder - Writing the labels to the label file: swh/graph/tests/dataset/compressed/example-transposed-labelled.*
2022-12-01 10:50:10,331 5671 INFO [main] o.s.g.c.LabelMapBuilder - Writing label map to file...
2022-12-01 10:50:10,331 5671 INFO [main] o.s.g.c.LabelMapBuilder - Completed.
2022-12-01 10:50:10,332 5672 INFO [main] o.s.g.c.LabelMapBuilder - Elapsed: 1ms [28 edges, ∞ edges/s, 0.00 ns/edge]
2022-12-01 10:50:10,332 5672 INFO [main] o.s.g.c.LabelMapBuilder - Done
Compression step EDGE_LABELS finished at 2022-12-01 10:50:10.521093 (in 0:00:05.939704)
About · Developer information · Legal