Differential D8910 Diff 32113 swh/graph/tests/dataset/compressed/logs/example-1669888191558-extract_nodes.log
Changeset View
Changeset View
Standalone View
Standalone View
swh/graph/tests/dataset/compressed/logs/example-1669888191558-extract_nodes.log
- This file was added.
Starting compression step EXTRACT_NODES at 2022-12-01 10:49:51.558883 | |||||
Running: java org.softwareheritage.graph.compress.ExtractNodes --format orc --temp-dir swh/graph/tests/dataset/compressed/tmp swh/graph/tests/dataset/orc swh/graph/tests/dataset/compressed/example | |||||
Picked up JAVA_TOOL_OPTIONS: -Xmx30120409497 -XX:PretenureSizeThreshold=512M -XX:MaxNewSize=4G -XX:+UseLargePages -XX:+UseTransparentHugePages -XX:+UseNUMA -XX:+UseTLAB -XX:+ResizeTLAB -Dlogback.configurationFile=swh/graph/tests/dataset/compressed/tmp/logback.xml -Djava.io.tmpdir=swh/graph/tests/dataset/compressed/tmp | |||||
2022-12-01 10:49:51,783 126 INFO [main] o.s.g.c.ExtractNodes - Reading node/edge files and writing sorted batches. | |||||
2022-12-01 10:49:51,911 254 WARN [ForkJoinPool-1-worker-19] o.a.h.u.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | |||||
2022-12-01 10:49:52,056 399 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/release/release-all.orc with {include: [false, true, false, false, true, true, false, false, false, false, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,057 400 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,name:binary,message:binary,target:string,target_type:string,author:binary,date:timestamp,date_offset:smallint,date_raw_offset_bytes:binary,raw_manifest:binary> | |||||
2022-12-01 10:49:52,092 435 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/origin/origin-all.orc with {include: [false, false, true], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,092 435 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,url:string> | |||||
2022-12-01 10:49:52,103 446 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/directory_entry/directory_entry-all.orc with {include: [false, true, true, true, true, true], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,103 446 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<directory_id:string,name:binary,type:string,target:string,perms:int> | |||||
2022-12-01 10:49:52,114 457 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/revision_history/revision_history-all.orc with {include: [false, true, true, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,114 457 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,parent_id:string,parent_rank:int> | |||||
2022-12-01 10:49:52,123 466 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/snapshot_branch/snapshot_branch-all.orc with {include: [false, true, true, true, true], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,123 466 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<snapshot_id:string,name:binary,target:string,target_type:string> | |||||
2022-12-01 10:49:52,132 475 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/origin_visit_status/origin_visit_status-all.orc with {include: [false, true, false, false, false, true, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,132 475 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<origin:string,visit:bigint,date:timestamp,status:string,snapshot:string,type:string> | |||||
2022-12-01 10:49:52,140 483 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/directory/directory-all.orc with {include: [false, true, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,140 483 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,raw_manifest:binary> | |||||
2022-12-01 10:49:52,148 491 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/skipped_content/skipped_content-all.orc with {include: [false, false, true, false, false, false, false, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,148 491 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<sha1:string,sha1_git:string,sha256:string,blake2s256:string,length:bigint,status:string,reason:string> | |||||
2022-12-01 10:49:52,156 499 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/snapshot/snapshot-all.orc with {include: [false, true], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,156 499 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string> | |||||
2022-12-01 10:49:52,164 507 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/content/content-all.orc with {include: [false, false, true, false, false, false, false, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,164 507 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<sha1:string,sha1_git:string,sha256:string,blake2s256:string,length:bigint,status:string,data:binary> | |||||
2022-12-01 10:49:52,172 515 INFO [ForkJoinPool-1-worker-19] o.a.o.i.ReaderImpl - Reading ORC rows from swh/graph/tests/dataset/orc/revision/revision-all.orc with {include: [false, true, false, false, false, false, false, false, false, false, false, true, false, false], offset: 0, length: 9223372036854775807, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false} | |||||
2022-12-01 10:49:52,172 515 INFO [ForkJoinPool-1-worker-19] o.a.o.i.RecordReaderImpl - Reader schema not provided -- using file schema struct<id:string,message:binary,author:binary,date:timestamp,date_offset:smallint,date_raw_offset_bytes:binary,committer:binary,committer_date:timestamp,committer_offset:smallint,committer_date_raw_offset_bytes:binary,directory:string,type:string,raw_manifest:binary> | |||||
2022-12-01 10:49:52,175 518 INFO [main] o.s.g.c.ExtractNodes - Completed. | |||||
2022-12-01 10:49:52,176 519 INFO [main] o.s.g.c.ExtractNodes - Elapsed: 393ms | |||||
2022-12-01 10:49:52,187 530 INFO [main] o.s.g.c.ExtractNodes - Waiting for merge-sort and writing output files... | |||||
Compression step EXTRACT_NODES finished at 2022-12-01 10:49:52.233891 (in 0:00:00.675008) |