Page MenuHomeSoftware Heritage
Paste P1406

large repository ingestion failure
ActivePublic

Authored by ardumont on Jul 13 2022, 2:05 PM.
swhworker@worker17:~$ url=https://github.com/liferay/liferay-portal; /usr/bin/time -v swh loader run git $url lister_name=github lister_instance_name=github pack_size_bytes=34359738368 | tee liferay-portal-pack-size-limit-32g.txt
INFO:swh.loader.git.loader.GitLoader:Load origin 'https://github.com/liferay/liferay-portal' with type 'git'
Enumerating objects: 14950278, done.
Counting objects: 100% (647/647), done.
Compressing objects: 100% (468/468), done.
Total 14950278 (delta 280), reused 202 (delta 159), pack-reused 14949631
INFO:swh.loader.git.loader:Listed 5023 refs for repo https://github.com/liferay/liferay-portal
Command terminated by signal 9
Command being timed: "swh loader run git https://github.com/liferay/liferay-portal lister_name=github lister_instance_name=github pack_size_bytes=34359738368"
User time (seconds): 6876.97
System time (seconds): 884.79
Percent of CPU this job got: 4%
Elapsed (wall clock) time (h:mm:ss or m:ss): 45:26:16
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 1818876
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 178
Minor (reclaiming a frame) page faults: 7752880
Voluntary context switches: 47687
Involuntary context switches: 82009
Swaps: 0
File system inputs: 30952
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0

Event Timeline

swhworker@worker18:~$ url=https://github.com/chromium/chromium; /usr/bin/time -v swh loader run git $url lister_name=gi
thub lister_instance_name=github pack_size_bytes=34359738368 | tee chromium-pack-size-limit-32g.txt
INFO:swh.loader.git.loader.GitLoader:Load origin 'https://github.com/chromium/chromium' with type 'git'
Enumerating objects: 17330770, done.
Counting objects: 100% (4699/4699), done.
Compressing objects: 100% (2828/2828), done.
Total 17330770 (delta 3123), reused 2761 (delta 1827), pack-reused 17326071
INFO:swh.loader.git.loader:Listed 29322 refs for repo https://github.com/chromium/chromium
Command terminated by signal 9
        Command being timed: "swh loader run git https://github.com/chromium/chromium lister_name=github lister_instanc
e_name=github pack_size_bytes=34359738368"
        User time (seconds): 6315.51
        System time (seconds): 828.86
        Percent of CPU this job got: 54%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 3:38:12
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 2301852
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 138
        Minor (reclaiming a frame) page faults: 1463220
        Voluntary context switches: 19512
        Involuntary context switches: 140759
        Swaps: 0
        File system inputs: 18656
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
ardumont changed the title of this paste from untitled to large repository ingestion failure.Jul 13 2022, 2:06 PM