Event Timeline
Comment Actions
4 visit status always popping up over and over again for some reason:
18:05:11 swh-web@belvedere:5432=> select * from save_origin_request where status='accepted' and (visit_date is null or visit_status is null or visit_status in ('ongoing', 'created')) and request_date >= now() - interval '30 days';
+-------+-------------------------------+------------+-------------------------------------------------------+----------+-----------------+-------------------------------+---------------------+--------------+----------+
| id | request_date | visit_type | origin_url | status | loading_task_id | visit_date | loading_task_status | visit_status | user_ids |
+-------+-------------------------------+------------+-------------------------------------------------------+----------+-----------------+-------------------------------+---------------------+--------------+----------+
| 86401 | 2021-07-17 22:43:29.870783+00 | git | https://github.com/keybase/client | accepted | 396435047 | 2021-07-17 22:43:35.725966+00 | running | created | (null) |
| 87538 | 2021-07-26 12:59:50.038714+00 | svn | https://svn.r-project.org/R-dev-web/trunk/ | accepted | 396985198 | 2021-07-26 12:59:52.299569+00 | running | created | (null) |
| 87536 | 2021-07-26 12:54:25.784339+00 | svn | svn://svn.code.sf.net/p/codeblocks/code/trunk | accepted | 396984987 | 2021-07-26 12:54:31.857131+00 | running | created | (null) |
| 87544 | 2021-07-26 13:34:16.987444+00 | git | https://android.googlesource.com/platform/system/core | accepted | 396985908 | 2021-07-26 13:34:25.182417+00 | running | created | (null) |
+-------+-------------------------------+------------+-------------------------------------------------------+----------+-----------------+-------------------------------+---------------------+--------------+----------+
(4 rows)
Time: 48.099 msComment Actions
4 visit status always popping up over and over again for some reason:
"A quick look", those are big repositories that may have been oom killed.
(i've rescheduled those)
root@pergamon:~# clush -b -w @swh-workers systemctl status swh-worker@loader_high_priority | grep keybase
Aug 03 16:15:46 worker11 python3[3836]: [2021-08-03 16:15:46,315: INFO/ForkPoolWorker-1] Load origin 'https://github.com/keybase/client' with type 'git'
Aug 03 16:17:12 worker11 python3[3836]: [2021-08-03 16:17:12,887: INFO/ForkPoolWorker-1] Listed 19843 refs for repo https://github.com/keybase/client
root@pergamon:~# clush -b -w @swh-workers systemctl status swh-worker@loader_high_priority | grep android
Aug 03 16:15:46 worker12 python3[3872]: [2021-08-03 16:15:46,341: INFO/ForkPoolWorker-1] Load origin 'https://android.googlesource.com/platform/system/core' with type 'git'
Aug 03 16:17:00 worker12 python3[3872]: [2021-08-03 16:17:00,671: INFO/ForkPoolWorker-1] Listed 48703 refs for repo https://android.googlesource.com/platform/system/core
root@pergamon:~# clush -b -w @swh-workers systemctl status swh-worker@loader_high_priority | grep trunk
└─340690 svnrdump dump svn://svn.code.sf.net/p/codeblocks/code/trunk
Aug 03 16:15:46 worker06 python3[3724]: [2021-08-03 16:15:46,888: INFO/ForkPoolWorker-1] Load origin 'svn://svn.code.sf.net/p/codeblocks/code/trunk' with type 'svn'
├─334757 cat /tmp/tmp5je6yu23/tmpmi2i2ya_/httpssvnrprojectorgRdevwebtrunk.svndump
Aug 03 16:15:46 worker14 python3[3759]: [2021-08-03 16:15:46,955: INFO/ForkPoolWorker-1] Load origin 'https://svn.r-project.org/R-dev-web/trunk/' with type 'svn'Comment Actions
somehow for keybase, that ends up in:
root@pergamon:~# clush -b -w @swh-workers systemctl status swh-worker@loader_high_priority | grep -A3 keybase
Aug 03 16:15:46 worker11 python3[3836]: [2021-08-03 16:15:46,315: INFO/ForkPoolWorker-1] Load origin 'https://github.com/keybase/client' with type 'git'
Aug 03 16:15:52 worker11 python3[3836]: [62B blob data]
Aug 03 16:15:52 worker11 python3[3836]: [3.4K blob data]
Aug 03 16:15:53 worker11 python3[3836]: [3.7K blob data]
--
Aug 03 16:17:12 worker11 python3[3836]: [2021-08-03 16:17:12,887: INFO/ForkPoolWorker-1] Listed 19843 refs for repo https://github.com/keybase/client
Aug 03 16:22:13 worker11 python3[3775]: Process 'ForkPoolWorker-1' pid:3836 exited with 'signal 9 (SIGKILL)'
Aug 03 16:22:14 worker11 python3[3775]: [2021-08-03 16:22:13,883: ERROR/MainProcess] Task handler raised error: WorkerLostError('Worker exited prematurely: signal 9 (SIGKILL).')
Traceback (most recent call last):somehow ^
Comment Actions
android one went well though:
Aug 03 16:15:46 worker12 python3[3872]: [2021-08-03 16:15:46,341: INFO/ForkPoolWorker-1] Load origin 'https://android.googlesource.com/platform/system/core' with type 'git'
Aug 03 16:17:00 worker12 python3[3872]: Sending approximately 188.57 MiB ...
Aug 03 16:17:00 worker12 python3[3872]: [51B blob data]
Aug 03 16:17:00 worker12 python3[3872]: [4.0K blob data]
--
Aug 03 16:17:00 worker12 python3[3872]: [2021-08-03 16:17:00,671: INFO/ForkPoolWorker-1] Listed 48703 refs for repo https://android.googlesource.com/platform/system/core
Aug 03 16:24:09 worker12 python3[3872]: [2021-08-03 16:24:09,256: INFO/ForkPoolWorker-1] Task swh.loader.git.tasks.UpdateGitRepository[3e1cdc0c-8d55-4bed-b455-188811852105] succeeded in 503.007593125978s: {'status': 'eventful'}Comment Actions
on worker17 which is a bit more involved as our current worker, this gets killed with oom:
swhworker@worker17:~$ swh loader -C /etc/softwareheritage/loader_oneshot.yml run git https://github.com/keybase/client
INFO:swh.loader.git.loader.GitLoader:Load origin 'https://github.com/keybase/client' with type 'git'
Enumerating objects: 556997, done.
Counting objects: 100% (2700/2700), done.
Compressing objects: 100% (2219/2219), done.
Total 556997 (delta 589), reused 2436 (delta 457), pack-reused 554297
INFO:swh.loader.git.loader.GitLoader:Listed 19843 refs for repo https://github.com/keybase/client
ERROR:swh.loader.git.loader.GitLoader:Loading failure, updating to `failed` status
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 339, in load
self.store_data()
File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 463, in store_data
for release in self.get_releases():
File "/usr/lib/python3/dist-packages/swh/loader/git/loader.py", line 349, in get_releases
for raw_obj in self.iter_objects(b"tag"):
File "/usr/lib/python3/dist-packages/swh/loader/git/loader.py", line 315, in iter_objects
PackData.from_file(self.pack_buffer, self.pack_size)
File "/usr/lib/python3/dist-packages/dulwich/pack.py", line 1337, in _walk_all_chains
for result in self._follow_chain(offset, type_num, None):
File "/usr/lib/python3/dist-packages/dulwich/pack.py", line 1393, in _follow_chain
unpacked = self._resolve_object(offset, obj_type_num, base_chunks)
File "/usr/lib/python3/dist-packages/dulwich/pack.py", line 1385, in _resolve_object
unpacked.decomp_chunks)
MemoryError
{'status': 'failed'}