Page MenuHomeSoftware Heritage
Paste P508

Error while running gnu lister
ActivePublic
Actions

Authored by nahimilega on Aug 18 2019, 2:35 PM.
(swh) archit@work-pc:~/swh-environment/swh-lister/swh/lister$ python jjj.py
DEBUG:swh.lister.core.lister_base:Loading config from lister_gnu
INFO:swh.core.config:Loading config file /home/archit/.config/swh/lister_gnu.yml
DEBUG:swh.lister.core.lister_base:<swh.lister.gnu.lister.GNULister object at 0x7ffacb9f7a20> CONFIG={'content_size_limit': 104857600, 'log_db': 'dbname=softwareheritage-log', 'storage': {'cls': 'remote', 'args': {'url': 'http://localhost:5002/'}}, 'scheduler': {'cls': 'remote', 'args': {'url': 'http://localhost:5008/'}}, 'lister': {'cls': 'local', 'args': {'db': 'postgresql:///lister-gnu'}}, 'credentials': [], 'cache_responses': True, 'cache_dir': '/home/archit/.cache/swh/lister/gnu/'}
DEBUG:urllib3.util.retry:Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/swh/scheduler/__init__.py:69: DeprecationWarning: Call to deprecated class SWHRemoteAPI. (Use the RPCClient instead) -- Deprecated since version 0.0.64.
return SchedulerBackend(**args)
DEBUG:urllib3.util.retry:Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): ftp.gnu.org:443
DEBUG:urllib3.connectionpool:https://ftp.gnu.org:443 "GET /tree.json.gz HTTP/1.1" 200 571296
DEBUG:root:models: 381
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:5008
DEBUG:urllib3.connectionpool:http://localhost:5008 "POST /create_tasks HTTP/1.1" 400 2203
Traceback (most recent call last):
File "jjj.py", line 5, in <module>
gnu_lister()
File "/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/celery/local.py", line 191, in __call__
return self._get_current_object()(*a, **kw)
File "/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/swh/scheduler/task.py", line 45, in __call__
return super().__call__(*args, **kwargs)
File "/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/celery/app/task.py", line 394, in __call__
return self.run(*args, **kwargs)
File "/home/archit/swh-environment/swh-lister/swh/lister/gnu/tasks.py", line 12, in gnu_lister
GNULister(**lister_args).run()
File "/home/archit/swh-environment/swh-lister/swh/lister/core/simple_lister.py", line 72, in run
response, injected_repos = self.ingest_data(dump_not_used_identifier)
File "/home/archit/swh-environment/swh-lister/swh/lister/core/simple_lister.py", line 54, in ingest_data
self.schedule_missing_tasks(models, injected)
File "/home/archit/swh-environment/swh-lister/swh/lister/core/lister_base.py", line 467, in schedule_missing_tasks
(task_dicts for (_, _, task_dicts) in tasks.values()))
File "/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/swh/scheduler/api/client.py", line 31, in create_tasks
return self.post('create_tasks', {'tasks': tasks})
File "/home/archit/swh-environment/swh-core/swh/core/api/__init__.py", line 205, in post
return self._decode_response(response)
File "/home/archit/swh-environment/swh-core/swh/core/api/__init__.py", line 237, in _decode_response
raise pickle.loads(decode_response(response))
psycopg2.errors.NotNullViolation: null value in column "retries_left" violates not-null constraint
DETAIL: Failing row contains (226191, load-gnu, {"args": ["osip", "https://ftp.gnu.org/gnu/osip/"], "kwargs": {"..., 2019-08-18 11:37:59.131191+00, null, next_run_not_scheduled, recurring, null, null).
CONTEXT: SQL statement "insert into task (type, arguments, next_run, status, current_interval, policy,
retries_left, priority)
select type, arguments, next_run, status, current_interval, policy,
retries_left, priority
from tmp_task t
where not exists(select 1
from task
where type = t.type and
arguments->'args' = t.arguments->'args' and
arguments->'kwargs' = t.arguments->'kwargs' and
policy = t.policy and
priority is not distinct from t.priority and
status = t.status)"
PL/pgSQL function swh_scheduler_create_tasks_from_temp() line 12 at SQL statement
swh-lister logs
swh-lister_1 | [2019-08-18 11:40:03,807: DEBUG/ForkPoolWorker-1] using channel_id: 1
swh-lister_1 | [2019-08-18 11:40:03,811: DEBUG/ForkPoolWorker-1] Channel open
swh-lister_1 | [2019-08-18 11:40:03,847: INFO/ForkPoolWorker-1] Task swh.lister.gnu.tasks.GNUListerTask[3253bc41-78cd-4f54-9a69-c36f0030ccbc] succeeded in 12.527293941999233s: None
swh-scheduler logs
swh-scheduler-api_1 | [2019-08-18 11:38:00 +0000] [39] [DEBUG] Closing connection.
swh-scheduler-api_1 | [2019-08-18 11:39:47 +0000] [39] [DEBUG] POST /create_tasks
swh-scheduler-api_1 | [2019-08-18 11:39:48 +0000] [39] [DEBUG] Closing connection.
swh-scheduler-api_1 | [2019-08-18 11:40:03 +0000] [39] [DEBUG] POST /create_tasks
swh-scheduler-api_1 | [2019-08-18 11:40:03 +0000] [39] [DEBUG] Closing connection.

Event Timeline

nahimilega created this paste.Aug 18 2019, 2:35 PM
nahimilega created this object in space S1 Public.
About · Developer information · Legal