(swh) archit@work-pc:~/swh-environment/swh-lister/swh/lister$ python jjj.py 
DEBUG:swh.lister.core.lister_base:Loading config from lister_gnu
INFO:swh.core.config:Loading config file /home/archit/.config/swh/lister_gnu.yml
DEBUG:swh.lister.core.lister_base:<swh.lister.gnu.lister.GNULister object at 0x7ffacb9f7a20> CONFIG={'content_size_limit': 104857600, 'log_db': 'dbname=softwareheritage-log', 'storage': {'cls': 'remote', 'args': {'url': 'http://localhost:5002/'}}, 'scheduler': {'cls': 'remote', 'args': {'url': 'http://localhost:5008/'}}, 'lister': {'cls': 'local', 'args': {'db': 'postgresql:///lister-gnu'}}, 'credentials': [], 'cache_responses': True, 'cache_dir': '/home/archit/.cache/swh/lister/gnu/'}
DEBUG:urllib3.util.retry:Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/swh/scheduler/__init__.py:69: DeprecationWarning: Call to deprecated class SWHRemoteAPI. (Use the RPCClient instead) -- Deprecated since version 0.0.64.
  return SchedulerBackend(**args)
DEBUG:urllib3.util.retry:Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): ftp.gnu.org:443
DEBUG:urllib3.connectionpool:https://ftp.gnu.org:443 "GET /tree.json.gz HTTP/1.1" 200 571296
DEBUG:root:models: 381
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:5008
DEBUG:urllib3.connectionpool:http://localhost:5008 "POST /create_tasks HTTP/1.1" 400 2203
Traceback (most recent call last):
  File "jjj.py", line 5, in <module>
    gnu_lister()
  File "/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/celery/local.py", line 191, in __call__
    return self._get_current_object()(*a, **kw)
  File "/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/swh/scheduler/task.py", line 45, in __call__
    return super().__call__(*args, **kwargs)
  File "/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/celery/app/task.py", line 394, in __call__
    return self.run(*args, **kwargs)
  File "/home/archit/swh-environment/swh-lister/swh/lister/gnu/tasks.py", line 12, in gnu_lister
    GNULister(**lister_args).run()
  File "/home/archit/swh-environment/swh-lister/swh/lister/core/simple_lister.py", line 72, in run
    response, injected_repos = self.ingest_data(dump_not_used_identifier)
  File "/home/archit/swh-environment/swh-lister/swh/lister/core/simple_lister.py", line 54, in ingest_data
    self.schedule_missing_tasks(models, injected)
  File "/home/archit/swh-environment/swh-lister/swh/lister/core/lister_base.py", line 467, in schedule_missing_tasks
    (task_dicts for (_, _, task_dicts) in tasks.values()))
  File "/home/archit/.virtualenvs/swh/lib/python3.7/site-packages/swh/scheduler/api/client.py", line 31, in create_tasks
    return self.post('create_tasks', {'tasks': tasks})
  File "/home/archit/swh-environment/swh-core/swh/core/api/__init__.py", line 205, in post
    return self._decode_response(response)
  File "/home/archit/swh-environment/swh-core/swh/core/api/__init__.py", line 237, in _decode_response
    raise pickle.loads(decode_response(response))
psycopg2.errors.NotNullViolation: null value in column "retries_left" violates not-null constraint
DETAIL:  Failing row contains (226191, load-gnu, {"args": ["osip", "https://ftp.gnu.org/gnu/osip/"], "kwargs": {"..., 2019-08-18 11:37:59.131191+00, null, next_run_not_scheduled, recurring, null, null).
CONTEXT:  SQL statement "insert into task (type, arguments, next_run, status, current_interval, policy,
                    retries_left, priority)
    select type, arguments, next_run, status, current_interval, policy,
           retries_left, priority
    from tmp_task t
    where not exists(select 1
                     from task
                     where type = t.type and
                           arguments->'args' = t.arguments->'args' and
                           arguments->'kwargs' = t.arguments->'kwargs' and
                           policy = t.policy and
                           priority is not distinct from t.priority and
                           status = t.status)"
PL/pgSQL function swh_scheduler_create_tasks_from_temp() line 12 at SQL statement



















swh-lister logs

swh-lister_1                  | [2019-08-18 11:40:03,807: DEBUG/ForkPoolWorker-1] using channel_id: 1
swh-lister_1                  | [2019-08-18 11:40:03,811: DEBUG/ForkPoolWorker-1] Channel open
swh-lister_1                  | [2019-08-18 11:40:03,847: INFO/ForkPoolWorker-1] Task swh.lister.gnu.tasks.GNUListerTask[3253bc41-78cd-4f54-9a69-c36f0030ccbc] succeeded in 12.527293941999233s: None


swh-scheduler logs

swh-scheduler-api_1           | [2019-08-18 11:38:00 +0000] [39] [DEBUG] Closing connection. 
swh-scheduler-api_1           | [2019-08-18 11:39:47 +0000] [39] [DEBUG] POST /create_tasks
swh-scheduler-api_1           | [2019-08-18 11:39:48 +0000] [39] [DEBUG] Closing connection. 
swh-scheduler-api_1           | [2019-08-18 11:40:03 +0000] [39] [DEBUG] POST /create_tasks
swh-scheduler-api_1           | [2019-08-18 11:40:03 +0000] [39] [DEBUG] Closing connection.