DEBUG:urllib3.connectionpool:http://localhost:5002 "POST /origin/visit/get_latest HTTP/1.1" 400 74
ERROR:swh.loader.pypi.PyPILoader:Loading failure, updating to `partial` status
Traceback (most recent call last):
File "/home/archit/swh-environment/swh-loader-core/swh/loader/core/loader.py", line 868, in load
self.prepare(*args, **kwargs)
File "/home/archit/swh-environment/swh-loader-pypi/swh/loader/pypi/loader.py", line 151, in prepare
self._prepare_state()
File "/home/archit/swh-environment/swh-loader-pypi/swh/loader/pypi/loader.py", line 159, in _prepare_state
last_snapshot = self._last_snapshot()
File "/home/archit/swh-environment/swh-loader-pypi/swh/loader/pypi/loader.py", line 131, in _last_snapshot
self.origin, require_snapshot=True)
File "/home/archit/swh-environment/swh-storage/swh/storage/api/client.py", line 207, in origin_visit_get_latest
'require_snapshot': require_snapshot})
File "/home/archit/swh-environment/swh-core/swh/core/api/__init__.py", line 205, in post
return self._decode_response(response)
File "/home/archit/swh-environment/swh-core/swh/core/api/__init__.py", line 237, in _decode_response
raise pickle.loads(decode_response(response))
psycopg2.ProgrammingError: can't adapt type 'dict'
DEBUG:swh.loader.pypi.PyPILoader:Updating origin_visit for origin https://pypi.org/pypi/numpy/ with status partial
DEBUG:urllib3.connectionpool:http://localhost:5002 "POST /origin/visit/update HTTP/1.1" 200 1
The script I used
import logging
logging.basicConfig(level=logging.DEBUG)
from swh.loader.pypi.loader import PyPILoader
project='numpy'
PyPILoader().load(project, 'https://pypi.org/pypi/%s/' % project, project_metadata_url= 'https://pypi.org/pypi/%s/json' % project)
swh-storage logs
swh-storage_1 | [2019-07-11 13:27:19 +0000] [39] [DEBUG] POST /origin/visit/get_latest
swh-storage_1 | ERROR:root:can't adapt type 'dict'
swh-storage_1 | Traceback (most recent call last):
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
swh-storage_1 | rv = self.dispatch_request()
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
swh-storage_1 | return self.view_functions[rule.endpoint](**req.view_args)
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/api/server.py", line 36, in d
swh-storage_1 | return f(*a, **kw)
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/api/server.py", line 417, in origin_visit_get_latest
swh-storage_1 | get_storage().origin_visit_get_latest(**decode_request(request)))
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/db/common.py", line 49, in _meth
swh-storage_1 | return meth(self, *args, db=db, cur=cur, **kwargs)
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/storage.py", line 1408, in origin_visit_get_latest
swh-storage_1 | origin = self.origin_get({'url': origin}, db=db, cur=cur)
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/db/common.py", line 41, in _meth
swh-storage_1 | ret = meth(self, *args, **kwargs)
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/storage.py", line 1509, in origin_get
swh-storage_1 | for result in results]
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/storage.py", line 1508, in <listcomp>
swh-storage_1 | results = [dict(zip(self.origin_keys, result))
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/db.py", line 675, in origin_get_with
swh-storage_1 | cur, query, origins)
swh-storage_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/db/db_utils.py", line 145, in execute_values_generator
swh-storage_1 | parts.append(cur.mogrify(template, args))
swh-storage_1 | psycopg2.ProgrammingError: can't adapt type 'dict'
swh-storage_1 | [2019-07-11 13:27:19 +0000] [39] [DEBUG] POST /origin/visit/update