Page MenuHomeSoftware Heritage
Paste P722

new serialization issue kombu/celery
ActivePublic
Actions

Authored by ardumont on Jul 9 2020, 4:13 PM.
GLOB sdist-make: /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/setup.py
black recreate: /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/black
black installdeps: black
black installed: appdirs==1.4.4,attrs==19.3.0,black==19.10b0,click==7.1.2,pathspec==0.8.0,regex==2020.6.8,toml==0.10.1,typed-ast==1.4.1
black run-test-pre: PYTHONHASHSEED='2865243897'
black run-test: commands[0] | /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/black/bin/python -m black --check swh
All done! ✨ 🍰 ✨
67 files would be left unchanged.
flake8 recreate: /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/flake8
flake8 installdeps: flake8
flake8 installed: flake8==3.8.3,importlib-metadata==1.7.0,mccabe==0.6.1,pycodestyle==2.6.0,pyflakes==2.2.0,zipp==3.1.0
flake8 run-test-pre: PYTHONHASHSEED='2865243897'
flake8 run-test: commands[0] | /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/flake8/bin/python -m flake8
mypy recreate: /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/mypy
mypy installdeps: mypy
mypy inst: /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/.tmp/package/1/swh.loader.core-0.5.3.post2.zip
mypy installed: aiohttp==3.6.2,amqp==2.6.0,arrow==0.15.7,async-timeout==3.0.1,attrs==19.3.0,attrs-strict==0.1.1,billiard==3.6.3.0,cassandra-driver==3.24.0,celery==4.4.6,certifi==2020.6.20,chardet==3.0.4,click==7.1.2,Deprecated==1.2.10,elasticsearch==7.8.0,Flask==1.1.2,future==0.18.2,geomet==0.2.1.post1,hypothesis==5.19.0,idna==2.10,importlib-metadata==1.7.0,iso8601==0.1.12,itsdangerous==1.1.0,Jinja2==2.11.2,kombu==4.6.11,MarkupSafe==1.1.1,more-itertools==8.4.0,multidict==4.7.6,mypy==0.782,mypy-extensions==0.4.3,packaging==20.4,pika==1.1.0,pkginfo==1.5.0.1,pluggy==0.13.1,psutil==5.7.0,psycopg2==2.8.5,py==1.9.0,pyparsing==2.4.7,pytest==5.4.3,pytest-mock==3.1.1,python-dateutil==2.8.1,python-debian==0.1.37,pytz==2020.1,PyYAML==5.3.1,requests==2.24.0,requests-mock==1.8.0,sentry-sdk==0.16.0,six==1.15.0,sortedcontainers==2.2.2,swh.core==0.1.2,swh.loader.core @ file:///home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/.tmp/package/1/swh.loader.core-0.5.3.post2.zip,swh.model==0.5.0,swh.objstorage==0.1.1,swh.scheduler==0.5.1,swh.storage==0.10.1,tenacity==6.2.0,typed-ast==1.4.1,typing-extensions==3.7.4.2,urllib3==1.25.9,vcversioner==2.16.0.0,vine==1.3.0,wcwidth==0.2.5,Werkzeug==1.0.1,wrapt==1.12.1,yarl==1.4.2,zipp==3.1.0
mypy run-test-pre: PYTHONHASHSEED='2865243897'
mypy run-test: commands[0] | mypy swh
Success: no issues found in 67 source files
py3 recreate: /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/py3
py3 installdeps: swh.core[http], swh.storage[testing], pytest-cov
py3 inst: /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/.tmp/package/1/swh.loader.core-0.5.3.post2.zip
py3 installed: aiohttp==3.6.2,aiohttp-utils==3.1.1,amqp==2.6.0,apipkg==1.5,arrow==0.15.7,async-timeout==3.0.1,attrs==19.3.0,attrs-strict==0.1.1,billiard==3.6.3.0,blinker==1.4,cassandra-driver==3.24.0,celery==4.4.6,certifi==2020.6.20,chardet==3.0.4,click==7.1.2,confluent-kafka==1.4.2,coverage==5.2,decorator==4.4.2,Deprecated==1.2.10,elasticsearch==7.8.0,execnet==1.7.1,Flask==1.1.2,future==0.18.2,geomet==0.2.1.post1,gunicorn==20.0.4,hypothesis==5.19.0,idna==2.10,importlib-metadata==1.7.0,iso8601==0.1.12,itsdangerous==1.1.0,Jinja2==2.11.2,kombu==4.6.11,MarkupSafe==1.1.1,mirakuru==2.3.0,more-itertools==8.4.0,msgpack==1.0.0,multidict==4.7.6,mypy==0.782,mypy-extensions==0.4.3,packaging==20.4,pika==1.1.0,pkginfo==1.5.0.1,pluggy==0.13.1,port-for==0.4,psutil==5.7.0,psycopg2==2.8.5,py==1.9.0,pyparsing==2.4.7,pytest==5.4.3,pytest-cov==2.10.0,pytest-forked==1.2.0,pytest-mock==3.1.1,pytest-postgresql==2.4.0,pytest-xdist==1.32.0,python-dateutil==2.8.1,python-debian==0.1.37,python-mimeparse==1.6.0,pytz==2020.1,PyYAML==5.3.1,requests==2.24.0,requests-mock==1.8.0,sentry-sdk==0.16.0,six==1.15.0,sortedcontainers==2.2.2,sqlalchemy-stubs==0.3,swh.core==0.1.2,swh.journal==0.4.0,swh.loader.core @ file:///home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/.tmp/package/1/swh.loader.core-0.5.3.post2.zip,swh.model==0.5.0,swh.objstorage==0.1.1,swh.scheduler==0.5.1,swh.storage==0.10.1,tenacity==6.2.0,typed-ast==1.4.1,typing-extensions==3.7.4.2,urllib3==1.25.9,vcversioner==2.16.0.0,vine==1.3.0,wcwidth==0.2.5,Werkzeug==1.0.1,wrapt==1.12.1,yarl==1.4.2,zipp==3.1.0
py3 run-test-pre: PYTHONHASHSEED='2865243897'
py3 run-test: commands[0] | pytest --cov=/home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/py3/lib/python3.7/site-packages/swh/loader/ --cov-branch /home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/py3/lib/python3.7/site-packages/swh/loader/ -x -s
[2020-07-09 16:11:00,098: ERROR/MainProcess] Received invalid task message: Refusing to deserialize untrusted content of type msgpack (application/x-msgpack)
The message has been ignored and discarded.
Please ensure your message conforms to the task
message protocol as described here:
http://docs.celeryq.org/en/latest/internals/protocol.html
The full contents of the message body was:
b'\x93\x90\x82\xa3url\xa8some-url\xa9artifacts\x90\x84\xa9callbacks\xc0\xa8errbacks\xc0\xa5chain\xc0\xa5chord\xc0' (63b)
Traceback (most recent call last):
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/py3/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 571, in on_task_received
callbacks,
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/py3/lib/python3.7/site-packages/celery/worker/strategy.py", line 137, in task_message_handler
if body is None and 'args' not in message.payload:
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/py3/lib/python3.7/site-packages/kombu/message.py", line 208, in payload
return self._decoded_cache if self._decoded_cache else self.decode()
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/py3/lib/python3.7/site-packages/kombu/message.py", line 193, in decode
self._decoded_cache = self._decode()
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/py3/lib/python3.7/site-packages/kombu/message.py", line 198, in _decode
self.content_encoding, accept=self.accept)
File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/.tox/py3/lib/python3.7/site-packages/kombu/serialization.py", line 253, in loads
raise self._for_untrusted_content(content_type, 'untrusted')
kombu.exceptions.ContentDisallowed: Refusing to deserialize untrusted content of type msgpack (application/x-msgpack)
About · Developer information · Legal