Page MenuHomeSoftware Heritage

Mirror: unsupported Unicode escape sequence
Closed, ResolvedPublic

Description

2021-04-01T23:58:53.195097589Z swh_storage.1.yjtfpr7ambpm@libra    | ERROR:root:unsupported Unicode escape sequence
2021-04-01T23:58:53.195111929Z swh_storage.1.yjtfpr7ambpm@libra    | DETAIL:  \u0000 cannot be converted to text.
2021-04-01T23:58:53.195115416Z swh_storage.1.yjtfpr7ambpm@libra    | CONTEXT:  JSON data, line 1: .../cookbook-ui", "version": "0.0.8", "description":...
2021-04-01T23:58:53.195118594Z swh_storage.1.yjtfpr7ambpm@libra    | COPY tmp_revision, line 135, column metadata: "{"intrinsic": {"tool": "package.json", "raw": {"name": "@caigua/cookbook-ui", "version": "0.0.8", "d..."
2021-04-01T23:58:53.195121602Z swh_storage.1.yjtfpr7ambpm@libra    | Traceback (most recent call last):
2021-04-01T23:58:53.195123371Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/flask/app.py", line 1813, in full_dispatch_request
2021-04-01T23:58:53.195125245Z swh_storage.1.yjtfpr7ambpm@libra    |     rv = self.dispatch_request()
2021-04-01T23:58:53.195126966Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/flask/app.py", line 1799, in dispatch_request
2021-04-01T23:58:53.195129345Z swh_storage.1.yjtfpr7ambpm@libra    |     return self.view_functions[rule.endpoint](**req.view_args)
2021-04-01T23:58:53.195132261Z swh_storage.1.yjtfpr7ambpm@libra    |   File "<decorator-gen-40>", line 2, in revision_add
2021-04-01T23:58:53.195134643Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/swh/core/api/negotiation.py", line 147, in _negotiate
2021-04-01T23:58:53.195137012Z swh_storage.1.yjtfpr7ambpm@libra    |     return f.negotiator(*args, **kwargs)
2021-04-01T23:58:53.195139267Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/swh/core/api/negotiation.py", line 81, in __call__
2021-04-01T23:58:53.195141571Z swh_storage.1.yjtfpr7ambpm@libra    |     result = self.func(*args, **kwargs)
2021-04-01T23:58:53.195143939Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/swh/core/api/__init__.py", line 458, in _f
2021-04-01T23:58:53.195146511Z swh_storage.1.yjtfpr7ambpm@libra    |     return obj_meth(**kw)
2021-04-01T23:58:53.195149288Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/swh/storage/metrics.py", line 24, in d
2021-04-01T23:58:53.195151859Z swh_storage.1.yjtfpr7ambpm@libra    |     return f(*a, **kw)
2021-04-01T23:58:53.195154111Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/swh/storage/metrics.py", line 77, in d
2021-04-01T23:58:53.195156797Z swh_storage.1.yjtfpr7ambpm@libra    |     r = f(*a, **kw)
2021-04-01T23:58:53.195159094Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/swh/core/db/common.py", line 62, in _meth
2021-04-01T23:58:53.195168730Z swh_storage.1.yjtfpr7ambpm@libra    |     return meth(self, *args, db=db, cur=cur, **kwargs)
2021-04-01T23:58:53.195171362Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/swh/storage/postgresql/storage.py", line 581, in revision_add
2021-04-01T23:58:53.195173928Z swh_storage.1.yjtfpr7ambpm@libra    |     lambda rev: parents_filtered.extend(rev["parents"]),
2021-04-01T23:58:53.195176318Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/swh/core/db/__init__.py", line 294, in copy_to
2021-04-01T23:58:53.195178855Z swh_storage.1.yjtfpr7ambpm@libra    |     raise exc_info[1].with_traceback(exc_info[2])
2021-04-01T23:58:53.195180970Z swh_storage.1.yjtfpr7ambpm@libra    |   File "/usr/lib/python3/dist-packages/swh/core/db/__init__.py", line 244, in writer
2021-04-01T23:58:53.195183038Z swh_storage.1.yjtfpr7ambpm@libra    |     "COPY %s (%s) FROM STDIN" % (tblname, ", ".join(columns)), f
2021-04-01T23:58:53.195185541Z swh_storage.1.yjtfpr7ambpm@libra    | psycopg2.errors.UntranslatableCharacter: unsupported Unicode escape sequence
2021-04-01T23:58:53.195188056Z swh_storage.1.yjtfpr7ambpm@libra    | DETAIL:  \u0000 cannot be converted to text.
2021-04-01T23:58:53.195189718Z swh_storage.1.yjtfpr7ambpm@libra    | CONTEXT:  JSON data, line 1: .../cookbook-ui", "version": "0.0.8", "description":...
2021-04-01T23:58:53.195191996Z swh_storage.1.yjtfpr7ambpm@libra    | COPY tmp_revision, line 135, column metadata: "{"intrinsic": {"tool": "package.json", "raw": {"name": "@caigua/cookbook-ui", "version": "0.0.8", "d..."

context;

ii  python3-swh.core        0.12.0-1~swh1~bpo10+1 all          Software Heritage core utilities
ii  python3-swh.journal     0.7.1-1~swh1~bpo10+1  all          Software Heritage Journal utilities
ii  python3-swh.model       2.3.0-1~swh1~bpo10+1  all          Software Heritage data model
ii  python3-swh.objstorage  0.2.2-1~swh1~bpo10+1  all          Software Heritage Object Storage
ii  python3-swh.scheduler   0.10.0-1~swh1~bpo10+1 all          Software Heritage Scheduler
ii  python3-swh.storage     0.27.0-1~swh1~bpo10+1 all          Software Heritage storage utilities

Related Objects

StatusAssignedTask
OpenNone
Resolveddouardda
Openvlorentz
Resolvedvlorentz
Resolveddouardda
Resolvedvlorentz
Resolvedvlorentz
Resolvedvlorentz
Resolvedvlorentz
Resolvedvlorentz
Resolvedvlorentz
Work in Progressvlorentz
Work in Progressolasd
Resolvedvlorentz
Resolvedvlorentz
Resolvedvlorentz
Resolvedvlorentz
Resolvedvlorentz
OpenNone
Resolvedvlorentz
Resolvedvlorentz
Resolvedvlorentz
Resolvedvlorentz
OpenNone

Event Timeline

douardda triaged this task as High priority.Apr 2 2021, 9:54 AM
douardda created this task.

from #swh-devel

09:45 <+vlorentz> douardda: yes and the only way around it (short of dropping data) is T3089
09:46 -swhbot:#swh-devel- T3089 (submitter: vlorentz, owner: vlorentz, status: Open): Remove the 'metadata' column of the 'revision' table <https://forge.softwareheritage.org/T3089>
09:46 <+vlorentz> or switching to cassandra
09:46 <+vlorentz> the good news is, they couldn't be inserted in the storage either, so you can safely drop them for now

easy fix: modify the replayer to ignore this 'metadata' column while inserting revisions