Page MenuHomeSoftware Heritage

cassandra.converters: Fix revision type
AbandonedPublic

Authored by ardumont on Jun 3 2020, 12:29 PM.

Details

Reviewers
douardda
Group Reviewers
Reviewers
Summary

Related to latest swh.model change (from List to Tuple).

This should fix the latest ci build failure [1]

[1] https://jenkins.softwareheritage.org/job/DSTO/job/tests/1217/console

Related to D3177

Test Plan

tox -e mypy is happier
unfortunately tox is not

Diff Detail

Repository
rDSTO Storage manager
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 12595
Build 19134: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 19133: arc lint + arc unit

Unit TestsFailed

TimeTest
8,687 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.algos.test_snapshot::test_snapshot_large
self = <hypothesis.core.StateForActualGivenExecution object at 0x7f6aa40a4518> data = ConjectureData(INTERESTING, 24 bytes, frozen)
569 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_cassandra.TestCassandraStorage::test_revision_get
self = <swh.storage.tests.test_cassandra.TestCassandraStorage object at 0x7f6b1c6f57f0> swh_storage = <swh.storage.validate.ValidatingProxyStorage object at 0x7f6b3c0d7a58>
573 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_cassandra.TestCassandraStorage::test_revision_get_no_parents
self = <swh.storage.tests.test_cassandra.TestCassandraStorage object at 0x7f6b3c0e0a20> swh_storage = <swh.storage.validate.ValidatingProxyStorage object at 0x7f6b3c0e0a58>
677 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_cassandra.TestCassandraStorage::test_revision_get_order
self = <swh.storage.tests.test_cassandra.TestCassandraStorage object at 0x7f6b7c6a1358> swh_storage = <swh.storage.validate.ValidatingProxyStorage object at 0x7f6b3c57c668>
808 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_cassandra.TestCassandraStorage::test_revision_log
self = <swh.storage.tests.test_cassandra.TestCassandraStorage object at 0x7f6b5c060fd0> swh_storage = <swh.storage.validate.ValidatingProxyStorage object at 0x7f6b5c0b5f28>
View Full Test Results (15 Failed · 875 Passed · 18 Skipped)

Event Timeline

This revision is now accepted and ready to land.Jun 3 2020, 12:31 PM

Build has FAILED

Patch application report for D3211 (id=11394)

Rebasing onto eef4900db5...

Current branch diff-target is up to date.
Changes applied before test
commit 076ac326789f15979c2a80260d323e360ac756dc
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 3 12:29:04 2020 +0200

    cassandra.converters: Fix revision type
    
    Related to latest swh.model change (from List to Tuple).
    
    Fixes the latest ci build failure [1]
    
    [1] https://jenkins.softwareheritage.org/job/DSTO/job/tests/1217/console

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/208/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/208/console

ardumont edited the test plan for this revision. (Show Details)EditedJun 3 2020, 12:45 PM

Hum, yes, i only triggered tox -e mypy... so there you go ¯\_(ツ)_/¯

build has FAILED

yes, trying to fix that.

There is something somewhere which converts back tuple into a list... [1]

[1]

.tox/py3/lib/python3.7/site-packages/swh/storage/tests/test_api_client.py::TestStorage::test_revision_get_order <- .tox/py3/lib/python3.7/site-packages/swh/storage/tests/test_storage.py
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/tony/work/inria/repo/swh/swh-environment/swh-storage/.tox/py3/lib/python3.7/site-packages/swh/storage/in_memory.py(482)revision_get()
-> yield rev
(Pdb) rev
{'message': b'hello', 'author': {'fullname': b'Nicolas Dandrimont <nicolas@example.com> ', 'name': b'Nicolas Dandrimont', 'email': b'nicolas@example.com'}, 'committer': {'fullname': b'St\xc3fano Zacchiroli <stefano@example.com>', 'name': b'St\xc3fano Zacchiroli', 'email': b'stefano@example.com'}, 'date': {'timestamp': {'seconds': 1234567890, 'microseconds': 0}, 'offset': 120, 'negative_utc': False}, 'committer_date': {'timestamp': {'seconds': 1123456789, 'microseconds': 0}, 'offset': 0, 'negative_utc': True}, 'type': 'git', 'directory': b'4\x013B2S1\x000\xf51\xe62\xa73\xff7\xc3\xa90', 'synthetic': True, 'metadata': {'checksums': {'sha1': 'tarball-sha1', 'sha256': 'tarball-sha256'}, 'signed-off-by': 'some-dude', 'extra_headers': [['gpgsig', b'test123'], ['mergetags', [b'foo\\bar', b'"\xaf\x89\x80\x01\x00']]]}, 'parents': (b'01234567890123456789', b'23434512345123456789'), 'id': b'56789012345678901234'}
(Pdb) rev['parents']
(b'01234567890123456789', b'23434512345123456789')
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/tony/work/inria/repo/swh/swh-environment/swh-storage/.tox/py3/lib/python3.7/site-packages/swh/storage/in_memory.py(482)revision_get()
-> yield rev
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/tony/work/inria/repo/swh/swh-environment/swh-storage/.tox/py3/lib/python3.7/site-packages/swh/storage/tests/test_storage.py(1020)test_revision_get_order()
-> assert res1 == expected_revs
(Pdb) [r['parents'] for r in res1]
[[b'01234567890123456789', b'23434512345123456789'], [b'01234567890123456789']]

My feeling for now is it's serialization related... (we are using msgpack)

In [1]:  import msgpack

In [2]: msgpack.packb(tuple([1, 2, 3]), use_bin_type=True)
Out[2]: b'\x93\x01\x02\x03'

In [3]: msgpack.unpackb(_, raw=False)
Out[3]: [1, 2, 3]In [1]:  import msgpack

In [2]: msgpack.packb(tuple([1, 2, 3]), use_bin_type=True)
Out[2]: b'\x93\x01\x02\x03'

In [3]: msgpack.unpackb(_, raw=False)
Out[3]: [1, 2, 3]

In [2]: msgpack.packb({'a': (1, 2, 3)}, use_bin_type=True)
Out[2]: b'\x81\xa1a\x93\x01\x02\x03'

In [3]: msgpack.unpackb(_, raw=False)
Out[3]: {'a': [1, 2, 3]}
This revision now requires review to proceed.Jun 3 2020, 2:46 PM
ardumont edited the test plan for this revision. (Show Details)
ardumont added a reviewer: Reviewers.