Page MenuHomeSoftware Heritage

Fix type of snapshot_count_branches.
ClosedPublic

Authored by vlorentz on Aug 10 2020, 9:37 PM.

Details

Summary

Depends on D3756.

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Aug 10 2020, 9:37 PM

Build is green

Patch application report for D3757 (id=13221)

Could not rebase; Attempt merge onto 7d332f5967...

Updating 7d332f59..564a4d25
Fast-forward
 swh/storage/cassandra/common.py     |   8 +-
 swh/storage/cassandra/converters.py |  37 ++-
 swh/storage/cassandra/cql.py        | 541 +++++++++++++++++-------------------
 swh/storage/cassandra/model.py      | 196 +++++++++++++
 swh/storage/cassandra/schema.py     |   1 -
 swh/storage/cassandra/storage.py    | 174 +++++++-----
 swh/storage/in_memory.py            |   4 +-
 swh/storage/interface.py            |   4 +-
 swh/storage/storage.py              |   2 +-
 swh/storage/tests/test_cassandra.py |  30 +-
 10 files changed, 602 insertions(+), 395 deletions(-)
 create mode 100644 swh/storage/cassandra/model.py
Changes applied before test
commit 564a4d2590ae7f468f2b0b7377d0b2d91eeaac56
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Aug 10 21:36:41 2020 +0200

    Fix type of snapshot_count_branches.

commit 3d3ac1605129097740c24ccc48a8519a1b9b78a3
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Aug 10 21:33:00 2020 +0200

    cassandra.cql: Use static dataclasses instead of generating namedtuples on the fly.
    
    Before this commit, python-cassandra used the default row factory,
    which creates anonymous named tuple on each query, which makes it
    impossible to type CqlRunner properly.
    
    This commit replaces the row factory with dict_factory, which creates
    only dicts, and converts them to well-defined dataclasses.
    Additionally, this stop leaking python-cassandra internals to
    cassandra.storage.
    
    This also has some great side-effects:
    
    * methods of CqlRunner are now consistent with each other (eg. _add_one
      methods used to be a mix of objects, dictionaries, and taking each value
      as argument)
    * it will allow me to deduplicate more codes in further commits (I
      already deduplicated insertions methods to use self._add_one, as
      it was meant on the initial write of this class)
    * CqlRunner no longer needs to define lists with column names, they are
      automatically detected from the dataclasses

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/721/ for more details.

Build has FAILED

Patch application report for D3757 (id=13225)

Rebasing onto 7d332f5967...

Current branch diff-target is up to date.
Changes applied before test
commit 3d0cc143da58d6facb106060a544876e2fe04a59
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Aug 10 21:36:41 2020 +0200

    Fix type of snapshot_count_branches.

commit 319de05d5fbebbebb47532209490a2f8380f5343
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Aug 10 21:33:00 2020 +0200

    cassandra.cql: Use static dataclasses instead of generating namedtuples on the fly.
    
    Before this commit, python-cassandra used the default row factory,
    which creates anonymous named tuple on each query, which makes it
    impossible to type CqlRunner properly.
    
    This commit replaces the row factory with dict_factory, which creates
    only dicts, and converts them to well-defined dataclasses.
    Additionally, this stop leaking python-cassandra internals to
    cassandra.storage.
    
    This also has some great side-effects:
    
    * methods of CqlRunner are now consistent with each other (eg. _add_one
      methods used to be a mix of objects, dictionaries, and taking each value
      as argument)
    * it will allow me to deduplicate more codes in further commits (I
      already deduplicated insertions methods to use self._add_one, as
      it was meant on the initial write of this class)
    * CqlRunner no longer needs to define lists with column names, they are
      automatically detected from the dataclasses

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/725/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/725/console

anlambert added a subscriber: anlambert.EditedAug 11 2020, 10:41 AM

Looks like you did not rebase the diff to the right branch as it also includes the changes from D3756.

Hopefully, I can get the diff through git after applying arc patch locally.

vlorentz updated this revision to Diff 13226.Aug 11 2020, 11:08 AM

change base

Build has FAILED

Patch application report for D3757 (id=13226)

Could not rebase; Attempt merge onto 7d332f5967...

Updating 7d332f59..3d0cc143
Fast-forward
 swh/storage/cassandra/common.py     |   8 +-
 swh/storage/cassandra/converters.py |  37 ++-
 swh/storage/cassandra/cql.py        | 541 +++++++++++++++++-------------------
 swh/storage/cassandra/model.py      | 196 +++++++++++++
 swh/storage/cassandra/schema.py     |   1 -
 swh/storage/cassandra/storage.py    | 170 ++++++-----
 swh/storage/in_memory.py            |   4 +-
 swh/storage/interface.py            |   4 +-
 swh/storage/storage.py              |   2 +-
 swh/storage/tests/test_cassandra.py |  30 +-
 10 files changed, 599 insertions(+), 394 deletions(-)
 create mode 100644 swh/storage/cassandra/model.py
Changes applied before test
commit 3d0cc143da58d6facb106060a544876e2fe04a59
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Aug 10 21:36:41 2020 +0200

    Fix type of snapshot_count_branches.

commit 319de05d5fbebbebb47532209490a2f8380f5343
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Aug 10 21:33:00 2020 +0200

    cassandra.cql: Use static dataclasses instead of generating namedtuples on the fly.
    
    Before this commit, python-cassandra used the default row factory,
    which creates anonymous named tuple on each query, which makes it
    impossible to type CqlRunner properly.
    
    This commit replaces the row factory with dict_factory, which creates
    only dicts, and converts them to well-defined dataclasses.
    Additionally, this stop leaking python-cassandra internals to
    cassandra.storage.
    
    This also has some great side-effects:
    
    * methods of CqlRunner are now consistent with each other (eg. _add_one
      methods used to be a mix of objects, dictionaries, and taking each value
      as argument)
    * it will allow me to deduplicate more codes in further commits (I
      already deduplicated insertions methods to use self._add_one, as
      it was meant on the initial write of this class)
    * CqlRunner no longer needs to define lists with column names, they are
      automatically detected from the dataclasses

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/726/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/726/console

vlorentz updated this revision to Diff 13228.Aug 11 2020, 11:13 AM

fix commit mess

Build is green

Patch application report for D3757 (id=13228)

Could not rebase; Attempt merge onto 7d332f5967...

Updating 7d332f59..f954714d
Fast-forward
 swh/storage/cassandra/common.py     |   8 +-
 swh/storage/cassandra/converters.py |  37 ++-
 swh/storage/cassandra/cql.py        | 541 +++++++++++++++++-------------------
 swh/storage/cassandra/model.py      | 196 +++++++++++++
 swh/storage/cassandra/schema.py     |   1 -
 swh/storage/cassandra/storage.py    | 174 +++++++-----
 swh/storage/in_memory.py            |   4 +-
 swh/storage/interface.py            |   4 +-
 swh/storage/storage.py              |   2 +-
 swh/storage/tests/test_cassandra.py |  30 +-
 10 files changed, 602 insertions(+), 395 deletions(-)
 create mode 100644 swh/storage/cassandra/model.py
Changes applied before test
commit f954714d95fa3e2124fbeddd3e81ad09e18ca313
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Aug 10 21:36:41 2020 +0200

    Fix type of snapshot_count_branches.

commit 319de05d5fbebbebb47532209490a2f8380f5343
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Aug 10 21:33:00 2020 +0200

    cassandra.cql: Use static dataclasses instead of generating namedtuples on the fly.
    
    Before this commit, python-cassandra used the default row factory,
    which creates anonymous named tuple on each query, which makes it
    impossible to type CqlRunner properly.
    
    This commit replaces the row factory with dict_factory, which creates
    only dicts, and converts them to well-defined dataclasses.
    Additionally, this stop leaking python-cassandra internals to
    cassandra.storage.
    
    This also has some great side-effects:
    
    * methods of CqlRunner are now consistent with each other (eg. _add_one
      methods used to be a mix of objects, dictionaries, and taking each value
      as argument)
    * it will allow me to deduplicate more codes in further commits (I
      already deduplicated insertions methods to use self._add_one, as
      it was meant on the initial write of this class)
    * CqlRunner no longer needs to define lists with column names, they are
      automatically detected from the dataclasses

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/727/ for more details.

anlambert accepted this revision.Aug 11 2020, 1:21 PM
This revision is now accepted and ready to land.Aug 11 2020, 1:21 PM
This revision was automatically updated to reflect the committed changes.