Details
Details
- Reviewers
anlambert - Group Reviewers
Reviewers - Commits
- rDSTOda287313765d: cassandra: Fix content_missing_per_sha1 when its parameter has length != 1.
Diff Detail
Diff Detail
- Repository
- rDSTO Storage manager
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
Build is green
Patch application report for D3794 (id=13313)
Could not rebase; Attempt merge onto 6675286c4f...
Updating 6675286c..0df48cd2 Fast-forward swh/storage/cassandra/cql.py | 131 ++- swh/storage/cassandra/model.py | 35 +- swh/storage/cassandra/storage.py | 63 +- swh/storage/db.py | 17 - swh/storage/in_memory.py | 1679 ++++++++++------------------------ swh/storage/interface.py | 31 - swh/storage/replay.py | 2 +- swh/storage/storage.py | 12 - swh/storage/tests/test_api_client.py | 45 +- swh/storage/tests/test_filter.py | 2 +- swh/storage/tests/test_in_memory.py | 144 ++- swh/storage/tests/test_replay.py | 67 +- swh/storage/tests/test_retry.py | 3 +- swh/storage/tests/test_storage.py | 191 ++-- swh/storage/writer.py | 4 +- 15 files changed, 870 insertions(+), 1556 deletions(-)
Changes applied before test
commit 0df48cd2518eea8cc4c512b5c6cf01943d2449c6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Fri Aug 14 15:15:54 2020 +0200
cassandra: Fix content_missing_per_sha1 when its parameter has length != 1.
commit 12c2117a64aaf086e3fe39746013ac167266d56a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Aug 13 11:06:14 2020 +0200
in_memory: Remove dead code.
commit 2f4bed8e8416073d65e3bc747b31465e4445abea
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 17:41:43 2020 +0200
in_memory: Remove InMemoryStorage.*metadata_* and implement InMemoryCqlRunner.*metadata_*
commit fa3455b218ea31bd6cb25bbed2d86bc409e5e384
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 17:26:03 2020 +0200
in_memory: Remove InMemoryStorage.origin_visit_* and implement InMemoryCqlRunner.origin_visit_*
commit da8e69d5728b5ef7bdb91e66bd939e95d346a616
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 17:24:58 2020 +0200
cassandra.cql: reorder origin_visit_* and origin_visit_status_* methods to be properly grouped.
commit fe2718e461086d29070bb8fb72d39baf64d7a74e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 17:23:55 2020 +0200
Remove unused arguments of CqlRunner.origin_visit_status_get.
commit 91777fbdb7ec89368ba51697f9444a47942e57ba
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 16:45:17 2020 +0200
in_memory: Remove InMemoryStorage.origin_* and implement InMemoryCqlRunner.origin_*
commit 9c04650a34f282eadbffc1458077d335eebf1ae5
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 16:21:01 2020 +0200
in_memory: Remove InMemoryStorage.snapshot_* and implement InMemoryCqlRunner.snapshot_*
commit 207afdfe0d168c85230d96f88424589a21b8ade1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 15:38:14 2020 +0200
Remove endpoint snapshot_get_by_origin_visit.
It's not used anywhere.
commit f7773c57d0e16adbbae12674e411e10505c97bc6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 15:27:15 2020 +0200
in_memory: Remove InMemoryStorage.release_* and implement InMemoryCqlRunner.release_*
commit 25bb491f71754efa7fb9782081fc46bfb0741e65
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 15:23:40 2020 +0200
in_memory: Remove InMemoryStorage.revision_* and implement InMemoryCqlRunner.revision_*
commit ea964da52b73014930fbc8d2d7b580cff77055e8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 14:40:20 2020 +0200
in_memory: Remove InMemoryStorage.directory_* and implement InMemoryCqlRunner.directory_*
commit d78264397e8a8fa3fdb5d5deb576098ff8e91a06
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 14:01:22 2020 +0200
in_memory: Remove InMemoryStorage.skipped_content_* and implement InMemoryCqlRunner.skipped_content_*
commit d92ab1dce1f0cbaf1b5c6483f5dc245ef25c4c9d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 13:54:32 2020 +0200
in_memory: Remove InMemoryStorage.content_* and implement InMemoryCqlRunner.content_*
commit 028ed6172fa17a58f844e1e3a824e03787a527f1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 13:44:50 2020 +0200
in_memory: make object_find_by_sha1_git merge results from the CassandraStorage.
For now this has no effect. However, in the near future, the CassandraStorage
will be in charge of some object types, so we need to merge objects
found in the CassandraStorage and those found directly in the InMemoryStorage.
commit ce50ee582d7b6ef07f54632f0bbd27b9a5883a3e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 13:41:42 2020 +0200
in_memory: Add InMemoryCqlRunner, a class that emulates cassandra.cql.CqlRunner without Cassandra.
For now it's only used for object counters; but future commits will
progressively move the in-memory's storage features to it.
commit d1cb98a120b5b8ffcac1f2927a12512433ef3e3f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 13:34:30 2020 +0200
Make InMemoryStorage inherit from CassandraStorage.
This has no effect for now, other than deduplicating a method
and causing a name clash.
commit 1c1c79636aa08211be6a5e63aa2b6267aa1b756e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 13:24:33 2020 +0200
in_memory: Add class Table, which emulates a Cassandra table.
It will be used to implement the in-memory storage as a backend for the
cassandra storage.
commit a0037c5d12a5694bfab749896db8546b7176b633
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 13:16:44 2020 +0200
cassandra.cql: Fix return type of stat_counters.
commit f53cc6fc8606e9f97af0c3eea0ce0d7ec46d486a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 13:16:13 2020 +0200
cassandra.model: Add PARTITION_KEY and CLUSTERING_KEY to the model classes.
They will be used by the in-mem implementation of CqlRunner.
commit dd320a63b32b0e79c6f90a83236d607093219359
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 20:25:23 2020 +0200
cassandra: Make origin_visit_get_latest filter using any status of a visit, instead of just the last.
This fixes a mismatch in behavior with the pg and the in-mem storages
commit 1d45c5a0debc2e45ceee5effe565aab36905eb1c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Wed Aug 12 19:08:17 2020 +0200
cassandra: Fix wrong algo reported in HashCollision, because of variable shadowing.See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/787/ for more details.
Comment Actions
Build is green
Patch application report for D3794 (id=13316)
Rebasing onto 6675286c4f...
Current branch diff-target is up to date.
Changes applied before test
commit da287313765da62d34dc0dca7c22dbd61f40504f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Fri Aug 14 15:15:54 2020 +0200
cassandra: Fix content_missing_per_sha1 when its parameter has length != 1.See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/788/ for more details.