Differential D4018
Make all storages compatible with buffering Authored by tenma on Sep 23 2020, 1:14 PM.
Details
storages because buffering would crash if storages where ordered differently:
Dynamic features based on dicts of literals (e.g. object buffers, statistics) Depends on D4017
Diff Detail
Event TimelineComment Actions Build has FAILED Patch application report for D4018 (id=14170)Could not rebase; Attempt merge onto 30cdb78f17... Updating 30cdb78f..b8367d18 Fast-forward swh/storage/buffer.py | 147 +++++++++++++++++++++----------------- swh/storage/cassandra/storage.py | 4 +- swh/storage/filter.py | 12 +++- swh/storage/interface.py | 6 +- swh/storage/postgresql/storage.py | 4 +- swh/storage/retry.py | 6 +- swh/storage/validate.py | 6 ++ 7 files changed, 105 insertions(+), 80 deletions(-) Changes applied before testcommit b8367d1899a6adeb06893d098e544f9dec196673
Author: tenma <tenma+swh@mailbox.org>
Date: Wed Sep 23 12:33:04 2020 +0200
Make all storages compatible with buffering
- implement buffering interface (which should be made explicit) in all
storages because buffering would crash if storages where ordered differently:
flush and clear_buffers, which just chain-call for non-buffering storages
- improve typing of the buffering interface, removing spurious Optionals
- improve BufferedProxyStorage typing with literals
Dynamic features based on dicts of literals (e.g. object buffers, statistics)
should be made more static to catch many potential mistakes.
commit aa7ca077a4c516f3fa30e45b0cf880a519232743
Author: tenma <tenma+swh@mailbox.org>
Date: Tue Sep 22 17:57:24 2020 +0200
Improve code quality and doc in BufferedProxyStorage
- better names related to the object buffers
- extracted parameter dicts from the constructor
- used more generic typing in function parameters and more specific in
other contexts in order to apply the principle of robustnessLink to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/964/ Comment Actions Found how to do write passtrough functions while having good default handling and keep the signature checker happy. A bit hacky. Comment Actions Build is green Patch application report for D4018 (id=14201)Could not rebase; Attempt merge onto c97b23bcd7... Merge made by the 'recursive' strategy. swh/storage/buffer.py | 149 +++++++++++++++++++---------------- swh/storage/cassandra/storage.py | 17 +++- swh/storage/filter.py | 12 ++- swh/storage/interface.py | 16 +++- swh/storage/postgresql/storage.py | 16 +++- swh/storage/retry.py | 10 +-- swh/storage/tests/test_postgresql.py | 4 +- swh/storage/validate.py | 6 ++ 8 files changed, 143 insertions(+), 87 deletions(-) Changes applied before testcommit 8dfbb44e6f0219b70d9a1e2ca5fe4f6fc967d9b6
Merge: c97b23bc 14fa58ad
Author: Jenkins user <jenkins@localhost>
Date: Wed Sep 23 22:18:47 2020 +0000
Merge branch 'diff-target' into HEAD
commit 14fa58ad90d284219a414e88d730fd8d5c35f40c
Author: tenma <tenma+swh@mailbox.org>
Date: Wed Sep 23 12:33:04 2020 +0200
Make all storages compatible with buffering
- implement buffering interface (which should be made explicit) in all
storages because buffering would crash if storages where chained in some way:
flush and clear_buffers, which just chain-call for non-buffering storages
- improve typing of the buffering interface, removing spurious Optionals
- improve BufferedProxyStorage by typing literals
Dynamic features based on dicts of literals (e.g. object buffers, statistics)
should be made more static to catch many potential mistakes.
commit a84fb99671564f731a86eee0ae6190632f3b04e8
Author: tenma <tenma+swh@mailbox.org>
Date: Tue Sep 22 17:57:24 2020 +0200
Improve code quality and doc in BufferedProxyStorage
- better names related to the object buffers
- extracted parameter dicts from the constructor
- used more generic typing in function parameters and more specific in
other contexts in order to apply the principle of robustnessSee https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/967/ for more details. Comment Actions Build is green Patch application report for D4018 (id=14203)Rebasing onto e37f63930b... Current branch diff-target is up to date. Changes applied before testcommit 19c66e3f730cc0c79ac8200bf47db224179864bf
Author: tenma <tenma+swh@mailbox.org>
Date: Wed Sep 23 12:33:04 2020 +0200
Make all storages compatible with buffering
- implement buffering interface (which should be made explicit) in all
storages because buffering would crash if storages where chained in some way:
flush and clear_buffers, which just chain-call for non-buffering storages
- improve typing of the buffering interface, removing spurious Optionals
- improve BufferedProxyStorage by typing literals
Dynamic features based on dicts of literals (e.g. object buffers, statistics)
should be made more static to catch many potential mistakes.See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/969/ for more details. Comment Actions This diff contains three unrelated changes:
They should be three different diffs. As for specific comments on each of these changes:
|