In D3010#72991, @douardda wrote:In D3010#72943, @ardumont wrote:In test_replay, get rid of test_storage_play and
test_storage_play_with_collision because these tests do require a proper
kafka setup to run, which we do not want to require kafka in the
swh.storage package.hu ho...
i'm not fine with getting rid of test_storage_play*.See for example D3015 which moves the hash collision policy within the retry proxy.
To ensure this continued to be ok, i've kept the test_storage_play_with_collision as is, only changed the test config with retry proxy and the test is green... (and it is, well locally because right now it's in diff).
My point is those are important tests.I understand, however, these 2 tests, as currently written, are too far "integration tests" rather than "unit tests". And it makes the testing stack too complex.
So IMHO we want "unit" tests dedicated to parts of this stack that handle collisions 9typically without kafka being involved, swh-storage side) where there are useful, and integration tests (in the docker-dev project typically) to check the whole stack behaves properly.
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Advanced Search
Apr 21 2020
Apr 14 2020
rebase
In D3010#72943, @ardumont wrote:In test_replay, get rid of test_storage_play and
test_storage_play_with_collision because these tests do require a proper
kafka setup to run, which we do not want to require kafka in the
swh.storage package.hu ho...
i'm not fine with getting rid of test_storage_play*.See for example D3015 which moves the hash collision policy within the retry proxy.
To ensure this continued to be ok, i've kept the test_storage_play_with_collision as is, only changed the test config with retry proxy and the test is green... (and it is, well locally because right now it's in diff).
My point is those are important tests.
Apr 10 2020
add the commit "cli: rename the command 'backfiller' as 'backfill'" in the diff
rebase and improved to make it also compatible with swj.journal 0.0.29
Apr 9 2020
Apr 8 2020
rephrase the commit msg
rebase
rebase
rebase
rebase
Apr 7 2020
why calling it _wfi_timeout (with the leading _)?
It seems not to be necessary (besides not being the proper fix, if any)
In T2310#43131, @ardumont wrote:Thanks for the questions. I'm unsure about some questions and i replied as best
i could.do we allow an OriginVisitUpdate(status='ongoing', snaphost=None)? what would
be the meaning of this?Yes. It means "loading started, so no snapshot yet".
That sounds sensible ;)
In T2310#43129, @vlorentz wrote:We currently don't have "created" (so no "start" either), but it would make sense to create it.
Regarding this model, a few questions come to my mind:
- do we allow an OriginVisitUpdate(status='ongoing', snaphost=None)? what would be the meaning of this? or do we enforce one just after the created step to model the start transition?
This could mean these things:
- on a first update, to mean the visit was created (but we don't need it if we have a "created" state)
Apr 6 2020
As I understand this, an origin visit, consisting in one OriginVisit object plus a list of OriginVisitUpdate represent the process of visiting an origin to load its content in the archive.
Apr 3 2020
rebase + add comment on @overload usage
rebase + modify tox as well
rebased
closed by f6a398f99a5fa57e590fc4dfaf4fdc880eefba9d