Page MenuHomeSoftware Heritage
Feed Advanced Search

Oct 11 2019

douardda created D2126: api: fix swh.core.api.decode_request on python 3.5.
Oct 11 2019, 1:25 PM
douardda committed rDCOREec7c3f84d114: api: removed deprecated bw-compat classes (authored by douardda).
api: removed deprecated bw-compat classes
Oct 11 2019, 10:42 AM
douardda closed D2108: api: removed deprecated bw-compat classes.
Oct 11 2019, 10:42 AM
douardda committed rDCOREf85802049637: api/tests: split test_api.py in 2 files and rewrite tests using pytest (authored by douardda).
api/tests: split test_api.py in 2 files and rewrite tests using pytest
Oct 11 2019, 10:42 AM
douardda closed D2107: api/tests: split test_api.py in 2 files and rewrite tests using pytest.
Oct 11 2019, 10:42 AM
douardda committed rDCORE9448d974c3cc: api: use content negotiation in RPCServerApp's automagic endpoint injection (authored by douardda).
api: use content negotiation in RPCServerApp's automagic endpoint injection
Oct 11 2019, 10:42 AM
douardda closed D2106: api: use content negotiation in RPCServerApp's automagic endpoint injection.
Oct 11 2019, 10:42 AM
douardda committed rDCORE36ac7a696554: api: make backend_factory argument of RPCServerApp() optional (authored by douardda).
api: make backend_factory argument of RPCServerApp() optional
Oct 11 2019, 10:42 AM
douardda closed D2105: api: make backend_factory argument of RPCServerApp() optional.
Oct 11 2019, 10:42 AM

Oct 10 2019

douardda added a comment to D2119: test_api_client: refactor the code for new pytest based infra.

Note that this Diff require also D2116

Oct 10 2019, 5:51 PM
douardda created D2119: test_api_client: refactor the code for new pytest based infra.
Oct 10 2019, 5:50 PM
douardda created D2118: api: add (missing) refresh_stat_counters() endpoint.
Oct 10 2019, 5:50 PM
douardda created D2117: tests/algos: rewrite test_snapshot with pytest.
Oct 10 2019, 5:49 PM
douardda updated the diff for D2096: in_memory: fix handling of 'hidden' content in InMemoryStorage.

rebase

Oct 10 2019, 5:48 PM
douardda updated the diff for D2095: test_in_memory: adapt tests for the InMemoryStorage to new storage test infra.

Update the code, tests in test_in_memory should now pass

Oct 10 2019, 5:48 PM
douardda added a comment to D2096: in_memory: fix handling of 'hidden' content in InMemoryStorage.

Eh, it needs a test ^^

Oct 10 2019, 5:42 PM
douardda created D2116: api/tests: add a few test for the whole rpc client/server stack.
Oct 10 2019, 4:31 PM
douardda updated the diff for D2108: api: removed deprecated bw-compat classes.

rebased

Oct 10 2019, 4:30 PM
douardda updated the diff for D2107: api/tests: split test_api.py in 2 files and rewrite tests using pytest.

rebased, get rid of pytest-flask and move some code in pytest_plugin.py

Oct 10 2019, 4:29 PM
douardda updated the diff for D2106: api: use content negotiation in RPCServerApp's automagic endpoint injection.

rebase

Oct 10 2019, 4:22 PM
douardda updated the diff for D2105: api: make backend_factory argument of RPCServerApp() optional.

Update the docstring

Oct 10 2019, 4:21 PM
douardda committed rDCOREf4db00bca020: api: fix abc's deprecation warnings (authored by douardda).
api: fix abc's deprecation warnings
Oct 10 2019, 4:16 PM
douardda closed D2104: api: fix abc's deprecation warnings.
Oct 10 2019, 4:16 PM
douardda accepted D2076: gnu: Separate listing and parsing logic and add integration test.
Oct 10 2019, 10:15 AM · Origin-GNU

Oct 9 2019

douardda accepted D2077: cgit.tests: Check the tasks from the scheduler.
Oct 9 2019, 6:11 PM · Origin-CGit, CGit lister
douardda created D2108: api: removed deprecated bw-compat classes.
Oct 9 2019, 5:47 PM
douardda created D2107: api/tests: split test_api.py in 2 files and rewrite tests using pytest.
Oct 9 2019, 5:40 PM
douardda created D2106: api: use content negotiation in RPCServerApp's automagic endpoint injection.
Oct 9 2019, 5:40 PM
douardda created D2105: api: make backend_factory argument of RPCServerApp() optional.
Oct 9 2019, 5:39 PM
douardda created D2104: api: fix abc's deprecation warnings.
Oct 9 2019, 5:39 PM
douardda committed rDCORE57368059acd4: test_async: make test_async.py compatible with the pytest-flask plugin (authored by douardda).
test_async: make test_async.py compatible with the pytest-flask plugin
Oct 9 2019, 5:39 PM
douardda closed D2099: test_async: make test_async.py compatible with the pytest-flask plugin.
Oct 9 2019, 5:39 PM
douardda created D2099: test_async: make test_async.py compatible with the pytest-flask plugin.
Oct 9 2019, 3:29 PM
douardda added a comment to D2082: Add pytest fixture to allow mocking requests with data file.
In D2082#48297, @olasd wrote:

From my tests and reading documentation further (pytest, tox), the issue really is a limitation of the way we're using the tox + pytest combination: we're running the tests on the checked out version of the module, instead of the version that's installed by tox (which is what pytest expects). If we adapt tox.ini to run tests on the installed version of the module (by running pytest [...] {envsitepackagesdir}/swh/core/..., then we can ship the pytest plugin wherever we want without triggering the dreaded ImportMismatchError.

well this is the default behavior of tox: execute tests from local working directory against installed python package. However if this default makes sense for project which tests are not part of the package itself, we may indeed consider executing tests that are installed with the package. It would probably be a sensible choice.

I am inclined to give this a try before landing this diff. Thanks for pointing this out.

Oct 9 2019, 10:13 AM
douardda added a comment to D2082: Add pytest fixture to allow mocking requests with data file.
In D2082#48297, @olasd wrote:

From my tests and reading documentation further (pytest, tox), the issue really is a limitation of the way we're using the tox + pytest combination: we're running the tests on the checked out version of the module, instead of the version that's installed by tox (which is what pytest expects). If we adapt tox.ini to run tests on the installed version of the module (by running pytest [...] {envsitepackagesdir}/swh/core/..., then we can ship the pytest plugin wherever we want without triggering the dreaded ImportMismatchError.

Oct 9 2019, 9:59 AM

Oct 8 2019

douardda created D2096: in_memory: fix handling of 'hidden' content in InMemoryStorage.
Oct 8 2019, 5:06 PM
douardda updated the diff for D2095: test_in_memory: adapt tests for the InMemoryStorage to new storage test infra.

apply TestStorageCommonProp renaming

Oct 8 2019, 5:03 PM
douardda updated the summary of D2094: test_storage: rename TestStorageCommonProp as TestStorageGeneratedData.
Oct 8 2019, 4:57 PM
douardda updated the summary of D2092: test_storage: move db specific methods from main TestStorage to TestPgStorage.
Oct 8 2019, 4:56 PM
douardda updated the summary of D2091: test_storage: kill StorageTestDbFixture class.
Oct 8 2019, 4:55 PM
douardda created D2095: test_in_memory: adapt tests for the InMemoryStorage to new storage test infra.
Oct 8 2019, 4:54 PM
douardda created D2094: test_storage: rename TestStorageCommonProp as TestStorageGeneratedData.
Oct 8 2019, 4:53 PM
douardda created D2093: test_storage: make test_origin_metadata_get more robust.
Oct 8 2019, 4:53 PM
douardda created D2092: test_storage: move db specific methods from main TestStorage to TestPgStorage.
Oct 8 2019, 4:53 PM
douardda created D2091: test_storage: kill StorageTestDbFixture class.
Oct 8 2019, 4:52 PM
douardda created D2090: tests: refactor main storage tests.
Oct 8 2019, 4:52 PM
douardda requested changes to D2082: Add pytest fixture to allow mocking requests with data file.
Oct 8 2019, 1:43 PM
douardda accepted D2074: gnu.tests: Check gnu lister.
Oct 8 2019, 1:39 PM · Origin-GNU, Lister

Oct 7 2019

douardda requested changes to D2074: gnu.tests: Check gnu lister.
Oct 7 2019, 10:24 AM · Origin-GNU, Lister
douardda requested changes to D2076: gnu: Separate listing and parsing logic and add integration test.
Oct 7 2019, 10:24 AM · Origin-GNU
douardda accepted D2075: simple_lister: Extract common behavior in base class.
Oct 7 2019, 10:13 AM
douardda accepted D2077: cgit.tests: Check the tasks from the scheduler.
Oct 7 2019, 10:13 AM · Origin-CGit, CGit lister

Oct 3 2019

douardda added inline comments to D2058: PersistentId constructor parameters validation.
Oct 3 2019, 5:23 PM

Oct 2 2019

douardda accepted D2055: typing: minimal changes to make a no-op mypy run pass.
Oct 2 2019, 10:51 AM
douardda added a comment to D2032: typing: minimal changes to make a no-op mypy run pass.

where should the mypy_django_plugin dependency be? looks like this is required.

Oct 2 2019, 10:49 AM

Oct 1 2019

douardda accepted D2051: Make the null compression use the "none" identifier instead of None.

See my comment, just in case.

Oct 1 2019, 9:44 AM
douardda accepted D2050: Implement pluggable compression in ObjStoragePathSlicing.
Oct 1 2019, 9:42 AM
douardda accepted D2047: Support pluggable compression in azure objstorage.

Gheez

Oct 1 2019, 9:38 AM
douardda accepted D2045: Move to streaming decompression API.

Adding tests 2 or 3 diffs after is a bold move... UNFAIR :-)

Oct 1 2019, 9:38 AM
douardda accepted D2049: Add tests for the behavior on trailing data after the compressed blob.

Oh! They are here. Ok then.

Oct 1 2019, 9:36 AM
douardda accepted D2048: Add raw compression test to seaweedfs backend.
Oct 1 2019, 9:35 AM
douardda requested changes to D2047: Support pluggable compression in azure objstorage.

Ok but also lacks a test case for trailing garbage generated Error(), I'm afraid.

Oct 1 2019, 9:34 AM
douardda requested changes to D2045: Move to streaming decompression API.

I would expect a (series of) test(s) with statements like with pytest.raises(Error, match='Corrupt object') somewhere in this diff.

Oct 1 2019, 9:32 AM
douardda accepted D2044: Add gzip compression support.
Oct 1 2019, 9:27 AM
douardda accepted D2043: Add compression tests to seaweedfs backend.
Oct 1 2019, 9:26 AM
douardda accepted D2042: Refactor cloud objstorage tests to avoid repeating themselves.
Oct 1 2019, 9:25 AM

Sep 30 2019

douardda added inline comments to D2023: typing: minimal changes to make a no-op mypy run pass.
Sep 30 2019, 1:20 PM

Sep 27 2019

douardda committed rDSTO9aa6b9d2b100: journal_writer: use journal writer from swh.journal (authored by douardda).
journal_writer: use journal writer from swh.journal
Sep 27 2019, 1:31 PM
douardda closed D1993: journal_writer: use journal writer from swh.journal.
Sep 27 2019, 1:31 PM
douardda added inline comments to D1993: journal_writer: use journal writer from swh.journal.
Sep 27 2019, 1:16 PM
douardda updated the diff for D1993: journal_writer: use journal writer from swh.journal.

remove journal_writer.py

Sep 27 2019, 11:10 AM

Sep 25 2019

douardda committed rDLDBASE7521f01a990e: package.tests: add a local_get fixture that install the get_response_cb mock (authored by douardda).
package.tests: add a local_get fixture that install the get_response_cb mock
Sep 25 2019, 10:25 AM
douardda committed rDLDBASE3a964bb4179d: package.tests: make get_response_cb generate a 404 if the requested file does… (authored by douardda).
package.tests: make get_response_cb generate a 404 if the requested file does…
Sep 25 2019, 10:25 AM
douardda committed rDLDBASE829345d0b11e: package.tests: improve get_response_cb docstring and add a bit of logging (authored by douardda).
package.tests: improve get_response_cb docstring and add a bit of logging
Sep 25 2019, 10:25 AM

Sep 24 2019

douardda committed rDLDBASEb1935a9a8556: package.tests: use a fixture to configure the loaders (authored by douardda).
package.tests: use a fixture to configure the loaders
Sep 24 2019, 4:31 PM

Sep 23 2019

douardda added inline comments to D1993: journal_writer: use journal writer from swh.journal.
Sep 23 2019, 10:12 AM
douardda requested changes to D2021: Add a bencode encoder..
Sep 23 2019, 9:58 AM

Sep 18 2019

douardda closed T2006: backfiller is broken for revision as Resolved.

closed by 1144f7dd1552

Sep 18 2019, 5:58 PM · Journal
douardda committed rDJNL1144f7dd1552: backfill: fix revision's converter (authored by douardda).
backfill: fix revision's converter
Sep 18 2019, 5:56 PM
douardda closed D2006: backfill: fix revision's converter.
Sep 18 2019, 5:56 PM
douardda accepted D2004: Mock more things in MockedKafkaConsumer.

Ok but a comment would be welcome

Sep 18 2019, 5:56 PM
douardda created D2006: backfill: fix revision's converter.
Sep 18 2019, 5:50 PM
douardda triaged T2006: backfiller is broken for revision as High priority.
Sep 18 2019, 2:31 PM · Journal

Sep 17 2019

douardda committed rDJNL6676f667a5d7: writer: rename writer.direct.DirectKafkaWriter as writer.kafka. (authored by douardda).
writer: rename writer.direct.DirectKafkaWriter as writer.kafka.
Sep 17 2019, 5:33 PM
douardda committed rDJNL9d505445b535: writer: move the InMemoryJournalWriter from swh.storage in swh.journal (authored by douardda).
writer: move the InMemoryJournalWriter from swh.storage in swh.journal
Sep 17 2019, 5:33 PM
douardda closed D1992: writer: move the InMemoryJournalWriter from swh.storage in swh.journal.
Sep 17 2019, 5:33 PM
douardda updated the diff for D1993: journal_writer: use journal writer from swh.journal.

also update test_api_client.py

Sep 17 2019, 1:48 PM
douardda updated the diff for D1992: writer: move the InMemoryJournalWriter from swh.storage in swh.journal.

writer: rename writer.direct.DirectKafkaWriter as writer.kafka.KafkaJournalWriter

Sep 17 2019, 1:18 PM
douardda added a comment to T2003: Content replayer may try to copy objects before they are available from an objstorage.

My guts on this task tell me that what we need (what we really really need) is a 3rd solution:

Sep 17 2019, 12:20 PM · Journal
Herald added a reviewer for D1993: journal_writer: use journal writer from swh.journal: Reviewers.
Sep 17 2019, 11:42 AM
Herald added a reviewer for D1992: writer: move the InMemoryJournalWriter from swh.storage in swh.journal: Reviewers.
Sep 17 2019, 11:40 AM
douardda added a reviewer for D1990: Phabricator: Prevent erroneous scheduler tasks disabling: Reviewers.
Sep 17 2019, 10:10 AM

Sep 12 2019

douardda added a comment to T1389: Implement a base "package" loader for package managers.
Package managerPackages listing url Package metadata urlPackage source tarball url
RubygemsNo public api endpoint availablehttps://rubygems.org/api/v2/rubygems/[package]/versions/[version].jsonavailable in the medata
Sep 12 2019, 11:08 AM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Sep 11 2019

douardda created P528 (An Untitled Masterwork).
Sep 11 2019, 3:18 PM

Sep 10 2019

douardda created P527 replayer TB.
Sep 10 2019, 5:14 PM
douardda created P526 replayer.
Sep 10 2019, 1:50 PM

Sep 9 2019

douardda committed rCDFP6305729dbdd0: first steps in a docker based swh mirror setup (authored by douardda).
first steps in a docker based swh mirror setup
Sep 9 2019, 11:06 AM
douardda added inline comments to D1967: core/loader: Fix get_save_data_path implementation.
Sep 9 2019, 10:29 AM
douardda requested changes to D1970: pid maps: add limited support for updatable maps.

Looks globally ok to me, so feel free to ignore my comments.

Sep 9 2019, 10:07 AM

Sep 6 2019

douardda committed rDCORE220d1c470c18: statsd: protect access to the statsd's socket (authored by douardda).
statsd: protect access to the statsd's socket
Sep 6 2019, 11:18 AM