Page MenuHomeSoftware Heritage
Feed Advanced Search

Apr 28 2020

douardda updated the diff for D3073: Refactor the sphinx-dev build environment to get rid of the link-stamp step.

fixes according to comments

Apr 28 2020, 5:35 PM
douardda committed rDDOC3001e735b592: Add missing dependency in swh-scanner in requirement files (authored by douardda).
Add missing dependency in swh-scanner in requirement files
Apr 28 2020, 5:33 PM
douardda closed D3072: Add missing dependency in swh-scanner in requirement files.
Apr 28 2020, 5:33 PM
douardda added a comment to D3073: Refactor the sphinx-dev build environment to get rid of the link-stamp step.

While individual doc build can still be launched in each subproject, I am fine with it.

Apr 28 2020, 5:19 PM
douardda created D3076: Use swh.model's sha1 hypothesis strategy in test_replay.py.
Apr 28 2020, 5:15 PM
douardda committed rDOBJSRPLe9c46618fd25: Initial version of the swh.objstorage.replayer package (authored by douardda).
Initial version of the swh.objstorage.replayer package
Apr 28 2020, 5:15 PM
douardda closed D3071: Initial version of the swh.objstorage.replayer package.
Apr 28 2020, 5:15 PM
douardda updated the diff for D3071: Initial version of the swh.objstorage.replayer package.

trigger jenkins (maybe)

Apr 28 2020, 5:11 PM
douardda updated the diff for D3071: Initial version of the swh.objstorage.replayer package.

trigger jenkins

Apr 28 2020, 4:58 PM
douardda committed rCJSWH2b1ab44aed8e: Enable Jenkins for swh-objstorage-replayer. (authored by douardda).
Enable Jenkins for swh-objstorage-replayer.
Apr 28 2020, 4:51 PM
douardda closed D3075: Enable Jenkins for swh-objstorage-replayer..
Apr 28 2020, 4:51 PM
douardda created D3075: Enable Jenkins for swh-objstorage-replayer..
Apr 28 2020, 4:43 PM
douardda updated the diff for D3071: Initial version of the swh.objstorage.replayer package.

update index.rst

Apr 28 2020, 4:37 PM
douardda created D3074: Add swh-objstorage-replayer.
Apr 28 2020, 4:33 PM
douardda created D3073: Refactor the sphinx-dev build environment to get rid of the link-stamp step.
Apr 28 2020, 4:33 PM
douardda created D3072: Add missing dependency in swh-scanner in requirement files.
Apr 28 2020, 4:33 PM
douardda updated the diff for D3071: Initial version of the swh.objstorage.replayer package.

and remaining references to the storage fixture in test_cli...

Apr 28 2020, 10:11 AM
douardda updated the diff for D3071: Initial version of the swh.objstorage.replayer package.

forgot tox.ini

Apr 28 2020, 10:07 AM
douardda updated the diff for D3071: Initial version of the swh.objstorage.replayer package.

fixes according to anlambert's comments

Apr 28 2020, 10:05 AM
douardda added inline comments to D3071: Initial version of the swh.objstorage.replayer package.
Apr 28 2020, 9:51 AM

Apr 27 2020

douardda created D3071: Initial version of the swh.objstorage.replayer package.
Apr 27 2020, 5:22 PM
douardda committed rDOBJSRPL88da644b4949: import template from swh-py-template (init-py-repo) (authored by douardda).
import template from swh-py-template (init-py-repo)
Apr 27 2020, 5:16 PM
douardda closed D3063: Normalize the handling of the config file loading in cli.

closed by 3043177f79b24d2c5791c3196bd17116f83086bc

Apr 27 2020, 3:20 PM
douardda committed rDOBJS3043177f79b2: Normalize the handling of the config file loading in cli (authored by douardda).
Normalize the handling of the config file loading in cli
Apr 27 2020, 3:19 PM
douardda committed rDOBJSbd2b5e651aed: Move the content of swh/objstorage/__init__.py in swh/objstorage/factory.py (authored by douardda).
Move the content of swh/objstorage/__init__.py in swh/objstorage/factory.py
Apr 27 2020, 3:19 PM
douardda closed D3062: Move the content of swh/objstorage/__init__.py in swh/objstorage/factory.py.
Apr 27 2020, 3:19 PM
douardda added inline comments to D3063: Normalize the handling of the config file loading in cli.
Apr 27 2020, 3:01 PM
douardda added inline comments to D3063: Normalize the handling of the config file loading in cli.
Apr 27 2020, 2:58 PM
douardda added inline comments to D3063: Normalize the handling of the config file loading in cli.
Apr 27 2020, 2:50 PM
douardda added inline comments to D3063: Normalize the handling of the config file loading in cli.
Apr 27 2020, 2:48 PM
douardda updated the diff for D3062: Move the content of swh/objstorage/__init__.py in swh/objstorage/factory.py.

update copyright dates of modifed files + fix a typo

Apr 27 2020, 1:57 PM

Apr 24 2020

douardda added a task to D3062: Move the content of swh/objstorage/__init__.py in swh/objstorage/factory.py: T2355: Make swh-journal independent from swh-storage or swh-objstorage.
Apr 24 2020, 3:54 PM
douardda added a revision to T2355: Make swh-journal independent from swh-storage or swh-objstorage: D3062: Move the content of swh/objstorage/__init__.py in swh/objstorage/factory.py.
Apr 24 2020, 3:54 PM · Object storage, Storage manager, Journal
douardda created D3063: Normalize the handling of the config file loading in cli.
Apr 24 2020, 3:52 PM
douardda created D3062: Move the content of swh/objstorage/__init__.py in swh/objstorage/factory.py.
Apr 24 2020, 3:51 PM
douardda committed rDSTOecadd5373783: Remove recently added test_cli.test_rpc_serve* tests (authored by douardda).
Remove recently added test_cli.test_rpc_serve* tests
Apr 24 2020, 3:47 PM
douardda closed D3061: Remove recently added test_cli.test_rpc_serve* tests.
Apr 24 2020, 3:47 PM
douardda created D3061: Remove recently added test_cli.test_rpc_serve* tests.
Apr 24 2020, 3:19 PM
douardda created P654 (An Untitled Masterwork).
Apr 24 2020, 2:38 PM
douardda committed rDSTO038c30b40838: Adapt journal client loading to swh.journal 0.0.31 (authored by douardda).
Adapt journal client loading to swh.journal 0.0.31
Apr 24 2020, 1:37 PM
douardda closed D3058: Adapt journal client loading to swh.journal 0.0.31.
Apr 24 2020, 1:37 PM
douardda committed rDSTO4b7ba1fc6bcc: Copy the graph replayer component from swh-journal (authored by douardda).
Copy the graph replayer component from swh-journal
Apr 24 2020, 1:37 PM
douardda closed D3010: Copy the graph replayer component from swh-journal.
Apr 24 2020, 1:37 PM
douardda updated the diff for D3058: Adapt journal client loading to swh.journal 0.0.31.

rebase

Apr 24 2020, 11:59 AM
douardda updated the diff for D3010: Copy the graph replayer component from swh-journal.

clean import statements in test_cli.py

Apr 24 2020, 11:59 AM
douardda committed rDSTO5fd9b563298f: Deprecate the `config-path` argument of the `swh storage rpc-serve` command (authored by douardda).
Deprecate the `config-path` argument of the `swh storage rpc-serve` command
Apr 24 2020, 11:54 AM
douardda committed rDSTOcd32cf42538f: Normalize the configuration file handling in the `swh storage` CLI command (authored by douardda).
Normalize the configuration file handling in the `swh storage` CLI command
Apr 24 2020, 11:54 AM
douardda closed D3056: Deprecate the `config-path` argument of the `swh storage rpc-serve` command.
Apr 24 2020, 11:54 AM
douardda closed D3055: Normalize the configuration file handling in the `swh storage` CLI command.
Apr 24 2020, 11:54 AM
douardda committed rDSTO698af8ca0028: cli: rename the command 'backfiller' as 'backfill' (authored by douardda).
cli: rename the command 'backfiller' as 'backfill'
Apr 24 2020, 11:54 AM
douardda committed rDSTOb2bba450aaa8: Copy the backfiller component from swh-journal (authored by douardda).
Copy the backfiller component from swh-journal
Apr 24 2020, 11:54 AM
douardda closed D3008: Copy the backfiller component from swh-journal.
Apr 24 2020, 11:54 AM
douardda updated the diff for D3058: Adapt journal client loading to swh.journal 0.0.31.

rebase

Apr 24 2020, 11:31 AM
douardda updated the diff for D3010: Copy the graph replayer component from swh-journal.

rebase + add forgotten test_kafka_writer test file

Apr 24 2020, 11:31 AM
douardda updated the diff for D3056: Deprecate the `config-path` argument of the `swh storage rpc-serve` command.

Adapt according to ardumont's comments

Apr 24 2020, 11:29 AM
douardda added a revision to T2355: Make swh-journal independent from swh-storage or swh-objstorage: D3056: Deprecate the `config-path` argument of the `swh storage rpc-serve` command.
Apr 24 2020, 11:29 AM · Object storage, Storage manager, Journal
douardda added inline comments to D3056: Deprecate the `config-path` argument of the `swh storage rpc-serve` command.
Apr 24 2020, 11:20 AM
douardda added a comment to D3011: dataset: add graph export based on kafka.

looks fine to me, but the few minor comments. I'll let @vlorentz gives his green stamp on this.

Apr 24 2020, 11:13 AM

Apr 23 2020

douardda created D3058: Adapt journal client loading to swh.journal 0.0.31.
Apr 23 2020, 4:58 PM
douardda added a revision to T2355: Make swh-journal independent from swh-storage or swh-objstorage: D3058: Adapt journal client loading to swh.journal 0.0.31.
Apr 23 2020, 4:58 PM · Object storage, Storage manager, Journal
douardda updated the diff for D3010: Copy the graph replayer component from swh-journal.

rebased

Apr 23 2020, 4:58 PM
douardda updated the diff for D3056: Deprecate the `config-path` argument of the `swh storage rpc-serve` command.

yet more comments

Apr 23 2020, 4:50 PM
douardda updated the diff for D3056: Deprecate the `config-path` argument of the `swh storage rpc-serve` command.

Add more comments in the convoluted test code

Apr 23 2020, 4:26 PM
douardda accepted D3057: indexer.cli: Adapt get_journal_client instantiation.

ok but IMHO something should be done, some day, to "normalize" also this command so it accepts the "standard" configuration scheme (via --config-file or the SWH_CONFIG_FILE env var). But that's another task. Thanks for the fix.

Apr 23 2020, 4:24 PM
douardda added a comment to D3056: Deprecate the `config-path` argument of the `swh storage rpc-serve` command.
In D3056#74217, @olasd wrote:

Jeez. The check_rpc_serve context manager is so backwards it broke my brain a little. Took me quite a while to figure out that the thread actually runs the test rather than the server. I also think the synchronization between the test and the server initialization should be done more explicitly.

(but I don't really have a better thing to suggest right now)

Apr 23 2020, 4:12 PM
douardda added inline comments to D3008: Copy the backfiller component from swh-journal.
Apr 23 2020, 4:05 PM
douardda added inline comments to D3008: Copy the backfiller component from swh-journal.
Apr 23 2020, 4:02 PM
douardda created D3056: Deprecate the `config-path` argument of the `swh storage rpc-serve` command.
Apr 23 2020, 3:31 PM
douardda created D3055: Normalize the configuration file handling in the `swh storage` CLI command.
Apr 23 2020, 3:30 PM
douardda updated the diff for D3008: Copy the backfiller component from swh-journal.

rebase

Apr 23 2020, 1:32 PM
douardda committed rDJNLdc96335dbf5a: Normalize the get_journal_client() function (authored by douardda).
Normalize the get_journal_client() function
Apr 23 2020, 12:27 PM
douardda committed rDJNL72ea87ccf784: Move get_journal_client function to swh.journal.client (authored by douardda).
Move get_journal_client function to swh.journal.client
Apr 23 2020, 12:27 PM
douardda closed D3044: Move get_journal_client function to swh.journal.client.
Apr 23 2020, 12:27 PM
douardda updated the diff for D3044: Move get_journal_client function to swh.journal.client.

Add a commit to "normalize" the get_journal_client() function

Apr 23 2020, 11:54 AM
douardda added a comment to D3044: Move get_journal_client function to swh.journal.client.

I'd say it'd be neat to add some docstrings and type at the same time, don't you think?

Apr 23 2020, 9:50 AM
douardda added a comment to D3040: Add a split_content argument to object_dicts() and objects() strategies.

Technically, this sounds fine ;)

What is the new use case? Will we merge back the content topics into one or something?

Apr 23 2020, 9:47 AM

Apr 22 2020

douardda added a revision to T2355: Make swh-journal independent from swh-storage or swh-objstorage: D3044: Move get_journal_client function to swh.journal.client.
Apr 22 2020, 4:50 PM · Object storage, Storage manager, Journal
douardda created D3044: Move get_journal_client function to swh.journal.client.
Apr 22 2020, 4:50 PM
douardda committed rDJNL42f4a89abc72: Extract kafka-related pytest fixtures in a pytest plugin module (authored by douardda).
Extract kafka-related pytest fixtures in a pytest plugin module
Apr 22 2020, 4:48 PM
douardda closed D3043: Extract kafka-related pytest fixtures in a pytest plugin module.
Apr 22 2020, 4:48 PM
douardda updated the diff for D3043: Extract kafka-related pytest fixtures in a pytest plugin module.

improve commit message

Apr 22 2020, 4:45 PM
douardda updated the diff for D3043: Extract kafka-related pytest fixtures in a pytest plugin module.

attempt to fix the mess

Apr 22 2020, 4:17 PM
douardda added a comment to D3043: Extract kafka-related pytest fixtures in a pytest plugin module.

Also, @ardumont I agree with your other comments, but these will come later, if you may :-)

Apr 22 2020, 4:13 PM
douardda added a comment to D3043: Extract kafka-related pytest fixtures in a pytest plugin module.

It seems we duplicate the objects data (CONTENT, etc...) both in the pytest_plugin and in the journal_data module.
is that expected?

Apr 22 2020, 4:12 PM
douardda renamed T2355: Make swh-journal independent from swh-storage or swh-objstorage from Merge parts of swh-journal in swh-storage to Make swh-journal independant from swh-storage or swh-objstorage.
Apr 22 2020, 3:41 PM · Object storage, Storage manager, Journal
douardda created D3043: Extract kafka-related pytest fixtures in a pytest plugin module.
Apr 22 2020, 3:38 PM
douardda added a revision to T2355: Make swh-journal independent from swh-storage or swh-objstorage: D3043: Extract kafka-related pytest fixtures in a pytest plugin module.
Apr 22 2020, 3:38 PM · Object storage, Storage manager, Journal
douardda accepted D3030: Remove the use of pytest-kafka.

Thanks a lot

Apr 22 2020, 1:44 PM

Apr 21 2020

douardda committed rDMOD1037e88d9738: Add a split_content argument to object_dicts() and objects() strategies (authored by douardda).
Add a split_content argument to object_dicts() and objects() strategies
Apr 21 2020, 4:18 PM
douardda closed D3040: Add a split_content argument to object_dicts() and objects() strategies.
Apr 21 2020, 4:18 PM
douardda committed rDMODebd380779626: Add a blacklist_types argument to object_dicts() and objects() hypothesis… (authored by douardda).
Add a blacklist_types argument to object_dicts() and objects() hypothesis…
Apr 21 2020, 4:18 PM
douardda closed D3037: Add a blacklist_types argument to object_dicsts() and objects() hypothesis generators.
Apr 21 2020, 4:18 PM
douardda accepted D3039: deposit.loader: Fix committer date appropriately.

Fine for me. I agree the semantic of this date in the context of a deposit could be documented somehow (a simple comment may be enough).

Apr 21 2020, 3:35 PM
douardda created D3040: Add a split_content argument to object_dicts() and objects() strategies.
Apr 21 2020, 2:54 PM
douardda updated the diff for D3037: Add a blacklist_types argument to object_dicsts() and objects() hypothesis generators.

fix the test test_generation_blacklist

Apr 21 2020, 2:53 PM
douardda updated the diff for D3037: Add a blacklist_types argument to object_dicsts() and objects() hypothesis generators.

typo in the commit message

Apr 21 2020, 2:26 PM
douardda added inline comments to D3037: Add a blacklist_types argument to object_dicsts() and objects() hypothesis generators.
Apr 21 2020, 2:12 PM
douardda updated the diff for D3037: Add a blacklist_types argument to object_dicsts() and objects() hypothesis generators.

Set the default value for blacklist_types to ("origin_visit_status",)

Apr 21 2020, 12:13 PM
douardda updated the diff for D3008: Copy the backfiller component from swh-journal.

rebase *and* pick all the revisions involved

Apr 21 2020, 11:48 AM
douardda updated the diff for D3008: Copy the backfiller component from swh-journal.

rebase

Apr 21 2020, 11:48 AM