Page MenuHomeSoftware Heritage
Feed Advanced Search

May 25 2020

douardda committed rDJNL3cac9cffe243: Add support for a priviledged "channel" of topics for non-anonymized objects (authored by douardda).
Add support for a priviledged "channel" of topics for non-anonymized objects
May 25 2020, 10:34 AM
douardda closed D3172: Add support for a priviledged "channel" of topics for non-anonymized objects.
May 25 2020, 10:34 AM
douardda updated the diff for D3172: Add support for a priviledged "channel" of topics for non-anonymized objects.

typo

May 25 2020, 10:29 AM
douardda updated the diff for D3172: Add support for a priviledged "channel" of topics for non-anonymized objects.

bump dep on swh-model to 0.2

May 25 2020, 10:16 AM

May 20 2020

douardda committed rDMOD29312dff6d96: Add support for model object anonymization (authored by douardda).
Add support for model object anonymization
May 20 2020, 4:33 PM
douardda closed D3171: Add support for model object anonymization.
May 20 2020, 4:33 PM
douardda updated the diff for D3171: Add support for model object anonymization.

use ModelType instead of T for type annotation

May 20 2020, 4:30 PM
douardda updated the diff for D3172: Add support for a priviledged "channel" of topics for non-anonymized objects.

typos, rename a couple of tests, and better comments

May 20 2020, 3:40 PM
douardda updated the diff for D3171: Add support for model object anonymization.

properly annotate BaseModel.anonymize()

May 20 2020, 3:37 PM
douardda added inline comments to D3172: Add support for a priviledged "channel" of topics for non-anonymized objects.
May 20 2020, 2:59 PM
douardda updated the summary of D3171: Add support for model object anonymization.
May 20 2020, 2:48 PM
douardda updated the diff for D3171: Add support for model object anonymization.

Typos + comments/docstrings + hash on the fullname in Person.anonymize()

May 20 2020, 2:47 PM
douardda abandoned D3161: Add support for anonymized journal topics.

Replaced by D3171 + D3172

May 20 2020, 2:45 PM
douardda added inline comments to D3171: Add support for model object anonymization.
May 20 2020, 2:29 PM
douardda added a comment to D3171: Add support for model object anonymization.

Shouldn't we make anonymized objects error when their compute_hash() method is called?

May 20 2020, 2:24 PM
douardda added inline comments to D3172: Add support for a priviledged "channel" of topics for non-anonymized objects.
May 20 2020, 12:22 PM
douardda abandoned D3160: Add support for a priviledged "channel" of topics for non-anonymized objects.

Replaced by D3171 + D3172

May 20 2020, 11:22 AM
douardda created D3172: Add support for a priviledged "channel" of topics for non-anonymized objects.
May 20 2020, 11:21 AM
douardda created D3171: Add support for model object anonymization.
May 20 2020, 11:20 AM
douardda abandoned D3150: Add support for anonymized journal topics.
May 20 2020, 11:16 AM
douardda abandoned D3149: Add support for "flags" in object_type used for kafka topics.
May 20 2020, 11:15 AM

May 18 2020

douardda updated the diff for D3160: Add support for a priviledged "channel" of topics for non-anonymized objects.

typo

May 18 2020, 4:39 PM
douardda created D3161: Add support for anonymized journal topics.
May 18 2020, 1:53 PM
douardda created D3160: Add support for a priviledged "channel" of topics for non-anonymized objects.
May 18 2020, 1:47 PM

May 15 2020

douardda committed rDSTO87f7bee6935b: journal: add a skipped_content topic dedicated to SkippedContent objects (authored by douardda).
journal: add a skipped_content topic dedicated to SkippedContent objects
May 15 2020, 10:24 AM
douardda closed D3140: journal: add a skipped_content topic dedicated to SkippedContent objects.
May 15 2020, 10:24 AM
douardda updated the diff for D3152: Tag model entities with their "type".

same, but with the actual chuck

May 15 2020, 10:21 AM
douardda updated the diff for D3152: Tag model entities with their "type".

assert object_type is unset is the class has subclasses

May 15 2020, 10:20 AM
douardda updated the diff for D3152: Tag model entities with their "type".

add the test suggested by olasd

May 15 2020, 10:17 AM
douardda added a comment to D3152: Tag model entities with their "type".

Looks good.

Do you plan to add the test olasd mentioned? [1]

We should also have a test checking that no two objects types have the same object_type value. This test should check all BaseModel subclasses (instead of just stuff generated by strategies.objects).

[1] D3152#76613

May 15 2020, 10:09 AM
douardda added a comment to D3152: Tag model entities with their "type".

so be it :-)

May 15 2020, 9:58 AM
douardda updated the diff for D3152: Tag model entities with their "type".

rename tag as object_type and use a Final instead of a Literal.

May 15 2020, 9:57 AM

May 14 2020

douardda added a comment to D3152: Tag model entities with their "type".

And I'm not a big fan of the name. What about object_type, as we already use this name in other places?

May 14 2020, 10:52 AM
douardda added inline comments to D3152: Tag model entities with their "type".
May 14 2020, 10:46 AM

May 13 2020

douardda created D3152: Tag model entities with their "type".
May 13 2020, 5:10 PM
douardda committed rDJNL89cd8f7bea87: Fix test data: only create aware datetime objects (authored by douardda).
Fix test data: only create aware datetime objects
May 13 2020, 4:26 PM
douardda committed rDJNLbad6e3d67d50: Move OBJECT_TYPES global variable to tests/journal_data.py (authored by douardda).
Move OBJECT_TYPES global variable to tests/journal_data.py
May 13 2020, 4:26 PM
douardda closed D3148: Fix test data: only create aware datetime objects.
May 13 2020, 4:26 PM
douardda updated the diff for D3150: Add support for anonymized journal topics.

typos (thx ardumont)

May 13 2020, 4:21 PM
douardda added a comment to D3151: (fix ci) Fix search journal client tests.

Remove global var and put that definition local to its use

May 13 2020, 4:06 PM · Journal, Archive search
douardda accepted D3151: (fix ci) Fix search journal client tests.
May 13 2020, 4:02 PM · Journal, Archive search
douardda added inline comments to D3151: (fix ci) Fix search journal client tests.
May 13 2020, 3:50 PM · Journal, Archive search
douardda accepted D3147: README: Update necessary dependencies for test purposes.
May 13 2020, 3:50 PM · Archive search
douardda added a comment to D3149: Add support for "flags" in object_type used for kafka topics.

What about using a different prefix instead of adding a suffix, eg. swh.journal.anonymized_objects.?

It would require less special handling/configuration

May 13 2020, 3:34 PM
douardda added a comment to D3149: Add support for "flags" in object_type used for kafka topics.

I'm ok, but for one question.

Shouldn't this also accept no : in the object type (the prior behavior if i understood correctly)?

May 13 2020, 3:02 PM
douardda added a comment to D3149: Add support for "flags" in object_type used for kafka topics.

Did we already decide anonymized topics should use this name scheme?

Nope, this is a base for discussion :-)

May 13 2020, 3:00 PM
douardda added a comment to D3149: Add support for "flags" in object_type used for kafka topics.

Did we already decide anonymized topics should use this name scheme?

May 13 2020, 2:54 PM
douardda created D3150: Add support for anonymized journal topics.
May 13 2020, 12:23 PM
douardda updated the diff for D3140: journal: add a skipped_content topic dedicated to SkippedContent objects.

typos (thx ardumont)

May 13 2020, 12:22 PM
douardda created D3149: Add support for "flags" in object_type used for kafka topics.
May 13 2020, 11:40 AM
douardda created D3148: Fix test data: only create aware datetime objects.
May 13 2020, 11:39 AM
douardda committed rDCIDX10251a186621: Fix test_cli for swh.journal 0.1.0 (authored by douardda).
Fix test_cli for swh.journal 0.1.0
May 13 2020, 9:48 AM
douardda closed D3144: Fix test_cli for swh.journal 0.1.0.
May 13 2020, 9:48 AM
douardda updated the diff for D3144: Fix test_cli for swh.journal 0.1.0.

rebase

May 13 2020, 9:46 AM

May 12 2020

douardda created D3144: Fix test_cli for swh.journal 0.1.0.
May 12 2020, 4:27 PM
douardda created D3140: journal: add a skipped_content topic dedicated to SkippedContent objects.
May 12 2020, 9:52 AM
douardda committed rDSTO306aa69b26d0: Add missing return annotations on JournalWriter methods (authored by douardda).
Add missing return annotations on JournalWriter methods
May 12 2020, 9:51 AM
douardda committed rDSTOb04cb8f92c04: Improve a bit the exception message of JournalWriter.content_update (authored by douardda).
Improve a bit the exception message of JournalWriter.content_update
May 12 2020, 9:51 AM
douardda committed rDSTO46f9a7a15113: Refactor the JournalWriter class to normalize its methods (authored by douardda).
Refactor the JournalWriter class to normalize its methods
May 12 2020, 9:51 AM
douardda closed D3138: Refactor the JournalWriter class to normalize its methods.
May 12 2020, 9:51 AM
douardda closed D3137: tests: fix test_replay; do only use aware datetime objects.
May 12 2020, 9:51 AM
douardda committed rDSTOe64944bc53c9: tests: fix test_replay; do only use aware datetime objects (authored by douardda).
tests: fix test_replay; do only use aware datetime objects
May 12 2020, 9:51 AM
douardda added a comment to D3137: tests: fix test_replay; do only use aware datetime objects.

we usually define a now() function at the beginning of modules to return an aware datetime, but that's ok too

May 12 2020, 9:50 AM

May 11 2020

douardda created D3138: Refactor the JournalWriter class to normalize its methods.
May 11 2020, 4:27 PM
douardda created D3137: tests: fix test_replay; do only use aware datetime objects.
May 11 2020, 4:23 PM

May 7 2020

douardda accepted D3134: test_kafka_writer: Add missing object type skipped_content.
May 7 2020, 4:34 PM
douardda committed rDSTO68eb6b3b60a1: Update test_kafka_writer for swh.journal 0.1.0 (authored by douardda).
Update test_kafka_writer for swh.journal 0.1.0
May 7 2020, 3:34 PM
douardda closed D3131: Update test_kafka_writer for swh.journal 0.1.0.
May 7 2020, 3:33 PM
douardda created D3131: Update test_kafka_writer for swh.journal 0.1.0.
May 7 2020, 10:36 AM
douardda committed rDJNL25339889f89e: pytest_plugin: use the object_types fixture in test_config (authored by douardda).
pytest_plugin: use the object_types fixture in test_config
May 7 2020, 9:33 AM
douardda committed rDJNL381ceafe5b2d: pytest_plugin: modify the kafka_server fixture so topics are created beforehand (authored by douardda).
pytest_plugin: modify the kafka_server fixture so topics are created beforehand
May 7 2020, 9:33 AM
douardda committed rDJNL957333752e41: pytest_plugin: remove '.swh.journal.objects' from test_config fixture (authored by douardda).
pytest_plugin: remove '.swh.journal.objects' from test_config fixture
May 7 2020, 9:33 AM
douardda committed rDJNL302050a6254c: test: replace usage of a strategy .example() by a hardcoded value (authored by douardda).
test: replace usage of a strategy .example() by a hardcoded value
May 7 2020, 9:33 AM
douardda committed rDJNL3e8e7dff4ec5: pytest_plugin: do not crash if the list of consumed messages is empty (authored by douardda).
pytest_plugin: do not crash if the list of consumed messages is empty
May 7 2020, 9:33 AM
douardda committed rDJNL84e32e6a90e4: pytest_plugin: improve error message in consume_messages() helper function (authored by douardda).
pytest_plugin: improve error message in consume_messages() helper function
May 7 2020, 9:33 AM
douardda committed rDJNL8ea0de47a79c: client: replace the hardcoded ACCEPTED_OBJECT_TYPES by kafka introspection (authored by douardda).
client: replace the hardcoded ACCEPTED_OBJECT_TYPES by kafka introspection
May 7 2020, 9:33 AM
douardda closed D3130: Refactor the pytest_plugin.
May 7 2020, 9:33 AM
douardda closed D3126: client: replace the hardcoded ACCEPTED_OBJECT_TYPES by kafka introspection.
May 7 2020, 9:33 AM
douardda updated the summary of D3130: Refactor the pytest_plugin.
May 7 2020, 9:31 AM
douardda updated the diff for D3130: Refactor the pytest_plugin.

several typos reported by anlambert

May 7 2020, 9:26 AM
douardda updated the diff for D3126: client: replace the hardcoded ACCEPTED_OBJECT_TYPES by kafka introspection.

typo in commit message

May 7 2020, 9:25 AM
douardda added a comment to D3130: Refactor the pytest_plugin.

Looks good to me. Just one nitpick, there is some typos in commit messages:

  • instanciation -> instantiation
  • instanciating -> instantiating
May 7 2020, 9:18 AM

May 6 2020

douardda created D3130: Refactor the pytest_plugin.
May 6 2020, 5:40 PM
douardda updated the diff for D3126: client: replace the hardcoded ACCEPTED_OBJECT_TYPES by kafka introspection.

fix a few typos

May 6 2020, 5:34 PM

May 5 2020

douardda updated the diff for D3126: client: replace the hardcoded ACCEPTED_OBJECT_TYPES by kafka introspection.

few changes according to comments on IRC

May 5 2020, 5:11 PM
douardda abandoned D3119: Add skipped_content to the list of accepted objects.

Preferred solution: D3126

May 5 2020, 4:40 PM
douardda created D3126: client: replace the hardcoded ACCEPTED_OBJECT_TYPES by kafka introspection.
May 5 2020, 4:39 PM
douardda added a comment to D3122: (fix ci) search.cli: Fix journal client instantiation and add config checks.

why was get_journal_client removed from swh.journal.cli?

All the code you're adding in swh/search/cli.py should be in swh-journal, so it can be used by other CLIs using a journal client

May 5 2020, 2:00 PM · Journal, Archive search
douardda created P663 (An Untitled Masterwork).
May 5 2020, 12:17 PM
douardda accepted D3116: Reimplement the azure objstorage on top of recent azure-storage-blob.

nice work! thx

May 5 2020, 10:02 AM

May 4 2020

douardda added a revision to T2355: Make swh-journal independent from swh-storage or swh-objstorage: D3119: Add skipped_content to the list of accepted objects.
May 4 2020, 5:37 PM · Object storage, Storage manager, Journal
douardda created D3119: Add skipped_content to the list of accepted objects.
May 4 2020, 5:37 PM
douardda accepted D3116: Reimplement the azure objstorage on top of recent azure-storage-blob.

I'd like one or 2 comments (as stated in the comment above), and I don't like the inheritance between PrefixedAzureCloudObjStorage and AzureCloudObjStorage, but that's not the object of this diff.

May 4 2020, 3:26 PM

Apr 30 2020

douardda closed T2355: Make swh-journal independent from swh-storage or swh-objstorage as Resolved.

Let's consider this is done now.

Apr 30 2020, 4:09 PM · Object storage, Storage manager, Journal
douardda accepted D3106: tests: Fix new failing ones after swh-storage update.
Apr 30 2020, 3:56 PM

Apr 29 2020

douardda accepted D3090: Set SWHPKGDIR on the main sphinx job as well.
Apr 29 2020, 4:06 PM
douardda closed D3087: Remove the content replayer code.
Apr 29 2020, 2:56 PM
douardda committed rDJNLcdf9da82a425: Remove the content replayer code (authored by douardda).
Remove the content replayer code
Apr 29 2020, 2:56 PM
douardda committed rDJNL92e0379803d9: Remove the backfiller and the (storage) replayer (authored by douardda).
Remove the backfiller and the (storage) replayer
Apr 29 2020, 2:56 PM
douardda closed D3086: Remove the backfiller and the (storage) replayer.
Apr 29 2020, 2:56 PM