Page MenuHomeSoftware Heritage
Feed Advanced Search

Jun 10 2020

douardda accepted D3253: loader: Migrate to use origin visit status add storage endpoint.

lgtm but see my comments, in case you agree with them.

Jun 10 2020, 2:25 PM
douardda added inline comments to D3253: loader: Migrate to use origin visit status add storage endpoint.
Jun 10 2020, 2:24 PM
douardda added inline comments to D3253: loader: Migrate to use origin visit status add storage endpoint.
Jun 10 2020, 2:22 PM
douardda created D3256: Add hidden Content in tests' journal_data.
Jun 10 2020, 1:38 PM
douardda created D3255: Remove now unused swh/journal/tests/utils.py.
Jun 10 2020, 1:38 PM
douardda committed rDOBJSRPL599d70a487b9: Rewrite test_replay using the kafka_server fixture (authored by douardda).
Rewrite test_replay using the kafka_server fixture
Jun 10 2020, 1:34 PM
douardda closed D3252: Rewrite test_replay using the kafka_server fixture.
Jun 10 2020, 1:34 PM
douardda accepted D3248: Replace swh-worker-control with a swh scheduler celery-monitor subcommand.
Jun 10 2020, 12:19 PM
douardda created D3252: Rewrite test_replay using the kafka_server fixture.
Jun 10 2020, 12:07 PM
douardda added inline comments to D3248: Replace swh-worker-control with a swh scheduler celery-monitor subcommand.
Jun 10 2020, 10:59 AM

Jun 8 2020

douardda committed rDLDSVN2f08b0dfb7d1: test_loader: assert returned loader status is what we expect (authored by douardda).
test_loader: assert returned loader status is what we expect
Jun 8 2020, 9:52 AM
douardda closed D3227: test_loader: assert returned loader status is what we expect.
Jun 8 2020, 9:52 AM

Jun 5 2020

douardda accepted D3237: Fix race condition in multiplexer_objstorage add() return value.
Jun 5 2020, 6:44 PM
douardda committed rDSTO88271f80dc68: Add a replayer test for anonymized journal topics (authored by douardda).
Add a replayer test for anonymized journal topics
Jun 5 2020, 6:27 PM
douardda closed D3232: Add a replayer test for anonymized journal topics.
Jun 5 2020, 6:27 PM
douardda updated the diff for D3227: test_loader: assert returned loader status is what we expect.

fix the failing test

Jun 5 2020, 6:18 PM
douardda updated the diff for D3232: Add a replayer test for anonymized journal topics.

typo

Jun 5 2020, 6:13 PM
douardda committed rDSTOc75da7a90e1d: Small refactoring of the InMemoryStorage to make it more consistent (authored by douardda).
Small refactoring of the InMemoryStorage to make it more consistent
Jun 5 2020, 5:45 PM
douardda closed D3231: Small refactoring of the InMemoryStorage to make it more consistent.
Jun 5 2020, 5:45 PM
douardda accepted D3212: storage*: Add origin_visit_status_add endpoint.
Jun 5 2020, 5:37 PM · Storage manager
douardda created D3232: Add a replayer test for anonymized journal topics.
Jun 5 2020, 12:33 PM
douardda created D3231: Small refactoring of the InMemoryStorage to make it more consistent.
Jun 5 2020, 12:33 PM
douardda committed rDOBJS15358e9c2ac5: Prevent collections.abc related deprecation warning (authored by douardda).
Prevent collections.abc related deprecation warning
Jun 5 2020, 11:29 AM
douardda closed D3228: Prevent collections.abc related deprecation warning.
Jun 5 2020, 11:28 AM
douardda added inline comments to D3212: storage*: Add origin_visit_status_add endpoint.
Jun 5 2020, 11:16 AM · Storage manager
douardda committed rDVAUad66bb1b7b7d: Update tests to swh.model 0.3 (authored by douardda).
Update tests to swh.model 0.3
Jun 5 2020, 10:50 AM
douardda closed D3229: Update tests to swh.model 0.3.
Jun 5 2020, 10:50 AM
douardda created D3229: Update tests to swh.model 0.3.
Jun 5 2020, 10:05 AM

Jun 4 2020

douardda updated the diff for D3228: Prevent collections.abc related deprecation warning.

oops (thx anlambert)

Jun 4 2020, 5:23 PM
douardda created D3228: Prevent collections.abc related deprecation warning.
Jun 4 2020, 4:45 PM
douardda committed rDLDBASEc28799f674e3: Fix a potential UnboundLocalError in clean_dangling_folders() (authored by douardda).
Fix a potential UnboundLocalError in clean_dangling_folders()
Jun 4 2020, 4:34 PM
douardda closed D3225: Fix a potential UnboundLocalError in clean_dangling_folders().
Jun 4 2020, 4:34 PM
douardda created D3227: test_loader: assert returned loader status is what we expect.
Jun 4 2020, 4:33 PM
douardda committed rDLDSVN20e7618fed9b: Adapt to swh.model 0.3 (authored by douardda).
Adapt to swh.model 0.3
Jun 4 2020, 4:32 PM
douardda closed D3222: Adapt to swh.model 0.3.
Jun 4 2020, 4:32 PM
douardda accepted D3226: Use explicit configuration (without journal writer) for algos tests.
Jun 4 2020, 4:28 PM
douardda updated the diff for D3222: Adapt to swh.model 0.3.

more fix

Jun 4 2020, 4:21 PM
douardda created D3225: Fix a potential UnboundLocalError in clean_dangling_folders().
Jun 4 2020, 4:00 PM
douardda updated the diff for D3222: Adapt to swh.model 0.3.

missed lists

Jun 4 2020, 2:59 PM
douardda created D3222: Adapt to swh.model 0.3.
Jun 4 2020, 2:46 PM
douardda committed rDSTO48953aa2a6c6: d/changelog: fix the release (authored by douardda).
d/changelog: fix the release
Jun 4 2020, 2:39 PM
douardda committed rDSTO3d665d67ba16: d/changelog: version 0.1.0-2~swh1 (authored by douardda).
d/changelog: version 0.1.0-2~swh1
Jun 4 2020, 1:45 PM
douardda committed rDSTO54dadc4a5563: d/control: update dependencies (authored by douardda).
d/control: update dependencies
Jun 4 2020, 1:13 PM
douardda committed rDCIDX63600f892af8: Fix test data for swh.model 0.3 (authored by douardda).
Fix test data for swh.model 0.3
Jun 4 2020, 12:51 PM
douardda closed D3220: Fix test data for swh.model 0.3.
Jun 4 2020, 12:51 PM
douardda created D3220: Fix test data for swh.model 0.3.
Jun 4 2020, 12:25 PM
douardda committed rDSTOf9b2ca3fec14: Replace MockedJournalClient and MockedKafkaWriter by proper kafka test… (authored by douardda).
Replace MockedJournalClient and MockedKafkaWriter by proper kafka test…
Jun 4 2020, 12:03 PM
douardda closed D3203: Replace MockedJournalClient and MockedKafkaWriter by proper kafka test scaffolding.
Jun 4 2020, 12:03 PM
douardda committed rDSTOad9c9bbfd0d5: Adapt to swh.model 0.3 (authored by douardda).
Adapt to swh.model 0.3
Jun 4 2020, 12:03 PM
douardda closed D3218: Adapt to swh.model 0.3.
Jun 4 2020, 12:03 PM
douardda updated the diff for D3203: Replace MockedJournalClient and MockedKafkaWriter by proper kafka test scaffolding.

rebase

Jun 4 2020, 11:51 AM
douardda updated the diff for D3218: Adapt to swh.model 0.3.

thx olasd

Jun 4 2020, 11:48 AM
douardda closed D3177: Use Tuple instead of List in model declarations.

closed by 3d9f69444aeba903f701486d12978ea887a0a8f8

Jun 4 2020, 11:46 AM
douardda updated the diff for D3218: Adapt to swh.model 0.3.

another fix for cassandra

Jun 4 2020, 11:35 AM
douardda updated the diff for D3203: Replace MockedJournalClient and MockedKafkaWriter by proper kafka test scaffolding.

rebased on D3218

Jun 4 2020, 11:27 AM
douardda updated the diff for D3218: Adapt to swh.model 0.3.

adapt according to ardumont's comments and (maybe) fix cassandra tests

Jun 4 2020, 11:20 AM
douardda added a comment to D3218: Adapt to swh.model 0.3.

My understanding of the current log failure is that the cassandra returns you
"parents" as a list instead of tuple. This then is captured by the validation
proxy.

probably, the pb of running tests with 'not cassandra'...

yeah ;)

Jsyk, in D3211 (which was missing the rest of your current diff), I took the
party of keeping the list type in the conversion function (you changed it to
Tuple here) cassandra.converters.revision_from_db and force the parents to be
a tuple within that function. This, to avoid having to change the
cassandra.storage implementations. As you took the other path, you need to
check the revision_get and other revision functions which builds list for that field ;)

Jun 4 2020, 11:09 AM
douardda added a comment to D3218: Adapt to swh.model 0.3.

Build has FAILED

My understanding of the current log failure is that the cassandra returns you
"parents" as a list instead of tuple. This then is captured by the validation
proxy.

Jun 4 2020, 11:04 AM
douardda updated the diff for D3218: Adapt to swh.model 0.3.

add a word in the commit message about test_api_client_dicts.py removal

Jun 4 2020, 10:53 AM
douardda created D3218: Adapt to swh.model 0.3.
Jun 4 2020, 10:49 AM

Jun 3 2020

douardda committed rDJNL64cd01fc7c04: Improve tests coverage by adding missing model types in journal_data (authored by douardda).
Improve tests coverage by adding missing model types in journal_data
Jun 3 2020, 1:52 PM
douardda committed rDJNL5c27d5161eb9: Adapt test data to swh.model 0.3 (authored by douardda).
Adapt test data to swh.model 0.3
Jun 3 2020, 1:52 PM
douardda closed D3207: Improve tests coverage by adding missing model types in journal_data.
Jun 3 2020, 1:52 PM
douardda updated the diff for D3207: Improve tests coverage by adding missing model types in journal_data.

Adapt the code to swh.model 0.3

Jun 3 2020, 1:49 PM
douardda committed rDMOD340656dd3be8: Fix origin_visit hypothesis strategies (authored by douardda).
Fix origin_visit hypothesis strategies
Jun 3 2020, 11:30 AM

Jun 2 2020

douardda committed rDJNL9c87cda72990: Fix MockedKafkaWriter (authored by douardda).
Fix MockedKafkaWriter
Jun 2 2020, 6:00 PM
douardda created D3207: Improve tests coverage by adding missing model types in journal_data.
Jun 2 2020, 3:24 PM
douardda retitled D3177: Use Tuple instead of List in model declarations from Make all model objects hashable to Use Tuple instead of List in model declarations.
Jun 2 2020, 2:37 PM
douardda accepted D3206: from_disk: Fix ImportError with dulwich >= 0.20.

ok but why move the import statement in the method?

Jun 2 2020, 2:36 PM
douardda committed rDSTOeef4900db52a: Fix InMemoryStorage.origin_visit_upsert() method (authored by douardda).
Fix InMemoryStorage.origin_visit_upsert() method
Jun 2 2020, 2:31 PM
douardda closed D3202: Fix InMemoryStorage.origin_visit_upsert() method.
Jun 2 2020, 2:31 PM
douardda updated the diff for D3177: Use Tuple instead of List in model declarations.

rebase + rephrase the commit message

Jun 2 2020, 2:19 PM
douardda added inline comments to D3203: Replace MockedJournalClient and MockedKafkaWriter by proper kafka test scaffolding.
Jun 2 2020, 2:14 PM
douardda added a comment to D3203: Replace MockedJournalClient and MockedKafkaWriter by proper kafka test scaffolding.

This also kills test_write_replay.py file since it does not test anything
more than what is currently tested in test_replay.py.

Are you sure? replay.py seems to have only fixed examples, while test_write_replay.py uses anything hypothesis generates

Jun 2 2020, 2:12 PM

May 29 2020

douardda committed rDMODa95646fc8786: Exclude [Skipped]Content.ctime from hash/eq computation (authored by douardda).
Exclude [Skipped]Content.ctime from hash/eq computation
May 29 2020, 5:40 PM
douardda closed D3205: Exclude [Skipped]Content.ctime from hash/eq computation.
May 29 2020, 5:40 PM
douardda created D3205: Exclude [Skipped]Content.ctime from hash/eq computation.
May 29 2020, 5:18 PM
douardda created D3203: Replace MockedJournalClient and MockedKafkaWriter by proper kafka test scaffolding.
May 29 2020, 3:59 PM
douardda created D3202: Fix InMemoryStorage.origin_visit_upsert() method.
May 29 2020, 3:59 PM

May 27 2020

douardda added a comment to D3179: swh-storage: README: Update necessary dependencies for test purposes.

for other systems (non debian) looks a bit weird to specify he expected location for cassandra but not for postgresql.

May 27 2020, 1:58 PM
douardda added a comment to D3179: swh-storage: README: Update necessary dependencies for test purposes.

I don't recall the flag needed to exclude the cassandra tests from the tox or pytest run.
That'd be great to add it in there somewhere i guess.

May 27 2020, 1:53 PM
douardda added a comment to D3177: Use Tuple instead of List in model declarations.

The commit message should at least be amended, since it does not make all model objects immutable, but only fix a couple of model classes.

May 27 2020, 10:13 AM
douardda added a comment to T2421: Make model objects immutable.

@olasd any reason not to do the step 1. ? Wether or not we "fix" the remaining hashability issue by overwriting the __hash__ method, I see no harm and it makes the model (IMHO) cleaner.

May 27 2020, 10:09 AM · Data Model
douardda added a comment to T2421: Make model objects immutable.

Conceptually, both Snapshot.branches and Directory.entries are mappings, not arrays of objects: snapshots and directories must not contain entries with repeated names (which is something that's currently enforced for snapshots, but not for directories).

May 27 2020, 9:38 AM · Data Model

May 26 2020

douardda triaged T2423: Extract the `extra_headers` away from `Revision.metadata` into a top-level immutable object as Normal priority.
May 26 2020, 5:07 PM · Data Model
douardda added a task to D3177: Use Tuple instead of List in model declarations: T2421: Make model objects immutable.
May 26 2020, 5:04 PM
douardda added a revision to T2421: Make model objects immutable: D3177: Use Tuple instead of List in model declarations.
May 26 2020, 5:04 PM · Data Model
douardda added a revision to T2422: Add an `object_type` attribute to model classes: D3152: Tag model entities with their "type".
May 26 2020, 5:03 PM · Data Model
douardda added a task to D3152: Tag model entities with their "type": T2422: Add an `object_type` attribute to model classes.
May 26 2020, 5:03 PM
douardda triaged T2422: Add an `object_type` attribute to model classes as Normal priority.
May 26 2020, 5:03 PM · Data Model
douardda triaged T2421: Make model objects immutable as Normal priority.
May 26 2020, 4:52 PM · Data Model

May 25 2020

douardda added a comment to D3177: Use Tuple instead of List in model declarations.

We wouldn't need that if pytest had an equivalent for assertCountEqual :/

May 25 2020, 3:18 PM
douardda updated the diff for D3177: Use Tuple instead of List in model declarations.

improve the commit message

May 25 2020, 2:54 PM
douardda added a comment to D3177: Use Tuple instead of List in model declarations.

What is the motivation for this?

Be able to create sets of revisions.

Also, having frozen objects that are modifiable seems a bit fuzzy to me, to say the least.

May 25 2020, 2:48 PM
douardda added a comment to D3177: Use Tuple instead of List in model declarations.

What is the motivation for this?

Be able to create sets of revisions.

May 25 2020, 2:47 PM
douardda added a comment to D3177: Use Tuple instead of List in model declarations.

What is the motivation for this?

May 25 2020, 2:46 PM
douardda added inline comments to D2949: nixguix: catch and log artifact resolution failures.
May 25 2020, 2:30 PM · Package Loader
douardda updated the diff for D3177: Use Tuple instead of List in model declarations.

Forgot to update hypothesis strategies also.

May 25 2020, 2:14 PM
douardda updated the diff for D3177: Use Tuple instead of List in model declarations.

Bump dep on attrs_strict >= 0.0.7

May 25 2020, 1:58 PM
douardda created D3177: Use Tuple instead of List in model declarations.
May 25 2020, 1:44 PM