Page MenuHomeSoftware Heritage
Feed Advanced Search

Jul 30 2020

douardda added inline comments to D3641: storage*: add origin_visit_status_get(...) -> PagedResult[OriginVisitStatus].
Jul 30 2020, 11:36 AM
douardda added inline comments to D3641: storage*: add origin_visit_status_get(...) -> PagedResult[OriginVisitStatus].
Jul 30 2020, 11:34 AM
douardda added inline comments to D3641: storage*: add origin_visit_status_get(...) -> PagedResult[OriginVisitStatus].
Jul 30 2020, 11:32 AM
douardda added inline comments to D3641: storage*: add origin_visit_status_get(...) -> PagedResult[OriginVisitStatus].
Jul 30 2020, 11:30 AM
douardda added inline comments to D3641: storage*: add origin_visit_status_get(...) -> PagedResult[OriginVisitStatus].
Jul 30 2020, 11:23 AM
douardda added inline comments to D3527: Reimplement the GitHub lister using the new pattern class.
Jul 30 2020, 11:10 AM
douardda added inline comments to D3527: Reimplement the GitHub lister using the new pattern class.
Jul 30 2020, 10:42 AM

Jul 29 2020

douardda accepted D3600: Add visit/edges endpoint.
Jul 29 2020, 5:06 PM
douardda requested changes to D3435: Add mercurial.from_disk.HgLoaderFromDisk.

Hi, more comments/requests/grunts :-)

Jul 29 2020, 5:02 PM
douardda committed rDMODb58d901b3b22: Fix incorrectly typed null constants in extra_headers byte strings (authored by douardda).
Fix incorrectly typed null constants in extra_headers byte strings
Jul 29 2020, 2:56 PM
douardda committed rDMOD8f609e510d92: Import Mapping from collections.abc instead of collections (authored by douardda).
Import Mapping from collections.abc instead of collections
Jul 29 2020, 2:56 PM
douardda committed rDMOD81f9fbc57417: Declare pytest markers (authored by douardda).
Declare pytest markers
Jul 29 2020, 2:56 PM
douardda closed D3639: Series of small fixes.
Jul 29 2020, 2:56 PM
douardda updated the diff for D3639: Series of small fixes.

fix commit message

Jul 29 2020, 12:47 PM
douardda created D3639: Series of small fixes.
Jul 29 2020, 12:46 PM

Jul 27 2020

douardda accepted D3604: server: rewrite using class-based views.

besides a missing why paragraph in the commit message, lgtm

Jul 27 2020, 5:23 PM
douardda added a comment to D3604: server: rewrite using class-based views.

could you explain why you do this in your commit message? I have no opinion on whether one version is objectively better than the other, but you may have a reason for doing it :-)

Jul 27 2020, 5:13 PM
douardda requested changes to D3600: Add visit/edges endpoint.

looks overall good to me (see my comments), not sure I'm 100% confident there are enough corner case tests (for complex multi diamond shaped graphs), but I let zack judge this.

Jul 27 2020, 5:07 PM

Jul 17 2020

douardda accepted D3554: test_cassandra: Use data model object.

not sure I like too much the

cont, cont2 = [
          attr.evolve(c, ctime=now()) for c in sample_data_model["content"][:2]
      ]
Jul 17 2020, 5:04 PM
douardda accepted D3553: storage.test_db: Drop redundant test.
Jul 17 2020, 3:33 PM
douardda accepted D3552: test_init: Decrease assertion checks.

a bit sad that the real reason there are different results in debian, but well, does not worth the time either, so ok

Jul 17 2020, 3:00 PM
douardda accepted D3551: tests: Reuse pytest fixtures from swh.loader.core.
Jul 17 2020, 1:23 PM
douardda accepted D3548: test_nixguix: Simplify the nixguix specific check_snapshot function.

\o/

Jul 17 2020, 12:56 PM
douardda added a comment to D3548: test_nixguix: Simplify the nixguix specific check_snapshot function.

<mode=nitpciky>should be a comment, not a docstring </mode>

lol, that's cool, better agree to avoid modifying stuff on/off all the time heh.

I made it a docstring so it serves the purpose to also explain the allow_empty
parameter with such value (just above the docstring ;)

So with that in mind, is that ok as-is now?

Jul 17 2020, 12:43 PM
douardda accepted D3550: tests: Reuse pytest fixtures from swh.loader.core.
Jul 17 2020, 12:06 PM
douardda accepted D3549: tests: Reuse pytest fixtures from swh.loader.core.
Jul 17 2020, 12:05 PM
douardda added a comment to D3548: test_nixguix: Simplify the nixguix specific check_snapshot function.

<mode=nitpciky>should be a comment, not a docstring </mode>

Jul 17 2020, 12:04 PM
douardda added a comment to D3520: Add requirements-dev.txt.

I'm not very fond of yet another requirement file.

Jul 17 2020, 11:54 AM
douardda added a comment to D3519: Add isort configuration.

We do not use isort for now, so I'm a bit puzzled by the "The default isort configuration did not match the actual sorting of the
imports. " statement above.

Jul 17 2020, 11:47 AM
douardda added a comment to D3435: Add mercurial.from_disk.HgLoaderFromDisk.

following @ardumont comment, the first 2 revisions are actually unrelated with this diff, so please submit them independently.
(however they can both be submitted as a single diff, but I'm not sure I'll buy the requirement-dev one, but let's discuss this on the proper diff)

@douardda he did already opened them, that's respectively D3519 D3520 ;)

Jul 17 2020, 11:41 AM
douardda added a comment to D3435: Add mercurial.from_disk.HgLoaderFromDisk.

one question (not checked nor reread the diff, just wondering): does this version of the loader generates the same SWHID for mercurial revisions than the one currently running? I can't see a test that prove that is true, but I may have missed it.

Jul 17 2020, 11:40 AM
douardda added a comment to D3435: Add mercurial.from_disk.HgLoaderFromDisk.

following @ardumont comment, the first 2 revisions are actually unrelated with this diff, so please submit them independently.
(however they can both be submitted as a single diff, but I'm not sure I'll buy the requirement-dev one, but let's discuss this on the proper diff)

Jul 17 2020, 11:37 AM
douardda accepted D3547: tests: Check against snapshot model object.
Jul 17 2020, 11:17 AM
douardda accepted D3546: tests: Check against snapshot model object.
Jul 17 2020, 11:15 AM
douardda accepted D3545: svn.loader: Drop `build_swh_snapshot` function, oneliner used once.

but otherwise it's ok

Jul 17 2020, 11:13 AM
douardda added a comment to D3545: svn.loader: Drop `build_swh_snapshot` function, oneliner used once.

ok but the commit message should just say something like "kill build_swh_snapshot() function" + one or 2 sentences explaining it's unneeded bc it's a oneliner used only once.

Jul 17 2020, 11:13 AM
douardda accepted D3544: tests: Check against snapshot model object.
Jul 17 2020, 11:10 AM
douardda accepted D3543: check_snapshot: Drop accepting using dict for snapshot comparison.
Jul 17 2020, 11:09 AM
douardda accepted D3542: test_pypi: Check against snapshot model object.
Jul 17 2020, 11:08 AM
douardda accepted D3541: test_npm: Check against snapshot model object.
Jul 17 2020, 11:07 AM
douardda accepted D3540: test_nixguix: Merge check_snapshot* methods into 1.
Jul 17 2020, 11:04 AM
douardda accepted D3539: test_nixguix: Check against snapshot model object.

lgtm

Jul 17 2020, 11:00 AM

Jul 16 2020

douardda closed D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().

closed by 28b35a18c723ba3d11ccf3beaf2008638aefb362

Jul 16 2020, 11:04 AM
douardda committed rDLDBASE28b35a18c723: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot() (authored by douardda).
Use bytes as snapshot id and get rid of dict manipulation in check_snapshot()
Jul 16 2020, 10:59 AM
douardda accepted D3515: pytest_plugin: Avoid fixture client to declare optional dependency.

ok, but should check whether some other packages that uses this pytest plugin can be impacted.

Jul 16 2020, 10:44 AM
douardda committed rDLDHG786f8caa37af: Fix test_loader's "id" types for proper bytes (authored by douardda).
Fix test_loader's "id" types for proper bytes
Jul 16 2020, 10:39 AM
douardda closed D3516: Fix test_loader's "id" types for proper bytes.
Jul 16 2020, 10:39 AM
douardda committed rDLDGa6dd6356e412: Fix test_from_disk's snapshot "id" type for bytes (authored by douardda).
Fix test_from_disk's snapshot "id" type for bytes
Jul 16 2020, 10:38 AM
douardda closed D3517: Fix test_from_disk's snapshot "id" type for bytes.
Jul 16 2020, 10:38 AM
douardda updated the diff for D3516: Fix test_loader's "id" types for proper bytes.

inline expected_snapshot_id

Jul 16 2020, 10:36 AM
douardda updated the diff for D3517: Fix test_from_disk's snapshot "id" type for bytes.

inline SNAPSHOT_ID

Jul 16 2020, 10:32 AM
douardda added a comment to D3516: Fix test_loader's "id" types for proper bytes.

I'm not sure we still need the hex form now.

ok nonetheless ;)

Jul 16 2020, 10:01 AM
douardda added a comment to D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().

Otherwise, i think it's fine ;)
And according to the plan, the next step is to land it.

Jul 16 2020, 9:57 AM
douardda updated the summary of D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().
Jul 16 2020, 9:56 AM
douardda updated the test plan for D3516: Fix test_loader's "id" types for proper bytes.
Jul 16 2020, 9:55 AM
douardda created D3517: Fix test_from_disk's snapshot "id" type for bytes.
Jul 16 2020, 9:55 AM
douardda created D3516: Fix test_loader's "id" types for proper bytes.
Jul 16 2020, 9:51 AM

Jul 15 2020

douardda committed rDLDSVN9d16dc0f2283: Fix branches types in tests (authored by douardda).
Fix branches types in tests
Jul 15 2020, 4:04 PM
douardda closed D3498: Fix branches types in tests.
Jul 15 2020, 4:04 PM
douardda committed rDLDHGeb613d754dab: Fix branches types in tests (authored by douardda).
Fix branches types in tests
Jul 15 2020, 4:03 PM
douardda closed D3499: Fix branches types in tests.
Jul 15 2020, 4:03 PM
douardda closed D3496: Fix branches types in tests.
Jul 15 2020, 4:02 PM
douardda committed rDLDGc895b468ca2e: Fix branches types in tests (authored by douardda).
Fix branches types in tests
Jul 15 2020, 4:02 PM
douardda updated the diff for D3498: Fix branches types in tests.

same with actual commit included

Jul 15 2020, 4:01 PM
douardda updated the diff for D3499: Fix branches types in tests.

bump dep on swh.loader.core

Jul 15 2020, 3:59 PM
douardda updated the diff for D3498: Fix branches types in tests.

Bump dep on swh.loader.core

Jul 15 2020, 3:58 PM
douardda updated the diff for D3496: Fix branches types in tests.

bump dep on swh-loader-core

Jul 15 2020, 3:57 PM
douardda added a comment to D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().

I'm very fine with landing a reasonably consistent and stable baseline any time you want.

In that case, i think a plan matching those properties could be:

  • review/land first D3503 (so the check-snapshot implementation is complete, it checks up to the contents). It's not per say a blocker but that'd be more consistent.
  • Then release new loader-core.
  • Trigger back the builds of your mutiple diffs on the dvcs loaders (and land them when green, they should be ;)
  • Then getting back here, rebase on latest loader-core and land this one.
  • And then we (either you or me) should pick up those tests and replace the snapshots as dict with snapshot model objects (both core and again dvcs). <- that was my initial plan
  • Optionally (or not), then we can come back here again and drop the from_dict conversion from the input... (only model object!!! at least in tests)

Does that sound reasonable?

Jul 15 2020, 12:14 PM
douardda accepted D3503: check_snapshot: Check existence down to contents.

looks ok, but I find it temptating to factorize(!) the missing objects looking pattern (not asked right now!).

Jul 15 2020, 12:12 PM
douardda updated the diff for D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().

rebase

Jul 15 2020, 10:56 AM
douardda accepted D3502: check_snapshot: Check existence down to the first level directories.
Jul 15 2020, 10:51 AM
douardda updated the diff for D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().

no noqa

Jul 15 2020, 10:44 AM
douardda added inline comments to D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().
Jul 15 2020, 10:43 AM
douardda added a comment to D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().

fwiw, i was heading towards making the transition you started *after* having deployed

the current implementation of "check-snapshot" (without that diff so with the internal
manipulation) And then commit the current diff ;)

Do you still want to land it first?

Jul 15 2020, 10:42 AM

Jul 10 2020

douardda accepted D3495: Expose a pytest_plugin module so other loaders can reuse for tests.
Jul 10 2020, 4:53 PM
douardda accepted D3493: pytest: Remove no longer needed pytest setup.
Jul 10 2020, 4:52 PM
douardda updated the summary of D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().
Jul 10 2020, 4:50 PM
douardda created D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().
Jul 10 2020, 4:50 PM
douardda committed rDLDBASEfcc7e61bf9c9: Fix branches types in tests (authored by douardda).
Fix branches types in tests
Jul 10 2020, 4:47 PM
douardda committed rDLDBASEc05adcaf5c28: Small code improvement in package/loader.py (authored by douardda).
Small code improvement in package/loader.py
Jul 10 2020, 4:47 PM
douardda closed D3494: Fix branches types in tests.
Jul 10 2020, 4:47 PM
douardda accepted D3497: Explicit types in __init__ files.
Jul 10 2020, 4:45 PM
douardda updated the diff for D3494: Fix branches types in tests.

rebase

Jul 10 2020, 4:43 PM
douardda updated the diff for D3494: Fix branches types in tests.

Forgot one occurrence

Jul 10 2020, 4:26 PM
douardda created D3499: Fix branches types in tests.
Jul 10 2020, 4:04 PM
douardda created D3498: Fix branches types in tests.
Jul 10 2020, 3:58 PM
douardda created D3496: Fix branches types in tests.
Jul 10 2020, 3:50 PM
douardda created D3494: Fix branches types in tests.
Jul 10 2020, 3:31 PM
douardda accepted D3492: setup: Do no expose the pytest-plugin, let modules require it.
Jul 10 2020, 12:57 PM
douardda accepted D3473: loader.core: check_snapshot: Allow to check using both Snapshot and dict objects.
Jul 10 2020, 12:44 PM
douardda added a comment to T2421: Make model objects immutable.

Reading the code dealing with snapshot branches in several storage implementations, it really seems to me that storing them as a dict-like structure has no advantage.

Jul 10 2020, 12:17 PM · Data Model
douardda added a comment to T2421: Make model objects immutable.

But I'd like to use the opportunity of this cleanup to go a bit further than "the minimal amount of work for pedantic correctness", and actually make changes that have a conceptual meaning.

Jul 10 2020, 12:13 PM · Data Model
douardda added inline comments to D3456: Make metadata-related endpoints consistent with other endpoints by using Iterables of swh-model objects instead of a dict..
Jul 10 2020, 11:17 AM
douardda requested changes to D3473: loader.core: check_snapshot: Allow to check using both Snapshot and dict objects.
Jul 10 2020, 10:19 AM
douardda abandoned D762: tox: add a flake8 environment that generates Phabricator-friendly reports.
Jul 10 2020, 10:07 AM

Jul 9 2020

douardda committed rDSTOc3803ef8f797: Fix a typo I introduced in previous revision (authored by douardda).
Fix a typo I introduced in previous revision
Jul 9 2020, 12:47 PM
douardda closed D3477: Fix a typo in previous revision.
Jul 9 2020, 12:47 PM
douardda updated the diff for D3477: Fix a typo in previous revision.

make sure there is no doubt on who the dumb fellow is

Jul 9 2020, 11:58 AM
douardda created D3477: Fix a typo in previous revision.
Jul 9 2020, 11:56 AM
douardda committed rDSTO8bf3794874fa: Convert ImmutableDict to dict before passing it to json.dumps. (authored by vlorentz).
Convert ImmutableDict to dict before passing it to json.dumps.
Jul 9 2020, 11:44 AM