Page MenuHomeSoftware Heritage

douardda (David Douard)
User

Projects (7)

User Details

User Since
Jul 10 2018, 12:38 PM (104 w, 3 d)

Recent Activity

Yesterday

douardda accepted D3495: Expose a pytest_plugin module so other loaders can reuse for tests.
Fri, Jul 10, 4:53 PM
douardda accepted D3493: pytest: Remove no longer needed pytest setup.
Fri, Jul 10, 4:52 PM
douardda updated the summary of D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().
Fri, Jul 10, 4:50 PM
douardda created D3501: Use bytes as snapshot id and get rid of dict manipulation in check_snapshot().
Fri, Jul 10, 4:50 PM
douardda committed rDLDBASEfcc7e61bf9c9: Fix branches types in tests (authored by douardda).
Fix branches types in tests
Fri, Jul 10, 4:47 PM
douardda committed rDLDBASEc05adcaf5c28: Small code improvement in package/loader.py (authored by douardda).
Small code improvement in package/loader.py
Fri, Jul 10, 4:47 PM
douardda closed D3494: Fix branches types in tests.
Fri, Jul 10, 4:47 PM
douardda accepted D3497: Explicit types in __init__ files.
Fri, Jul 10, 4:45 PM
douardda updated the diff for D3494: Fix branches types in tests.

rebase

Fri, Jul 10, 4:43 PM
douardda updated the diff for D3494: Fix branches types in tests.

Forgot one occurrence

Fri, Jul 10, 4:26 PM
douardda created D3499: Fix branches types in tests.
Fri, Jul 10, 4:04 PM
douardda created D3498: Fix branches types in tests.
Fri, Jul 10, 3:58 PM
douardda created D3496: Fix branches types in tests.
Fri, Jul 10, 3:50 PM
douardda created D3494: Fix branches types in tests.
Fri, Jul 10, 3:31 PM
douardda accepted D3492: setup: Do no expose the pytest-plugin, let modules require it.
Fri, Jul 10, 12:57 PM
douardda accepted D3473: loader.core: check_snapshot: Allow to check using both Snapshot and dict objects.
Fri, Jul 10, 12:44 PM
douardda added a comment to T2421: Make model objects immutable.

Reading the code dealing with snapshot branches in several storage implementations, it really seems to me that storing them as a dict-like structure has no advantage.

Fri, Jul 10, 12:17 PM · Data Model
douardda added a comment to T2421: Make model objects immutable.

But I'd like to use the opportunity of this cleanup to go a bit further than "the minimal amount of work for pedantic correctness", and actually make changes that have a conceptual meaning.

Fri, Jul 10, 12:13 PM · Data Model
douardda added inline comments to D3456: Make metadata-related endpoints consistent with other endpoints by using Iterables of swh-model objects instead of a dict..
Fri, Jul 10, 11:17 AM
douardda requested changes to D3473: loader.core: check_snapshot: Allow to check using both Snapshot and dict objects.
Fri, Jul 10, 10:19 AM
douardda abandoned D762: tox: add a flake8 environment that generates Phabricator-friendly reports.
Fri, Jul 10, 10:07 AM

Thu, Jul 9

douardda committed rDSTOc3803ef8f797: Fix a typo I introduced in previous revision (authored by douardda).
Fix a typo I introduced in previous revision
Thu, Jul 9, 12:47 PM
douardda closed D3477: Fix a typo in previous revision.
Thu, Jul 9, 12:47 PM
douardda updated the diff for D3477: Fix a typo in previous revision.

make sure there is no doubt on who the dumb fellow is

Thu, Jul 9, 11:58 AM
douardda created D3477: Fix a typo in previous revision.
Thu, Jul 9, 11:56 AM
douardda committed rDSTO8bf3794874fa: Convert ImmutableDict to dict before passing it to json.dumps. (authored by vlorentz).
Convert ImmutableDict to dict before passing it to json.dumps.
Thu, Jul 9, 11:44 AM
douardda closed D3476: Convert ImmutableDict to dict before passing it to json.dumps..
Thu, Jul 9, 11:44 AM
douardda added inline comments to D3476: Convert ImmutableDict to dict before passing it to json.dumps..
Thu, Jul 9, 11:42 AM
douardda updated the diff for D3476: Convert ImmutableDict to dict before passing it to json.dumps..

simplify the diff by killing map_optional()

Thu, Jul 9, 11:33 AM
douardda commandeered D3476: Convert ImmutableDict to dict before passing it to json.dumps..
Thu, Jul 9, 11:09 AM
douardda added a comment to D3476: Convert ImmutableDict to dict before passing it to json.dumps..

Not sure I buy the map_optional here. When I read a line of code with [x if x else f(x)] I understand it right away. With map_optional, I need to go get the definition of that oneliner function.
Plus you have the type annotation headache...

Thu, Jul 9, 10:48 AM
douardda added a comment to D3475: setup: Do no expose the pytest-plugin any longer.

Adapt according to remarks:

  • Update docstring
Thu, Jul 9, 10:43 AM

Wed, Jul 8

douardda accepted D3469: Move all celery-related fixtures to the swh.scheduler pytest plugin.

looks fine, as far as I can tell

Wed, Jul 8, 6:15 PM
douardda committed rDLDSVNfe5e4d342f30: Update the svn loader to new Revision API (authored by douardda).
Update the svn loader to new Revision API
Wed, Jul 8, 5:11 PM
douardda closed D3464: Update the svn loader to new Revision API.
Wed, Jul 8, 5:11 PM
douardda updated the diff for D3464: Update the svn loader to new Revision API.

typo (thx ardumont)

Wed, Jul 8, 4:58 PM
douardda committed rDLDG8694ec70cda7: Small code simplification (authored by douardda).
Small code simplification
Wed, Jul 8, 4:13 PM
douardda created D3464: Update the svn loader to new Revision API.
Wed, Jul 8, 4:07 PM
douardda accepted D3463: loader: Adapt to swh-model >= 0.4.0.

lgtm

Wed, Jul 8, 3:43 PM
douardda committed rDLDG0394f0f85edf: Adapt the git loader to swh-model >0.4 (authored by douardda).
Adapt the git loader to swh-model >0.4
Wed, Jul 8, 3:13 PM
douardda closed D3454: Adapt the git loader to swh-model >0.4.
Wed, Jul 8, 3:13 PM
douardda updated the diff for D3454: Adapt the git loader to swh-model >0.4.

small simplification as suggested by anlambert

Wed, Jul 8, 3:11 PM
douardda updated the diff for D3454: Adapt the git loader to swh-model >0.4.

rebase

Wed, Jul 8, 2:12 PM
douardda updated the diff for D3454: Adapt the git loader to swh-model >0.4.

fix MANIFEST.in to include the git bundle file

Wed, Jul 8, 2:00 PM
douardda updated the diff for D3454: Adapt the git loader to swh-model >0.4.

bump dependencies

Wed, Jul 8, 12:25 PM
douardda committed rCDFJd502f6eb5a6d: Merge both "pip install" RUN commands in tox/Dockerfile (authored by douardda).
Merge both "pip install" RUN commands in tox/Dockerfile
Wed, Jul 8, 11:56 AM
douardda closed D3457: Merge both "pip install" RUN commands in tox/Dockerfile.
Wed, Jul 8, 11:56 AM
douardda created D3457: Merge both "pip install" RUN commands in tox/Dockerfile.
Wed, Jul 8, 10:18 AM
douardda accepted D3450: Use attr instead of NamedTuple to generate SWHID..

Thx, looks good. Just a small question about the modification of the swhid() API.

Wed, Jul 8, 9:32 AM
douardda added a comment to D3454: Adapt the git loader to swh-model >0.4.
In D3454#84921, @olasd wrote:

Oh, that needs a requirements-swh.txt bump as well, I guess.

Wed, Jul 8, 9:19 AM

Tue, Jul 7

douardda created D3454: Adapt the git loader to swh-model >0.4.
Tue, Jul 7, 7:03 PM
douardda added a comment to P718 (An Untitled Masterwork).
object 9768d0b576dbaaecd80abedad6dfd0d72f1476da
type commit
tag v0.0.1
tagger David Douard <david.douard@sdfa3.org> 1594138133 +0200
Tue, Jul 7, 6:42 PM
douardda added a comment to P718 (An Untitled Masterwork).

git cat-file -p 9768d0b576dbaaecd80abedad6dfd0d72f1476da
tree f0695c2e2fa7ce9d574023c3413761a473e500ca
parent c3c588713233609f5bbbb2d9e7f3fb4a660f3f72
author Stefano Zacchiroli <zack@upsilon.cc> 1443083765 +0200
committer Stefano Zacchiroli <zack@upsilon.cc> 1443083765 +0200

Tue, Jul 7, 6:41 PM
douardda created P718 (An Untitled Masterwork).
Tue, Jul 7, 6:39 PM
douardda committed rDSTO5ab70237d6c3: Extract revision's extra_header as a top level attribute (authored by douardda).
Extract revision's extra_header as a top level attribute
Tue, Jul 7, 5:18 PM
douardda closed D3426: Extract revision's extra_header as a top level attribute.
Tue, Jul 7, 5:18 PM
douardda updated the diff for D3426: Extract revision's extra_header as a top level attribute.

fix sql schema and simplify the migration script

Tue, Jul 7, 4:50 PM
douardda added inline comments to D3426: Extract revision's extra_header as a top level attribute.
Tue, Jul 7, 4:45 PM
douardda updated the diff for D3426: Extract revision's extra_header as a top level attribute.

typos/fixes in 158.sql

Tue, Jul 7, 3:35 PM
douardda updated the diff for D3426: Extract revision's extra_header as a top level attribute.

add a test for "true" bw compat, aka old data in pg still works

Tue, Jul 7, 3:20 PM
douardda created P717 (An Untitled Masterwork).
Tue, Jul 7, 2:21 PM
douardda accepted D3396: Add raw metadata to the model..

As said on IRC, I'd rather see this diff depends on D3417, but otherwise I'm fine with it.

Tue, Jul 7, 11:26 AM
douardda requested changes to D3435: WIP: Add mercurial.from_disk.HgLoaderFromDisk.

knowing this is still WIP, I have a few comments/questions:

Tue, Jul 7, 10:51 AM

Mon, Jul 6

douardda updated the diff for D3426: Extract revision's extra_header as a top level attribute.

use frozen<list<list<blob>>> in cassandra's schema for extra_headers

Mon, Jul 6, 6:55 PM
douardda added a comment to D3426: Extract revision's extra_header as a top level attribute.
In D3426#84268, @olasd wrote:

Looks nice.

Few comments:

  • I'm not convinced about the Cassandra storage of these. Have you tried a list <frozen list<blob>> or somesuch?
Mon, Jul 6, 4:33 PM
douardda updated the diff for D3426: Extract revision's extra_header as a top level attribute.

bump deps for swh-journal and swh-core

Mon, Jul 6, 2:35 PM
douardda updated the diff for D3426: Extract revision's extra_header as a top level attribute.

bump dep swh-model>= 0.4.0

Mon, Jul 6, 2:21 PM
douardda committed rDMODa7d9aca2b0df: Extract the extra_headers from metadata on the Revision model class (authored by douardda).
Extract the extra_headers from metadata on the Revision model class
Mon, Jul 6, 2:12 PM
douardda closed D3389: Extract the extra_headers from metadata on the Revision model class.
Mon, Jul 6, 2:12 PM
douardda committed rDJNLbf35ea79d3cc: Refactor pytest_plugins.assert_all_objects_consumed() to use model objects (authored by douardda).
Refactor pytest_plugins.assert_all_objects_consumed() to use model objects
Mon, Jul 6, 1:47 PM
douardda closed D3427: Refactor pytest_plugins.assert_all_objects_consumed() to use model objects.
Mon, Jul 6, 1:47 PM
douardda updated the diff for D3427: Refactor pytest_plugins.assert_all_objects_consumed() to use model objects.

typos

Mon, Jul 6, 1:42 PM
douardda added a comment to D3427: Refactor pytest_plugins.assert_all_objects_consumed() to use model objects.

ok

jsyk, noticed 2 typos in the commit message (and diff description):

  • "wee" -> "we"
  • "whic" -> "which"
Mon, Jul 6, 1:40 PM
douardda updated the task description for T2423: Extract the `extra_headers` away from `Revision.metadata` into a top-level immutable object.
Mon, Jul 6, 1:09 PM · Data Model
douardda created D3427: Refactor pytest_plugins.assert_all_objects_consumed() to use model objects.
Mon, Jul 6, 1:01 PM
douardda updated the diff for D3389: Extract the extra_headers from metadata on the Revision model class.

split the 2 test functions in parts, and fix the bwcompat hook

Mon, Jul 6, 11:59 AM
douardda added a comment to D3389: Extract the extra_headers from metadata on the Revision model class.
In D3389#83915, @olasd wrote:

Accepted, with the following caveat: I'm still not sure what the plan is to deploy this now? In the current state of this diff, as soon as we deploy this on workers, data loss will occur (as the new extra_headers field isn't supported by swh.storage).

Now that I think of it, introducing a new field in swh.model needs a coordinated deployment anyway so maybe it doesn't matter (as long as the swh.storage update to support the new field occurs before the next deployment).

Yes this needs a counterpart in storage (wip), and needs a coordinated deployment.

Mon, Jul 6, 10:49 AM
douardda created D3426: Extract revision's extra_header as a top level attribute.
Mon, Jul 6, 10:44 AM
douardda added a revision to T2423: Extract the `extra_headers` away from `Revision.metadata` into a top-level immutable object: D3426: Extract revision's extra_header as a top level attribute.
Mon, Jul 6, 10:44 AM · Data Model
douardda added a comment to D3389: Extract the extra_headers from metadata on the Revision model class.
In D3389#83915, @olasd wrote:

Accepted, with the following caveat: I'm still not sure what the plan is to deploy this now? In the current state of this diff, as soon as we deploy this on workers, data loss will occur (as the new extra_headers field isn't supported by swh.storage).

Now that I think of it, introducing a new field in swh.model needs a coordinated deployment anyway so maybe it doesn't matter (as long as the swh.storage update to support the new field occurs before the next deployment).

Mon, Jul 6, 10:11 AM
douardda accepted D3421: storage: Send metrics from the origin_add endpoint.
Mon, Jul 6, 9:50 AM

Fri, Jul 3

douardda added a comment to T2474: drop blake2 hashes.

not sure about the db space as an argument, but the CPU is by itself worth the move IMHO.

Fri, Jul 3, 4:30 PM · Data Model, Storage manager
douardda updated the diff for D3389: Extract the extra_headers from metadata on the Revision model class.

rebase

Fri, Jul 3, 11:57 AM
douardda requested changes to D3396: Add raw metadata to the model..

Any reason not to use type_validator() in these new schema entities?

Fri, Jul 3, 11:33 AM
douardda accepted D3403: loader-svn: tests: Refactor svn-eol-style scenario.

LGTM but I'd prefer the commit message be self-contained, i.e. explicitly explain the unttest->pytest conversion rather then delegating this to a task in the forge.

Fri, Jul 3, 10:54 AM
douardda accepted D3394: Improve test coverage and type coverage for copy_to.

Thx a lot, LGTM (can't say I've reviewed the tests very carefully though.)

Fri, Jul 3, 10:41 AM

Thu, Jul 2

douardda requested changes to D3394: Improve test coverage and type coverage for copy_to.

Looks globally fine to me, but I have a few comments/requests.

Thu, Jul 2, 2:17 PM
douardda created P710 (An Untitled Masterwork).
Thu, Jul 2, 12:31 PM

Wed, Jul 1

douardda updated the diff for D3389: Extract the extra_headers from metadata on the Revision model class.

more mypy vs. attrs-strict fighting

Wed, Jul 1, 6:02 PM
douardda updated the diff for D3389: Extract the extra_headers from metadata on the Revision model class.

make mypy happy (hopefully)

Wed, Jul 1, 5:39 PM
douardda updated the diff for D3389: Extract the extra_headers from metadata on the Revision model class.

restrict extra_headers to (bytes, bytes) only

Wed, Jul 1, 5:23 PM
douardda updated the diff for D3389: Extract the extra_headers from metadata on the Revision model class.

improve bw-compat support, tests and hypothesis strategies

Wed, Jul 1, 4:04 PM
douardda added a revision to T2423: Extract the `extra_headers` away from `Revision.metadata` into a top-level immutable object: D3389: Extract the extra_headers from metadata on the Revision model class.
Wed, Jul 1, 3:18 PM · Data Model
douardda created D3389: Extract the extra_headers from metadata on the Revision model class.
Wed, Jul 1, 3:18 PM
douardda added a comment to T2471: NPM package angular-ts-manage fails to be properly loaded.

This origin also fails to be mirrored with:

Wed, Jul 1, 10:10 AM · Core Loader
douardda triaged T2471: NPM package angular-ts-manage fails to be properly loaded as High priority.
Wed, Jul 1, 10:05 AM · Core Loader

Tue, Jun 30

douardda added a comment to T1805: Public API v2.

Rereading this task, I have a few comments/questions.

Tue, Jun 30, 2:24 PM · meta-task, Web app
douardda committed rDSTOdc1878b9fb50: Make release_add support adding the same object twice in the same call (authored by douardda).
Make release_add support adding the same object twice in the same call
Tue, Jun 30, 10:07 AM
douardda closed D2771: Make release_add support adding the same object twice in the same call.
Tue, Jun 30, 10:07 AM

Mon, Jun 29

douardda added inline comments to D3355: Generalize origin_metadata to allow support for other object types in the future..
Mon, Jun 29, 5:49 PM