Page MenuHomeSoftware Heritage

douardda (David Douard)
User

Projects (7)

User Details

User Since
Jul 10 2018, 12:38 PM (116 w, 23 h)

Recent Activity

Today

douardda committed rCDFP0264e6b611ca: Fix a typo in the README file (authored by douardda).
Fix a typo in the README file
Wed, Sep 30, 11:48 AM
douardda committed rCDFP110773478d07: Update the db initialization to swh-storage 0.15 (authored by douardda).
Update the db initialization to swh-storage 0.15
Wed, Sep 30, 11:48 AM
douardda requested changes to D4089: WIP Add tests and fix behavior of scanner cli.
Wed, Sep 30, 10:13 AM
douardda accepted D4092: packer: Clean up spurious blanks.
Wed, Sep 30, 9:41 AM
douardda closed T2313: Archive git.fsfe.org (Gitea) as Resolved.

Listed (oneshot full + recurring incremental) and loaded (as far as I can tell).

Wed, Sep 30, 9:37 AM · Archive coverage, Lister

Yesterday

douardda added a comment to D4055: Drop debian packaging, which is now handled on separate branches.

(just updated my swh-env, now I see where this diff comes from :-) )

Tue, Sep 29, 12:09 PM
douardda accepted D4055: Drop debian packaging, which is now handled on separate branches.

sure

Tue, Sep 29, 11:49 AM
douardda accepted D4068: jobs/swh-environment: Improve build script.

LGTM (except the "rm -f ../$module.log " for which I am not convinced it's a good idea)

Tue, Sep 29, 11:47 AM
douardda added a comment to T2313: Archive git.fsfe.org (Gitea).

I've sent an email to the fsfe.

Tue, Sep 29, 11:39 AM · Archive coverage, Lister
douardda triaged T2645: Add listing tasks for gitea instances as Normal priority.
Tue, Sep 29, 10:09 AM · Archive coverage, Lister

Mon, Sep 28

douardda added a comment to T2313: Archive git.fsfe.org (Gitea).

Can this be closed now? What's missing? Adding a listing task?

Mon, Sep 28, 9:47 AM · Archive coverage, Lister

Fri, Sep 25

douardda created P786 (An Untitled Masterwork).
Fri, Sep 25, 6:37 PM
douardda committed rDMODbe8f1a559d82: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Fri, Sep 25, 3:27 PM
douardda closed D4051: Adapt cli declaration entrypoint to swh.core 0.3.
Fri, Sep 25, 3:27 PM
douardda updated the diff for D4051: Adapt cli declaration entrypoint to swh.core 0.3.

and with the pytest.ini hunk we don't need a (non working) dependency on swh.core[testing]

Fri, Sep 25, 3:25 PM
douardda updated the diff for D4051: Adapt cli declaration entrypoint to swh.core 0.3.

add a precision in the ci message for the pytest.ini hunk

Fri, Sep 25, 3:22 PM
douardda created D4051: Adapt cli declaration entrypoint to swh.core 0.3.
Fri, Sep 25, 3:15 PM
douardda created P781 (An Untitled Masterwork).
Fri, Sep 25, 12:59 PM
douardda created P780 (An Untitled Masterwork).
Fri, Sep 25, 12:36 PM
douardda committed rDVAUec87dfe25879: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Fri, Sep 25, 12:09 PM
douardda committed rDSCHbe7a5aeafa7f: Rename sql files according to swh.core 0.3 (authored by douardda).
Rename sql files according to swh.core 0.3
Fri, Sep 25, 11:40 AM
douardda closed D4045: Rename sql files according to swh.core 0.3.
Fri, Sep 25, 11:40 AM
douardda added a comment to D4045: Rename sql files according to swh.core 0.3.

ah yeah, it'd be best to align indeed.

Fri, Sep 25, 11:39 AM
douardda committed rDOBJSRPL99571a1068b0: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Fri, Sep 25, 10:02 AM
douardda created D4045: Rename sql files according to swh.core 0.3.
Fri, Sep 25, 9:57 AM
douardda committed rDSCH5cc573d16f53: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Fri, Sep 25, 9:53 AM

Thu, Sep 24

douardda accepted D4039: Update sql paths for the moved SQL files.

crumbs everywhere

Thu, Sep 24, 8:13 PM
douardda accepted D4038: tests: Run SQL files with psql instead of psycopg2.

sure

Thu, Sep 24, 8:07 PM
douardda committed rCDFPf9dbb018395a: Update grafana dashboards (authored by douardda).
Update grafana dashboards
Thu, Sep 24, 5:50 PM
douardda added a comment to T2624: Create strategy for documentation with a map or a full table of content.

Also https://plan.io/blog/technical-documentation/

Thu, Sep 24, 3:43 PM · Documentation
douardda added a comment to D4012: core.loader: Log information about origin currently being ingested.
In D4012#99525, @olasd wrote:

I don't think the origin url and visit type should be sent in the task result; they're arguments of the task already.

If we want them logged by the worker when the task ends (which I agree would be useful), then we should improve logging on the worker/celery side to show some of the task arguments (for instance, if there's a "url" argument) instead / in addition of the task id.

Thu, Sep 24, 3:28 PM · Core Loader
douardda added inline comments to D4012: core.loader: Log information about origin currently being ingested.
Thu, Sep 24, 3:26 PM · Core Loader
douardda triaged T2621: running tox fails because C.UTF-8 is not available as Normal priority.
Thu, Sep 24, 3:10 PM · Development environment
douardda closed T2119: Monitoring of workers as Resolved.
Thu, Sep 24, 3:08 PM · Scheduling utilities, Sprint 2019/12 (Monitor and Conquer)
douardda closed T1296: Configure static analysis (eg. radon) reporting using jenkins warnings ng plugin, a subtask of T1024: Proper continuous integration setup, as Wontfix.
Thu, Sep 24, 3:07 PM · Restricted Project, Continuous Integration, System administration
douardda closed T1296: Configure static analysis (eg. radon) reporting using jenkins warnings ng plugin as Wontfix.

This is probably mostly deprecated now we have mypy & al. Also the reporting via warnings-ng-plugin may not be such a priority now.

Thu, Sep 24, 3:07 PM · Restricted Project, Continuous Integration, System administration
douardda accepted D4030: Add an optional swh.storage read replica to the docker setup.
Thu, Sep 24, 3:03 PM
douardda added a comment to T1330: Ensure documentation building process does not generates any rst syntax warning.

I'd like to close this task, but unfortunately:

Thu, Sep 24, 2:47 PM · Documentation
douardda closed T358: doc: high-level architecture diagram, a subtask of T509: Generate and publish Software Heritage Development Documentation, as Resolved.
Thu, Sep 24, 2:42 PM · Documentation
douardda closed T358: doc: high-level architecture diagram as Resolved.

I guess we can say that, let's close this.

Thu, Sep 24, 2:42 PM · Documentation
douardda closed T1601: Journal client of swh-storage mirrors as Resolved.

I guess this can be closed now

Thu, Sep 24, 2:42 PM · Sprint 2019 03
douardda accepted D3981: Support different database flavors in the SQL scripts.
Thu, Sep 24, 2:35 PM
douardda committed rDENV402b154052d5: Rename mirror-related things from replica to mirror (authored by douardda).
Rename mirror-related things from replica to mirror
Thu, Sep 24, 1:04 PM
douardda closed D4022: Rename mirror-related things from replica to mirror.
Thu, Sep 24, 1:04 PM
douardda accepted D4024: Docker: Migrate relevant services to the split `swh db create` / `swh db init`.

thx a lot

Thu, Sep 24, 1:03 PM
douardda accepted D4023: Update debian-like host instructions to reference the documentation.

sure go

Thu, Sep 24, 12:08 PM
douardda committed rDWCLI04375ab5713d: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Thu, Sep 24, 11:57 AM
douardda committed rDDEP047588a03f0b: Fix dependency-related hell in tox (authored by douardda).
Fix dependency-related hell in tox
Thu, Sep 24, 11:26 AM
douardda closed D4029: Attempt to fix dependency-related hell in tox.
Thu, Sep 24, 11:26 AM
douardda updated the diff for D4029: Attempt to fix dependency-related hell in tox.

update ci message

Thu, Sep 24, 11:26 AM
douardda added a comment to D4029: Attempt to fix dependency-related hell in tox.

commit message! ;)

Thu, Sep 24, 11:18 AM
douardda created D4029: Attempt to fix dependency-related hell in tox.
Thu, Sep 24, 11:13 AM

Wed, Sep 23

douardda committed rDSEA7b9a254fc016: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 6:40 PM
douardda committed rDTSCNf10d3d87cbcb: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 6:27 PM
douardda committed rDTPL19683b52773f: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 6:21 PM
douardda committed rDOBJSfd1cc6dc71c8: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 6:18 PM
douardda updated the diff for D4022: Rename mirror-related things from replica to mirror.

fix a couple of sedisms

Wed, Sep 23, 6:12 PM
douardda created D4022: Rename mirror-related things from replica to mirror.
Wed, Sep 23, 6:07 PM
douardda committed rDLDBASE7ed1927b9968: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 5:50 PM
douardda committed rDLSd10f78d80c95: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 5:42 PM
douardda committed rDCIDX8c4b33ba2c74: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 5:26 PM
douardda committed rDICP750a15e4f9ae: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 5:13 PM
douardda committed rDGRPHc293d0726a7b: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 5:04 PM
douardda committed rDDEP3416abd01019: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 4:37 PM
douardda committed rDDATASET6205fdee730b: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 4:36 PM
douardda committed rDSTOc97b23bcd78b: Adapt cli declaration entrypoint to swh.core 0.3 (authored by douardda).
Adapt cli declaration entrypoint to swh.core 0.3
Wed, Sep 23, 4:35 PM
douardda closed D4002: Adapt cli declaration entrypoint to swh.core 0.3.
Wed, Sep 23, 4:35 PM
douardda updated the diff for D4002: Adapt cli declaration entrypoint to swh.core 0.3.

rebase

Wed, Sep 23, 4:29 PM
douardda committed rCDFJ4e78deb8b89c: Add libfuse3-dev in the docker (authored by douardda).
Add libfuse3-dev in the docker
Wed, Sep 23, 3:45 PM
douardda closed D4021: Add libfuse3-dev in the docker.
Wed, Sep 23, 3:45 PM
douardda created D4021: Add libfuse3-dev in the docker.
Wed, Sep 23, 3:44 PM
douardda committed rCJSWHf46ad02ee918: Add swh-fuse (authored by douardda).
Add swh-fuse
Wed, Sep 23, 2:59 PM
douardda closed D4019: Add swh-fuse.
Wed, Sep 23, 2:59 PM
douardda created D4019: Add swh-fuse.
Wed, Sep 23, 2:52 PM
douardda added a comment to D3997: Add support for database flavors to `swh db-init`.

LGTM, just a couple of wishes (and we are not even close to xmas!)

Wed, Sep 23, 2:20 PM
douardda accepted D4014: pytest_plugin: Change dbname to storage to avoid clash in tests.
Wed, Sep 23, 1:11 PM · Storage manager
douardda accepted D4015: pytest_plugin: Order the fixture definitions in dependency order.
Wed, Sep 23, 1:10 PM
douardda accepted D4016: pytest_plugin: Reuse swh_storage_postgresql connection string.
Wed, Sep 23, 1:10 PM
douardda added a comment to D4014: pytest_plugin: Change dbname to storage to avoid clash in tests.

it's unclear to me if the hunk in swh_storage_backend_config is related with the fix or not.
isn't the very last diff hunk enough for the job?

Wed, Sep 23, 11:56 AM · Storage manager

Tue, Sep 22

douardda abandoned D3902: cli: do not add sub sub commands to the main swh cli group.
Tue, Sep 22, 2:41 PM
douardda committed rDCORE08788aa3783c: Do not automagically add click cmd from swh.cli.subcommands (authored by douardda).
Do not automagically add click cmd from swh.cli.subcommands
Tue, Sep 22, 2:40 PM
douardda closed D3930: Do not automagically add click cmd from swh.cli.subcommands.
Tue, Sep 22, 2:39 PM
douardda created D4002: Adapt cli declaration entrypoint to swh.core 0.3.
Tue, Sep 22, 12:06 PM
douardda committed rDSTO30cdb78f17eb: Drop the -swh- part of sql files (authored by douardda).
Drop the -swh- part of sql files
Tue, Sep 22, 10:32 AM
douardda committed rDSTO915575d5b50a: Rename 10-swh-init.sql as 10-superuser-init.sql (authored by douardda).
Rename 10-swh-init.sql as 10-superuser-init.sql
Tue, Sep 22, 10:32 AM
douardda closed D4001: Rename 10-swh-init.sql as 10-superuser-init.sql.
Tue, Sep 22, 10:32 AM
douardda created D4001: Rename 10-swh-init.sql as 10-superuser-init.sql.
Tue, Sep 22, 10:21 AM
douardda updated the diff for D3930: Do not automagically add click cmd from swh.cli.subcommands.

rebase

Tue, Sep 22, 9:40 AM
douardda committed rDCORE56d505f34a96: Split db-init cli in 2 (init and create) and move them under 'swh db' (authored by douardda).
Split db-init cli in 2 (init and create) and move them under 'swh db'
Tue, Sep 22, 9:37 AM
douardda closed D4000: Split db-init cli in 2 (init and create) and move them under 'swh db'.
Tue, Sep 22, 9:37 AM
douardda updated the diff for D4000: Split db-init cli in 2 (init and create) and move them under 'swh db'.

fix docstring (thx ardumont)

Tue, Sep 22, 9:36 AM

Mon, Sep 21

douardda added inline comments to D4000: Split db-init cli in 2 (init and create) and move them under 'swh db'.
Mon, Sep 21, 5:21 PM
douardda updated the diff for D4000: Split db-init cli in 2 (init and create) and move them under 'swh db'.

several fixes and add a --template option to 'swh db create'

Mon, Sep 21, 5:19 PM
douardda updated the diff for D4000: Split db-init cli in 2 (init and create) and move them under 'swh db'.

better ci mesg

Mon, Sep 21, 4:42 PM
douardda created D4000: Split db-init cli in 2 (init and create) and move them under 'swh db'.
Mon, Sep 21, 4:36 PM
douardda accepted D3997: Add support for database flavors to `swh db-init`.
Mon, Sep 21, 2:25 PM
douardda added a comment to D3997: Add support for database flavors to `swh db-init`.

I don't understand how this is supposed to work.

How does the flavor "name" is taken in consideration? Is it supposed to select "flavored" versions of sql files? What's the pattern ? As written, I don't understand how the flavor "name" is used besides being written the dbflavor table.

Oh now I get it (after reading D3981)

It's the presence of a xxx-flavor.sql that triggers the insertion/update of the flavor given as cli argument in the dbflavor table, which then specify how the other sql files are executed (thx to /if statements in there).

Mon, Sep 21, 11:57 AM
douardda added a comment to D3997: Add support for database flavors to `swh db-init`.

I don't understand how this is supposed to work.

How does the flavor "name" is taken in consideration? Is it supposed to select "flavored" versions of sql files? What's the pattern ? As written, I don't understand how the flavor "name" is used besides being written the dbflavor table.

Mon, Sep 21, 11:56 AM
douardda committed rDSTO67ee86b8e03f: Warn about skipped_content sneaking the 'content' topics (authored by douardda).
Warn about skipped_content sneaking the 'content' topics
Mon, Sep 21, 10:44 AM
douardda committed rDSTO8de656451239: Small fix in the graph replayer to prevent a wrong warning (authored by douardda).
Small fix in the graph replayer to prevent a wrong warning
Mon, Sep 21, 10:44 AM