Page MenuHomeSoftware Heritage

douardda (David Douard)
User

User Details

User Since
Jul 10 2018, 12:38 PM (41 w, 2 d)

Recent Activity

Fri, Apr 19

douardda updated the diff for D1426: cli: add a generic main 'swh' command and extract the db-init command.

no need to keep the cli plugins in a dict

Fri, Apr 19, 4:24 PM
douardda added inline comments to D1426: cli: add a generic main 'swh' command and extract the db-init command.
Fri, Apr 19, 4:19 PM
douardda updated the diff for D1429: docs: update the getting-started document to use 'swh deposit upload' command.

rebased

Fri, Apr 19, 3:50 PM
douardda updated the diff for D1428: cli: make the deposit cli command a subcommand of the main 'swh' one.

Add requests to requirements.txt since it does come with swh.core any more

Fri, Apr 19, 3:50 PM
douardda updated the summary of D1427: setup: extract db and http related parts in dedicated optional 'extras'.
Fri, Apr 19, 3:37 PM
douardda updated the diff for D1427: setup: extract db and http related parts in dedicated optional 'extras'.

kill the PostresHandler instead of "shuffling it around" (© olasd)

Fri, Apr 19, 3:36 PM
douardda added a comment to D1427: setup: extract db and http related parts in dedicated optional 'extras'.
In D1427#31359, @olasd wrote:

The PostgresHandler has been dead code for two years, I think we can just remove it rather than shuffling it around.

Fri, Apr 19, 3:28 PM
Herald added a reviewer for D1429: docs: update the getting-started document to use 'swh deposit upload' command: Reviewers.
Fri, Apr 19, 3:27 PM
douardda updated the summary of D1428: cli: make the deposit cli command a subcommand of the main 'swh' one.
Fri, Apr 19, 2:39 PM
douardda updated the summary of D1428: cli: make the deposit cli command a subcommand of the main 'swh' one.
Fri, Apr 19, 2:39 PM
Herald added a reviewer for D1428: cli: make the deposit cli command a subcommand of the main 'swh' one: Reviewers.
Fri, Apr 19, 2:38 PM
douardda updated the diff for D1427: setup: extract db and http related parts in dedicated optional 'extras'.

Fix commit messages

Fri, Apr 19, 2:31 PM
Herald added a reviewer for D1427: setup: extract db and http related parts in dedicated optional 'extras': Reviewers.
Fri, Apr 19, 12:13 PM
douardda added a revision to T1678: Make the swh.core package define some 'extras' features to limit dependencies of the 'plain' swh.core package: D1427: setup: extract db and http related parts in dedicated optional 'extras'.
Fri, Apr 19, 12:13 PM · SWH command line interface
Herald added a reviewer for D1426: cli: add a generic main 'swh' command and extract the db-init command: Reviewers.
Fri, Apr 19, 12:13 PM
douardda committed rDCORE22e60ae54315: Reorder a few import statements (authored by douardda).
Reorder a few import statements
Fri, Apr 19, 12:12 PM
douardda triaged T1678: Make the swh.core package define some 'extras' features to limit dependencies of the 'plain' swh.core package as Normal priority.
Fri, Apr 19, 11:54 AM · SWH command line interface
douardda added a comment to T1671: Implement a uniform and consistent cli to be used by all SWH packages.

This also requires we can install only a very light base version of swh.core (typically without db or aiohttp/flask dependencies).

Fri, Apr 19, 10:25 AM · SWH command line interface
douardda added a comment to T1671: Implement a uniform and consistent cli to be used by all SWH packages.

A possible way of doing such a thing is to have an almost empty swh.cli package that declares the main entry point: the swh command group (using click) which defines a few very command command line options (like --log-level), and a mecanism to load installed SWH cli plugins via setuptools' pkg_resources

Fri, Apr 19, 10:23 AM · SWH command line interface

Thu, Apr 18

douardda added a comment to T1671: Implement a uniform and consistent cli to be used by all SWH packages.

This generic swh cli should be done so that is minimal dependency installed especially for "pure client" commands (e.g. swh.deposit's deposit or swh.model's identify tools).

Thu, Apr 18, 3:14 PM · SWH command line interface
douardda added a parent task for T1651: Create a separate project for deposit-client: T1671: Implement a uniform and consistent cli to be used by all SWH packages.
Thu, Apr 18, 3:03 PM · SWH command line interface, SWORD deposit
douardda added a subtask for T1671: Implement a uniform and consistent cli to be used by all SWH packages: T1651: Create a separate project for deposit-client.
Thu, Apr 18, 3:03 PM · SWH command line interface
douardda added a parent task for T1658: Gather CLI tools spread in SWH packages: T1671: Implement a uniform and consistent cli to be used by all SWH packages.
Thu, Apr 18, 3:02 PM · SWH command line interface
douardda added a subtask for T1671: Implement a uniform and consistent cli to be used by all SWH packages: T1658: Gather CLI tools spread in SWH packages.
Thu, Apr 18, 3:02 PM · SWH command line interface
douardda triaged T1671: Implement a uniform and consistent cli to be used by all SWH packages as Normal priority.
Thu, Apr 18, 3:02 PM · SWH command line interface
douardda updated the task description for T1651: Create a separate project for deposit-client.
Thu, Apr 18, 2:59 PM · SWH command line interface, SWORD deposit
douardda updated the task description for T1651: Create a separate project for deposit-client.
Thu, Apr 18, 2:57 PM · SWH command line interface, SWORD deposit
douardda added a comment to T1658: Gather CLI tools spread in SWH packages.

Currently, we have:

Thu, Apr 18, 11:49 AM · SWH command line interface
douardda closed T1599: Analyze objstorage's Azure updateness as Resolved.
Thu, Apr 18, 10:35 AM · Sprint 2019 03
douardda triaged T1658: Gather CLI tools spread in SWH packages as Normal priority.
Thu, Apr 18, 9:36 AM · SWH command line interface
douardda added a project to T1651: Create a separate project for deposit-client: SWH command line interface.
Thu, Apr 18, 9:25 AM · SWH command line interface, SWORD deposit
douardda created SWH command line interface.
Thu, Apr 18, 9:25 AM
douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

It is these graphs we are still missing.

Thu, Apr 18, 9:19 AM · Sprint 2018 12, System administration

Tue, Apr 16

douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

Indeed it is the object of T1428. That's why I am a bit puzzled the work you have in progress does not simply target T1356. I was expecting some response to this very task in your grafanalib based code, which I did not find. So I was wondering if I missed something, that some data where still in munin only.

Tue, Apr 16, 3:04 PM · Sprint 2018 12, System administration
douardda added inline comments to D1416: deposit: Adapt revision dates setting according to new rules.
Tue, Apr 16, 12:35 PM · SWORD deposit
douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

When I asked where to put such work-in-progress, you suggested the snippets repository.

Tue, Apr 16, 12:04 PM · Sprint 2018 12, System administration
douardda accepted D1418: deposit.cli: Generate uuid as is.

LGTM

Tue, Apr 16, 11:30 AM
douardda added a comment to T1442: Replace Munin graphs with Grafana/Prometheus dashboards.

Work-in-progress Grafanalib dashboards have been added to the https://forge.softwareheritage.org/source/snippets/ repository.

Tue, Apr 16, 9:41 AM · Sprint 2018 12, System administration

Mon, Apr 15

douardda added inline comments to D1418: deposit.cli: Generate uuid as is.
Mon, Apr 15, 5:02 PM
douardda added a comment to T1651: Create a separate project for deposit-client.

As discussed, this task should be actually the first step of a more generic software-heritage cli client module (using HTTP based AIPs/RPCs).

Mon, Apr 15, 4:55 PM · SWH command line interface, SWORD deposit
douardda accepted D1335: Don't store empty metadata dicts..

LGTM but I'm not very familiar with this code.

Mon, Apr 15, 4:48 PM
douardda accepted D1346: docker-compose: Add journal-backfiller.
Mon, Apr 15, 4:42 PM
douardda accepted D1413: Add a from_dict() method to model classes, that does the inverse of to_dict()..
Mon, Apr 15, 4:40 PM
douardda accepted D1417: Fix connection leaks in revision_add and directory_add..
Mon, Apr 15, 4:39 PM
douardda requested changes to D1413: Add a from_dict() method to model classes, that does the inverse of to_dict()..
Mon, Apr 15, 1:40 PM
douardda requested changes to D1417: Fix connection leaks in revision_add and directory_add..
Mon, Apr 15, 1:27 PM
douardda added a comment to D1417: Fix connection leaks in revision_add and directory_add..

could you explain in the commit message why this solves a cnx leak?

Mon, Apr 15, 1:27 PM
douardda requested changes to D1346: docker-compose: Add journal-backfiller.

few minor fixes to be make, then should be ok.

Mon, Apr 15, 12:44 PM

Fri, Apr 12

douardda accepted D1411: Refactor deposit cli and some other improvments.

lgtm (but I'm obviously biased)

Fri, Apr 12, 4:40 PM
douardda accepted D1414: Improve deposit container to actually use loader/checker.

lgtm

Fri, Apr 12, 4:18 PM
douardda committed rDDEP9f8709da3b60: cli: fix logging setup (authored by douardda).
cli: fix logging setup
Fri, Apr 12, 9:41 AM
douardda committed rDDEPe07add73dbd7: config: do not overwrite the DJANGO_SETTINGS_MODULE if already defined (authored by douardda).
config: do not overwrite the DJANGO_SETTINGS_MODULE if already defined
Fri, Apr 12, 9:41 AM
douardda committed rDDEP4fb78f6e3672: cli/admin/user: add a user existence check (authored by douardda).
cli/admin/user: add a user existence check
Fri, Apr 12, 9:41 AM
douardda committed rDDEPc4de5bdb07c5: cli: kill the unneeded dry-run option (authored by douardda).
cli: kill the unneeded dry-run option
Fri, Apr 12, 9:41 AM
douardda committed rDDEPa1fc0e8808b9: client: use a global logger (authored by douardda).
client: use a global logger
Fri, Apr 12, 9:41 AM
douardda committed rDDEPcf58937eb04e: cli: split the cli.py module in a package with submodules (authored by douardda).
cli: split the cli.py module in a package with submodules
Fri, Apr 12, 9:41 AM
douardda committed rDDEP248c43674a8e: cli/admin/user: provide more information at creation (authored by douardda).
cli/admin/user: provide more information at creation
Fri, Apr 12, 9:41 AM
douardda committed rDDEP426500c3abee: cli: fix service document parsing to retrieve collection (authored by douardda).
cli: fix service document parsing to retrieve collection
Fri, Apr 12, 9:41 AM
douardda committed rDDEPb07d24405772: model: kill DepositRequestType (authored by douardda).
model: kill DepositRequestType
Fri, Apr 12, 9:41 AM
douardda committed rDDEP38b472f3d174: setup: extract server specific requirements in a dedicated 'server' extra (authored by douardda).
setup: extract server specific requirements in a dedicated 'server' extra
Fri, Apr 12, 9:41 AM

Thu, Apr 11

douardda committed R65:97981b2f2793: API: simplify a bit the RemoteStorage class (authored by douardda).
API: simplify a bit the RemoteStorage class
Thu, Apr 11, 12:19 PM
douardda committed R65:4ed7d0ae0dd6: API: refactor the /directory/ls endpoint (authored by douardda).
API: refactor the /directory/ls endpoint
Thu, Apr 11, 12:19 PM
douardda committed rDLSe5c355903323: tasks: fix handling of unsupported promise.save() calls (authored by douardda).
tasks: fix handling of unsupported promise.save() calls
Thu, Apr 11, 11:13 AM
douardda closed D1404: tasks: fix handling of unsupported promise.save() calls.
Thu, Apr 11, 11:13 AM
douardda added a comment to D1404: tasks: fix handling of unsupported promise.save() calls.

That code (and the lines before) look very similar to each other.

Thu, Apr 11, 11:13 AM
Herald added a reviewer for D1404: tasks: fix handling of unsupported promise.save() calls: Reviewers.
Thu, Apr 11, 11:06 AM
douardda committed rDJNL7ace7f201d31: Merge tag 'v0.0.9' (authored by douardda).
Merge tag 'v0.0.9'
Thu, Apr 11, 10:59 AM

Wed, Apr 10

douardda accepted D1396: config: Allow to specify development database file.

ok but see my comment plz

Wed, Apr 10, 4:12 PM
douardda committed rCDFD30e9c934ef20: lister: allways run the db initialization step (authored by douardda).
lister: allways run the db initialization step
Wed, Apr 10, 2:40 PM
douardda committed rCDFD6d49580cd56c: replica: ensure entrypoints are defined in replica related services (authored by douardda).
replica: ensure entrypoints are defined in replica related services
Wed, Apr 10, 2:40 PM
douardda committed rCDFDb7f02d8add7f: dockerfile: install python packages in a venv under the swh user (authored by douardda).
dockerfile: install python packages in a venv under the swh user
Wed, Apr 10, 2:40 PM
douardda committed rCDFD6558c6c8ad18: entrypoint: pgsql has been moved to ~swh/ (authored by douardda).
entrypoint: pgsql has been moved to ~swh/
Wed, Apr 10, 2:40 PM
douardda committed rCDFD7b0ea2803606: Remove entrypoint.sh files from the Dockerfile (authored by douardda).
Remove entrypoint.sh files from the Dockerfile
Wed, Apr 10, 2:40 PM
douardda committed rCDFDd1f5b87ba8c7: Dockerfile: small readability refactoring (authored by douardda).
Dockerfile: small readability refactoring
Wed, Apr 10, 2:40 PM
douardda committed rCDFD53fa588c7fc2: Factorise the pip install of source repos in a shell function (authored by douardda).
Factorise the pip install of source repos in a shell function
Wed, Apr 10, 2:40 PM
douardda closed D1394: Several refactorings.
Wed, Apr 10, 2:40 PM
douardda closed D1394: Several refactorings.
Wed, Apr 10, 2:40 PM
douardda committed rCDFD9359da6dbf95: Add support for a kafka-powered replica of the graph. (authored by douardda).
Add support for a kafka-powered replica of the graph.
Wed, Apr 10, 2:40 PM
douardda closed D1393: Add support for a kafka-powered replica of the graph..
Wed, Apr 10, 2:40 PM
douardda committed rCDFD587f9df57a80: lister: Use explicit postgresql URLs as a workaround for a sqlalchemy issue (authored by douardda).
lister: Use explicit postgresql URLs as a workaround for a sqlalchemy issue
Wed, Apr 10, 2:40 PM
douardda closed D1392: lister: Use explicit postgresql URLs as a workaround for a sqlalchemy issue.
Wed, Apr 10, 2:40 PM
douardda committed rCDFD20b0733b84f3: swh-web: enforce the database file to be created in either $HOME or /tmp (authored by douardda).
swh-web: enforce the database file to be created in either $HOME or /tmp
Wed, Apr 10, 2:40 PM
douardda closed D1391: swh-web: enforce the database file to be created in either $HOME or /tmp.
Wed, Apr 10, 2:40 PM
douardda committed rDJNL8d17a3c04195: bump dependencies to swh.storage>=0.0.138 and swh.model>=0.0.34 (authored by douardda).
bump dependencies to swh.storage>=0.0.138 and swh.model>=0.0.34
Wed, Apr 10, 1:43 PM
Herald added a reviewer for D1394: Several refactorings: Reviewers.
Wed, Apr 10, 12:29 PM
Herald added a reviewer for D1393: Add support for a kafka-powered replica of the graph.: Reviewers.
Wed, Apr 10, 12:24 PM
Herald added a reviewer for D1392: lister: Use explicit postgresql URLs as a workaround for a sqlalchemy issue: Reviewers.
Wed, Apr 10, 12:23 PM
Herald added a reviewer for D1391: swh-web: enforce the database file to be created in either $HOME or /tmp: Reviewers.
Wed, Apr 10, 12:23 PM
douardda committed rCDFD8a2458ee2bf0: compose: fix inconsistent dependency (authored by douardda).
compose: fix inconsistent dependency
Wed, Apr 10, 12:17 PM

Fri, Apr 5

douardda requested changes to D1346: docker-compose: Add journal-backfiller.

I'm not very fond adding this kind of service in the main docker-compose file.
I would prefer a dedicated docker-compose file to be used as an override, eg:

Fri, Apr 5, 4:51 PM
douardda committed rCJSWHa8e0f23044e0: templates: increase unit tests timeout to 10mn (authored by douardda).
templates: increase unit tests timeout to 10mn
Fri, Apr 5, 3:43 PM
douardda closed D1355: templates: raise unit tests timeout to 10mn.
Fri, Apr 5, 3:43 PM
douardda updated the diff for D1355: templates: raise unit tests timeout to 10mn.

reword

Fri, Apr 5, 3:24 PM
Herald added a reviewer for D1355: templates: raise unit tests timeout to 10mn: Reviewers.
Fri, Apr 5, 3:22 PM
douardda accepted D1352: Fix test_direct_writer to work with the latest version of swh-storage..
Fri, Apr 5, 2:50 PM

Tue, Apr 2

douardda accepted D1330: Add a draft of 'StorageReplayer', a tool that fills a Storage from Kafka..
Tue, Apr 2, 4:02 PM
douardda added inline comments to D1330: Add a draft of 'StorageReplayer', a tool that fills a Storage from Kafka..
Tue, Apr 2, 3:53 PM
douardda accepted D1319: Require sphinx < 2, sphinxcontrib_images does not support sphinx 2..
Tue, Apr 2, 3:47 PM
douardda added inline comments to D1319: Require sphinx < 2, sphinxcontrib_images does not support sphinx 2..
Tue, Apr 2, 2:59 PM
douardda requested changes to D1330: Add a draft of 'StorageReplayer', a tool that fills a Storage from Kafka..
Tue, Apr 2, 2:57 PM