Page MenuHomeSoftware Heritage
Feed Advanced Search

Jan 8 2020

douardda created D2508: Add a tox.ini for docker tests.
Jan 8 2020, 4:54 PM
douardda committed rDENVb5f575447b1d: Add testinfra based e2e tests for the deposit (authored by douardda).
Add testinfra based e2e tests for the deposit
Jan 8 2020, 4:54 PM
douardda closed D2491: Add testinfra based e2e tests for the deposit.
Jan 8 2020, 4:54 PM
douardda updated the diff for D2491: Add testinfra based e2e tests for the deposit.

forgot the renaming of host->deposit_host

Jan 8 2020, 4:51 PM
douardda updated the diff for D2491: Add testinfra based e2e tests for the deposit.

Update/fixes according vlorentz' comments

Jan 8 2020, 4:00 PM
douardda committed rDDEP19a10ba7cd43: api/private/read: use application/zip as content_type (authored by douardda).
api/private/read: use application/zip as content_type
Jan 8 2020, 3:48 PM
douardda closed D2507: api/private/read: use application/zip as content_type.
Jan 8 2020, 3:48 PM
douardda committed rDDEP90a5072b1f78: api/private/read: use shutil.make_archive directly to generate the zip file (authored by douardda).
api/private/read: use shutil.make_archive directly to generate the zip file
Jan 8 2020, 3:48 PM
douardda closed D2506: api/private/read: use shutil.make_archive directly to generate the zip file.
Jan 8 2020, 3:48 PM
douardda created D2507: api/private/read: use application/zip as content_type.
Jan 8 2020, 1:25 PM
douardda created D2506: api/private/read: use shutil.make_archive directly to generate the zip file.
Jan 8 2020, 1:24 PM
douardda committed rDDEP5e686848e353: Rename deposit/client/__init__.py as deposit/client.py (authored by douardda).
Rename deposit/client/__init__.py as deposit/client.py
Jan 8 2020, 1:24 PM

Jan 7 2020

douardda committed rDDEPc8cf77bd5d60: cli/client: add a --format option to cli client commands (authored by douardda).
cli/client: add a --format option to cli client commands
Jan 7 2020, 10:24 AM
douardda closed D2489: cli/client: add a --format option to cli client commands.
Jan 7 2020, 10:24 AM
douardda committed rDDEP5a6359516a3d: tests: make tests run with a proper scheduler (authored by douardda).
tests: make tests run with a proper scheduler
Jan 7 2020, 10:24 AM
douardda closed D2474: conftest: use pytest-postgresql for django related tests.
Jan 7 2020, 10:24 AM
douardda committed rDDEP3edb9134613d: Always generate a zip file from uploaded archives (authored by douardda).
Always generate a zip file from uploaded archives
Jan 7 2020, 10:24 AM
douardda committed rDDEP647725961068: conftest: use pytest-postgresql for django related tests (authored by douardda).
conftest: use pytest-postgresql for django related tests
Jan 7 2020, 10:24 AM
douardda closed D2473: Always generate a zip file from uploaded archives.
Jan 7 2020, 10:24 AM
douardda closed D2472: Move load-deposit task creation code in the deposit-check private API endpoint.
Jan 7 2020, 10:24 AM
douardda committed rDDEP00cc60839e3a: Move load-deposit task creation code in the deposit-check private API endpoint (authored by douardda).
Move load-deposit task creation code in the deposit-check private API endpoint
Jan 7 2020, 10:24 AM
douardda committed rDDEP5c85e6de2572: Make load-deposit and check-deposit URL argument absolute (authored by douardda).
Make load-deposit and check-deposit URL argument absolute
Jan 7 2020, 10:24 AM
douardda closed D2427: Make load-deposit and check-deposit URL argument absolute.
Jan 7 2020, 10:24 AM
douardda accepted D2478: Update documentation of --author to use names instead of emails..
Jan 7 2020, 9:49 AM

Jan 6 2020

douardda accepted D2495: package.loader: Fail loading early when failing to create an origin/visit.
Jan 6 2020, 3:34 PM · Core Loader
douardda accepted D2463: cran loader: Add implementation.

did only a quick review, lgtm. Note however my comment on using path.basename() instead of path.split()[-1]

Jan 6 2020, 2:00 PM · Origin-CRAN

Dec 20 2019

douardda added inline comments to D2491: Add testinfra based e2e tests for the deposit.
Dec 20 2019, 2:56 PM
douardda added a comment to D2491: Add testinfra based e2e tests for the deposit.

that looks quite awesomly promising.

will that replace the current shell scripts?

Dec 20 2019, 2:50 PM
douardda created D2491: Add testinfra based e2e tests for the deposit.
Dec 20 2019, 2:32 PM
douardda created D2489: cli/client: add a --format option to cli client commands.
Dec 20 2019, 2:24 PM
douardda updated the diff for D2474: conftest: use pytest-postgresql for django related tests.

oops forgot a commit

Dec 20 2019, 2:23 PM
douardda updated the diff for D2474: conftest: use pytest-postgresql for django related tests.

rebase

Dec 20 2019, 2:23 PM
douardda updated the diff for D2473: Always generate a zip file from uploaded archives.

rebase

Dec 20 2019, 2:22 PM
douardda updated the diff for D2472: Move load-deposit task creation code in the deposit-check private API endpoint.

rebase + small fix in client/__init__.py

Dec 20 2019, 2:22 PM
douardda updated the diff for D2427: Make load-deposit and check-deposit URL argument absolute.

rebase

Dec 20 2019, 2:21 PM
douardda created P580 (An Untitled Masterwork).
Dec 20 2019, 2:12 PM
douardda committed rDENVe235e05f35de: docker: fix loader task queues configured in the loader.yml file (authored by douardda).
docker: fix loader task queues configured in the loader.yml file
Dec 20 2019, 10:25 AM
douardda closed D2475: docker: fix loader task queues configured in the loader.yml file.
Dec 20 2019, 10:25 AM

Dec 19 2019

douardda requested changes to D2463: cran loader: Add implementation.
Dec 19 2019, 11:19 AM · Origin-CRAN
douardda added inline comments to D2421: docker/deposit: configure nginx so the WSGI mountpoint for /deposit is ok-ish.
Dec 19 2019, 10:33 AM
douardda updated the diff for D2475: docker: fix loader task queues configured in the loader.yml file.

rebase

Dec 19 2019, 10:21 AM
douardda closed D2421: docker/deposit: configure nginx so the WSGI mountpoint for /deposit is ok-ish.
Dec 19 2019, 10:17 AM
douardda committed rDENV06b74b1c4e70: docker/deposit: configure nginx so the WSGI mountpoint for /deposit is ok-ish (authored by douardda).
docker/deposit: configure nginx so the WSGI mountpoint for /deposit is ok-ish
Dec 19 2019, 10:17 AM
douardda abandoned D2476: docker/deposit: configure nginx so the WSGI mountpoint for /deposit is ok-ish.

same as D2421

Dec 19 2019, 10:17 AM
douardda updated the diff for D2421: docker/deposit: configure nginx so the WSGI mountpoint for /deposit is ok-ish.

rebase

Dec 19 2019, 10:17 AM
douardda created D2476: docker/deposit: configure nginx so the WSGI mountpoint for /deposit is ok-ish.
Dec 19 2019, 10:15 AM
douardda created D2475: docker: fix loader task queues configured in the loader.yml file.
Dec 19 2019, 10:15 AM
douardda committed rDENV199e56b8c8b8: Add a caveats section in docker/README.md (authored by douardda).
Add a caveats section in docker/README.md
Dec 19 2019, 10:14 AM
douardda closed D2435: Add a caveats section in docker/README.md.
Dec 19 2019, 10:14 AM
douardda closed D2434: Remove task_modules declarations for lister and loader conf files in docker.
Dec 19 2019, 10:14 AM
douardda committed rDENV285b5f0b2fe3: Attempt to rationalize a bit workers config in docker (authored by douardda).
Attempt to rationalize a bit workers config in docker
Dec 19 2019, 10:14 AM
douardda committed rDENV92148f29e3dd: Remove task_modules declarations for lister and loader conf files in docker (authored by douardda).
Remove task_modules declarations for lister and loader conf files in docker
Dec 19 2019, 10:14 AM
douardda closed D2433: Attempt to rationalize a bit workers config in docker.
Dec 19 2019, 10:14 AM
douardda updated the diff for D2435: Add a caveats section in docker/README.md.

rebase

Dec 19 2019, 10:14 AM
douardda updated the diff for D2434: Remove task_modules declarations for lister and loader conf files in docker.

rebase

Dec 19 2019, 10:12 AM
douardda updated the diff for D2433: Attempt to rationalize a bit workers config in docker.

rebase

Dec 19 2019, 10:12 AM
douardda added inline comments to D2427: Make load-deposit and check-deposit URL argument absolute.
Dec 19 2019, 10:03 AM
douardda created D2474: conftest: use pytest-postgresql for django related tests.
Dec 19 2019, 9:59 AM
douardda updated the diff for D2473: Always generate a zip file from uploaded archives.

rebase

Dec 19 2019, 9:58 AM
douardda updated the diff for D2472: Move load-deposit task creation code in the deposit-check private API endpoint.

Forgot a revision in this diff

Dec 19 2019, 9:57 AM
douardda created D2473: Always generate a zip file from uploaded archives.
Dec 19 2019, 9:54 AM
douardda created D2472: Move load-deposit task creation code in the deposit-check private API endpoint.
Dec 19 2019, 9:54 AM
douardda updated the diff for D2427: Make load-deposit and check-deposit URL argument absolute.

don't remember what I did change in here...

Dec 19 2019, 9:53 AM

Dec 17 2019

douardda added a comment to T2071: [HAL] Fix published zip in zip.

I am still not sure how to understand this.

Dec 17 2019, 2:50 PM · SWORD deposit
douardda added a comment to T2071: [HAL] Fix published zip in zip.

I'm not completely sure I get this ticket; I understand that HAL may (always?) produce invalid zip files when it uploads a deposit in SWH. So my questions are:

Dec 17 2019, 11:00 AM · SWORD deposit

Dec 13 2019

douardda accepted D2448: Use a btree of (task_type, md5(arguments)) to match task arguments.

LGTM

Dec 13 2019, 11:37 AM
douardda committed rDSCH0b04220c957b: Remove the creation of the 'load-deposit' task type (authored by douardda).
Remove the creation of the 'load-deposit' task type
Dec 13 2019, 11:20 AM
douardda closed D2441: Remove the creation of the 'load-deposit' task type.
Dec 13 2019, 11:20 AM

Dec 12 2019

douardda created D2441: Remove the creation of the 'load-deposit' task type.
Dec 12 2019, 5:39 PM
douardda closed D2436: Make --status option of 'swh scheduler task list' a click.Choice.
Dec 12 2019, 5:24 PM
douardda committed rDSCH4071d71784d9: Make --status option of 'swh scheduler task list' a click.Choice (authored by douardda).
Make --status option of 'swh scheduler task list' a click.Choice
Dec 12 2019, 5:24 PM
douardda created D2436: Make --status option of 'swh scheduler task list' a click.Choice.
Dec 12 2019, 12:29 PM
douardda created D2435: Add a caveats section in docker/README.md.
Dec 12 2019, 12:21 PM
douardda created D2434: Remove task_modules declarations for lister and loader conf files in docker.
Dec 12 2019, 12:21 PM
douardda updated the diff for D2433: Attempt to rationalize a bit workers config in docker.

rebase

Dec 12 2019, 12:20 PM
douardda committed rDENV519413bebbae: Fixup a few \t -> ws (authored by douardda).
Fixup a few \t -> ws
Dec 12 2019, 12:19 PM
douardda created D2433: Attempt to rationalize a bit workers config in docker.
Dec 12 2019, 12:17 PM
douardda accepted D2430: loader.cli: Add 2 subcommands, one to load, one to introspect.

Nice, thx

Dec 12 2019, 11:17 AM
douardda accepted D2371: tarball: Add support for tar.Z, tar.x, tar.lz files.
Dec 12 2019, 11:15 AM
douardda accepted D2431: docker: Fix nginx bound port on host.

sorry for the glitch

Dec 12 2019, 11:08 AM

Dec 11 2019

douardda added inline comments to D2424: Enable Sentry for CLIs, and Gunicorn and Celery workers..
Dec 11 2019, 2:06 PM
douardda added a comment to D2427: Make load-deposit and check-deposit URL argument absolute.

tests should pass ok however no test exists for the interaction with the
scheduler, so...

By adding the swh_scheduler fixture in one test (which adds a deposit), you could either:

  • check the check_task_id, load_task_id are set
  • introspect the swh_scheduler.search_task('check-deposit') tasks to ensure there is something [1]

[1] https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/cgit/tests/test_lister.py$53-55

Dec 11 2019, 11:10 AM
douardda added a task to D2427: Make load-deposit and check-deposit URL argument absolute: T2118: Deposit: End to End monitoring.
Dec 11 2019, 10:57 AM
douardda added a revision to T2118: Deposit: End to End monitoring: D2427: Make load-deposit and check-deposit URL argument absolute.
Dec 11 2019, 10:57 AM · Sprint 2019/12 (Monitor and Conquer)
douardda created D2427: Make load-deposit and check-deposit URL argument absolute.
Dec 11 2019, 10:56 AM
douardda moved T2118: Deposit: End to End monitoring from Backlog to in progress on the Sprint 2019/12 (Monitor and Conquer) board.
Dec 11 2019, 10:22 AM · Sprint 2019/12 (Monitor and Conquer)

Dec 10 2019

douardda updated the diff for D2421: docker/deposit: configure nginx so the WSGI mountpoint for /deposit is ok-ish.

add X-Forwarded-Host header and configure allowed_hosts as '*'

Dec 10 2019, 2:18 PM
douardda created D2421: docker/deposit: configure nginx so the WSGI mountpoint for /deposit is ok-ish.
Dec 10 2019, 1:16 PM
douardda committed rDENV22cb6ffb25c4: docker/compose: add rabbitmq_exporter service (authored by douardda).
docker/compose: add rabbitmq_exporter service
Dec 10 2019, 1:15 PM
douardda closed D2417: docker/compose: add rabbitmq_exporter service.
Dec 10 2019, 1:15 PM
douardda added a comment to T2124: Save Code Now: monitoring of admin infra.

prometheus_client looks quite low-level to me. We already have statsd for this kind of purpose, which does not imply having to deal with one more dedicated http server or so...

Dec 10 2019, 11:12 AM · Sprint 2019/12 (Monitor and Conquer)
douardda added a comment to T2134: loader: Implement uniform loading CLI.

Sorry I'm commenting this a bit late, but wouldn't it make more sense to have something like:

Dec 10 2019, 11:00 AM · Sprint 2019/12 (Monitor and Conquer)

Dec 9 2019

douardda committed rDENV3007fb37f47d: docker/compose: configure the statsd endpoint for most of the containers (authored by douardda).
docker/compose: configure the statsd endpoint for most of the containers
Dec 9 2019, 5:11 PM
douardda closed D2416: docker/compose: configure the statsd endpoint for most of the containers.
Dec 9 2019, 5:11 PM
douardda updated the diff for D2417: docker/compose: add rabbitmq_exporter service.

rebased

Dec 9 2019, 5:06 PM
douardda updated the diff for D2416: docker/compose: configure the statsd endpoint for most of the containers.

move statsd emv var declarations in an env/statsd.env file and use it on swh-web

Dec 9 2019, 5:06 PM
douardda added a comment to D2416: docker/compose: configure the statsd endpoint for most of the containers.

Great ! Could you also add that configuration to the swh-web service ? I am currently working on T2124 and I have added a couple of statsd metric to the webapp.

Dec 9 2019, 5:03 PM
douardda updated the diff for D2417: docker/compose: add rabbitmq_exporter service.

rebase

Dec 9 2019, 4:41 PM
douardda updated the diff for D2416: docker/compose: configure the statsd endpoint for most of the containers.

rebase

Dec 9 2019, 4:41 PM
douardda committed rDENVe4162838b388: docker/Dockefile: clean the $HOME/.cache/ at the end of the build (authored by douardda).
docker/Dockefile: clean the $HOME/.cache/ at the end of the build
Dec 9 2019, 4:40 PM