Page MenuHomeSoftware Heritage
Feed Advanced Search

Nov 21 2019

douardda committed rDSTO264cd33b77aa: Add a pre-commit-hooks.yaml config file (authored by douardda).
Add a pre-commit-hooks.yaml config file
Nov 21 2019, 1:26 PM
douardda closed D2312: Add a pre-commit-hooks.yaml config file.
Nov 21 2019, 1:26 PM
douardda updated the diff for D2312: Add a pre-commit-hooks.yaml config file.

remove useless mypy.ini entry

Nov 21 2019, 1:19 PM
douardda updated the diff for D2312: Add a pre-commit-hooks.yaml config file.

fix mypy.ini and rebase

Nov 21 2019, 11:24 AM
douardda committed rDSTOa97db9319dda: Fix swh-storage-add-dir (authored by douardda).
Fix swh-storage-add-dir
Nov 21 2019, 11:23 AM
douardda closed D2314: Fix swh-storage-add-dir.
Nov 21 2019, 11:23 AM
douardda committed rDSTOc7878084ac48: Remove utils/(dump|fix)_revisions scripts (authored by douardda).
Remove utils/(dump|fix)_revisions scripts
Nov 21 2019, 11:23 AM
douardda closed D2313: Remove utils/(dump|fix)_revisions scripts.
Nov 21 2019, 11:23 AM
douardda updated the summary of D2313: Remove utils/(dump|fix)_revisions scripts.
Nov 21 2019, 11:21 AM
douardda updated the diff for D2314: Fix swh-storage-add-dir.

rebased

Nov 21 2019, 11:21 AM
douardda updated the diff for D2313: Remove utils/(dump|fix)_revisions scripts.

rebased on origin/master

Nov 21 2019, 11:20 AM
douardda updated the diff for D2314: Fix swh-storage-add-dir.

rebased

Nov 21 2019, 11:16 AM
douardda updated the diff for D2313: Remove utils/(dump|fix)_revisions scripts.

rebase

Nov 21 2019, 11:15 AM
douardda updated the diff for D2312: Add a pre-commit-hooks.yaml config file.

rewrite the mypy hook as a language:system one

Nov 21 2019, 11:15 AM
douardda committed rDCORE1c84e9e12ff2: Add a mypy precommit hook (authored by douardda).
Add a mypy precommit hook
Nov 21 2019, 11:07 AM
douardda closed D2311: Add a mypy precommit hook.
Nov 21 2019, 11:07 AM
douardda updated the diff for D2311: Add a mypy precommit hook.

rebased

Nov 21 2019, 11:06 AM

Nov 20 2019

douardda updated the diff for D2311: Add a mypy precommit hook.

use '.[testing]' as additional dep

Nov 20 2019, 2:42 PM
douardda updated the diff for D2314: Fix swh-storage-add-dir.

rebased

Nov 20 2019, 12:32 PM
douardda updated the diff for D2313: Remove utils/(dump|fix)_revisions scripts.

rebased

Nov 20 2019, 12:32 PM
douardda updated the diff for D2312: Add a pre-commit-hooks.yaml config file.

Include fixes reported by pre-commit hooks in this diff

Nov 20 2019, 12:31 PM
douardda updated the diff for D2312: Add a pre-commit-hooks.yaml config file.

run mypy on the whole swh/ directory

Nov 20 2019, 12:28 PM
douardda updated the diff for D2311: Add a mypy precommit hook.

enforce running mypy on the whole swh package each time

Nov 20 2019, 12:24 PM
douardda added a comment to D2311: Add a mypy precommit hook.
In D2311#54287, @zack wrote:

seems to work just fine.
[...]

Found 3 errors in 1 file (checked 1 source file)

I don't think this is good enough, in the sense that in this test you've introduced a type error in the file that is also the only one modified here, whereas modifying one file might introduce type errors in *other* files.
What we want here is to always run mypy swh as a pre-commit test, no matter which files have been modified.

Nov 20 2019, 11:56 AM
douardda created D2314: Fix swh-storage-add-dir.
Nov 20 2019, 11:51 AM
douardda created D2313: Remove utils/(dump|fix)_revisions scripts.
Nov 20 2019, 11:51 AM
douardda created D2312: Add a pre-commit-hooks.yaml config file.
Nov 20 2019, 11:50 AM
douardda updated the diff for D2311: Add a mypy precommit hook.

add 'args: []' in mypy's pre-commit config and fix mypy.ini for missing stubs

Nov 20 2019, 10:12 AM
douardda added a comment to D2311: Add a mypy precommit hook.

In fact using:

- repo: https://github.com/pre-commit/mirrors-mypy
  rev: 'master'
  hooks:
  - id: mypy
    args: []
Nov 20 2019, 10:10 AM

Nov 19 2019

douardda created D2311: Add a mypy precommit hook.
Nov 19 2019, 4:53 PM
douardda accepted D2306: .mrconfig: Stop referencing old package loader implementations.
Nov 19 2019, 4:26 PM
douardda accepted D2307: req-swh*: Remove old package loader implementations references.
Nov 19 2019, 4:26 PM
douardda accepted D2308: Make sure TypeErrors can pass through the RPC layer.
Nov 19 2019, 3:15 PM
douardda accepted D2305: docker-dev: Use loader-package implems from swh.loader.core modules.
Nov 19 2019, 3:03 PM · Docker environment
douardda committed rCDFDc8591006d03b: Dockerfile: get rid of swh-loader-xxx for packages loaders (authored by douardda).
Dockerfile: get rid of swh-loader-xxx for packages loaders
Nov 19 2019, 2:23 PM
douardda closed D2304: Dockerfile: get rid of swh-loader-xxx for packages loaders.
Nov 19 2019, 2:23 PM
douardda committed rCDFD3b0c74bc80ba: Dockerfile: install cassandra-driver in a dedicated step (authored by douardda).
Dockerfile: install cassandra-driver in a dedicated step
Nov 19 2019, 2:23 PM
douardda closed D2303: Dockerfile: install cassandra-driver in a dedicated step.
Nov 19 2019, 2:23 PM
douardda committed rCDFD7b2e570a57da: Dockerfile: use --no-install-recommends for r-cran packages (authored by douardda).
Dockerfile: use --no-install-recommends for r-cran packages
Nov 19 2019, 2:23 PM
douardda closed D2302: Dockerfile: use --no-install-recommends for r-cran packages.
Nov 19 2019, 2:23 PM
douardda updated the diff for D2304: Dockerfile: get rid of swh-loader-xxx for packages loaders.

rebased

Nov 19 2019, 2:22 PM
douardda updated the diff for D2303: Dockerfile: install cassandra-driver in a dedicated step.

rebased

Nov 19 2019, 2:22 PM
douardda updated the diff for D2302: Dockerfile: use --no-install-recommends for r-cran packages.

rebase

Nov 19 2019, 2:21 PM
douardda closed D2301: Ensure BufferedLoader and UnbufferedLoader do flush their storage.

closed by 4399d6c7bc0b

Nov 19 2019, 2:20 PM
douardda reopened D2304: Dockerfile: get rid of swh-loader-xxx for packages loaders.

wrong ard diff --update...

Nov 19 2019, 2:18 PM
douardda committed rDLDBASE4399d6c7bc0b: Ensure BufferedLoader and UnbufferedLoader do flush their storage (authored by douardda).
Ensure BufferedLoader and UnbufferedLoader do flush their storage
Nov 19 2019, 2:16 PM
douardda closed D2304: Dockerfile: get rid of swh-loader-xxx for packages loaders.
Nov 19 2019, 2:16 PM
douardda updated the diff for D2304: Dockerfile: get rid of swh-loader-xxx for packages loaders.

rebase

Nov 19 2019, 2:16 PM
douardda accepted D2291: swh.scheduler.cli: Add `swh scheduler task-type register` cli.
Nov 19 2019, 1:48 PM
douardda updated the diff for D2304: Dockerfile: get rid of swh-loader-xxx for packages loaders.

mercurial should not be in this lot

Nov 19 2019, 1:44 PM
douardda created D2304: Dockerfile: get rid of swh-loader-xxx for packages loaders.
Nov 19 2019, 1:39 PM
douardda created D2303: Dockerfile: install cassandra-driver in a dedicated step.
Nov 19 2019, 1:39 PM
douardda created D2302: Dockerfile: use --no-install-recommends for r-cran packages.
Nov 19 2019, 1:39 PM
douardda created D2301: Ensure BufferedLoader and UnbufferedLoader do flush their storage.
Nov 19 2019, 1:23 PM
douardda closed T2094: KeyError: 'content:add' in swh.loader.core.loader as Resolved.

This has been fixed by cb42fea77070

Nov 19 2019, 11:26 AM · Git loader

Nov 18 2019

douardda committed rDLDBASEcb42fea77070: BufferedLoader: proxy storages may return empty dict for add_xxx methods (authored by douardda).
BufferedLoader: proxy storages may return empty dict for add_xxx methods
Nov 18 2019, 5:04 PM
douardda closed D2299: BufferedLoader: proxy storages may return empty dict for add_xxx methods.
Nov 18 2019, 5:04 PM
douardda created D2299: BufferedLoader: proxy storages may return empty dict for add_xxx methods.
Nov 18 2019, 4:37 PM
douardda triaged T2097: Merge swh-docker-dev into swh-environment as High priority.
Nov 18 2019, 3:29 PM · Docker environment, Development environment
douardda accepted D2294: worker: Reference the task-type registering step for loader/lister.
Nov 18 2019, 2:01 PM · Docker environment
douardda accepted D2292: loader.package: Register loader package tasks.
Nov 18 2019, 2:00 PM
douardda accepted D2293: lister.cli: Remove task type register cli.
Nov 18 2019, 1:59 PM
douardda requested changes to D2291: swh.scheduler.cli: Add `swh scheduler task-type register` cli.
Nov 18 2019, 1:58 PM
douardda accepted D2284: Rename scroll_token to page_token/next_page_token..
Nov 18 2019, 1:41 PM
douardda committed rDSTOea9aa4736fa7: and not only for an existing origin visit. (authored by douardda).
and not only for an existing origin visit.
Nov 18 2019, 1:38 PM
douardda closed D2295: db: check for existing snapshot when with_visit is True in origin_search().
Nov 18 2019, 1:38 PM
douardda updated the diff for D2295: db: check for existing snapshot when with_visit is True in origin_search().

remove sql migration file

Nov 18 2019, 1:33 PM
douardda added a comment to D2295: db: check for existing snapshot when with_visit is True in origin_search().

Apart the sql upgrade file which should not be here, looks good to me.

Nov 18 2019, 1:26 PM
douardda accepted D2266: RPCClient: check HTTP status code for errors also when streaming.
Nov 18 2019, 1:24 PM
douardda created D2295: db: check for existing snapshot when with_visit is True in origin_search().
Nov 18 2019, 12:28 PM

Nov 15 2019

douardda added a comment to T2093: Sending a POST to the /save/git/url API may return a 403.

This task is the result of this discussion https://octodon.social/@joeyh/103133965986721726

Nov 15 2019, 1:44 PM · Web app
douardda updated subscribers of T2093: Sending a POST to the /save/git/url API may return a 403.
Nov 15 2019, 1:42 PM · Web app
douardda triaged T2093: Sending a POST to the /save/git/url API may return a 403 as Normal priority.
Nov 15 2019, 1:39 PM · Web app

Nov 14 2019

douardda added a comment to D2276: Add get_storage_pipeline utility function for more readable configurations..

I'm not convinced this really is a good idea, it makes it difficult to document. Why not just replace get_xxx(cls, args): by get_xxx(cls, **args), then we can write this yaml as (with a little bit of manipulation on the yaml->arguments transformation):

storage:
  - filter
  - storage:
    - buffer
    - storage:
      - remote
      - url: http://swh-storage:5002/
Nov 14 2019, 3:12 PM
douardda added inline comments to D2266: RPCClient: check HTTP status code for errors also when streaming.
Nov 14 2019, 10:12 AM

Nov 13 2019

douardda added inline comments to D2266: RPCClient: check HTTP status code for errors also when streaming.
Nov 13 2019, 10:27 AM
douardda requested changes to D2266: RPCClient: check HTTP status code for errors also when streaming.

This needs one or 2 tests.

Nov 13 2019, 10:20 AM

Nov 12 2019

douardda added a comment to T2051: Fix loading process for deposits.

what's the current status of this task?

Nov 12 2019, 5:18 PM · SWORD deposit
douardda triaged T2078: Audit APIs for (in)consistencies as Normal priority.
Nov 12 2019, 10:15 AM

Nov 8 2019

douardda accepted D2241: Adapt tests mocks to work with swh.-journal v0.0.19.
Nov 8 2019, 4:46 PM
douardda committed rDCIDX01f1c385d0a1: tests: update/add copyright/license headers (authored by douardda).
tests: update/add copyright/license headers
Nov 8 2019, 4:24 PM
douardda committed rDCIDX5150eea140ed: tests: migrate storage tests to pytest (authored by douardda).
tests: migrate storage tests to pytest
Nov 8 2019, 4:24 PM
douardda closed D2228: tests: migrate indexer storage tests to pytest.
Nov 8 2019, 4:24 PM
douardda committed rDCIDXdb2a631da27f: proper pagination for IndexerStorage.origin_intrinsic_metadata_search_by_produc… (authored by douardda).
proper pagination for IndexerStorage.origin_intrinsic_metadata_search_by_produc…
Nov 8 2019, 4:24 PM
douardda closed D2222: proper pagination for IndexerStorage.origin_intrinsic_metadata_search_by_producer.
Nov 8 2019, 4:24 PM
douardda updated the summary of D2228: tests: migrate indexer storage tests to pytest.
Nov 8 2019, 3:21 PM
douardda updated the diff for D2228: tests: migrate indexer storage tests to pytest.

update/add headers in test files

Nov 8 2019, 2:14 PM
douardda added inline comments to D2228: tests: migrate indexer storage tests to pytest.
Nov 8 2019, 2:11 PM
douardda updated the diff for D2228: tests: migrate indexer storage tests to pytest.

rebase

Nov 8 2019, 2:09 PM
douardda updated the diff for D2222: proper pagination for IndexerStorage.origin_intrinsic_metadata_search_by_producer.

ok now I can get rid of the spurious if in list_origins_by_producer...

Nov 8 2019, 2:09 PM
douardda updated the diff for D2228: tests: migrate indexer storage tests to pytest.

rebase (hopefully)

Nov 8 2019, 1:58 PM
douardda updated the diff for D2222: proper pagination for IndexerStorage.origin_intrinsic_metadata_search_by_producer.

this version should address the points raised by vlorentz

Nov 8 2019, 1:58 PM
douardda requested changes to D2027: Add support for origin_intrinsic_metadata in the journal client..
Nov 8 2019, 11:16 AM
douardda accepted D2232: lister.cli: Clarify cli's configuration loading step.
Nov 8 2019, 10:57 AM
douardda added inline comments to D2228: tests: migrate indexer storage tests to pytest.
Nov 8 2019, 10:14 AM
douardda updated the diff for D2228: tests: migrate indexer storage tests to pytest.

Refactor the tests to prevent these ugly injected functions

Nov 8 2019, 10:14 AM

Nov 7 2019

douardda updated the diff for D2228: tests: migrate indexer storage tests to pytest.

rebase + better ci message + s/_w_/_with_/g

Nov 7 2019, 5:33 PM
douardda added inline comments to D2222: proper pagination for IndexerStorage.origin_intrinsic_metadata_search_by_producer.
Nov 7 2019, 5:31 PM
douardda updated the summary of D2222: proper pagination for IndexerStorage.origin_intrinsic_metadata_search_by_producer.
Nov 7 2019, 5:31 PM
douardda added inline comments to D2228: tests: migrate indexer storage tests to pytest.
Nov 7 2019, 4:46 PM