Page MenuHomeSoftware Heritage
Feed Advanced Search

Nov 4 2019

zack committed rDLDPYda4a61d33af4: typing: minimal changes to make a no-op mypy run pass (authored by zack).
typing: minimal changes to make a no-op mypy run pass
Nov 4 2019, 12:17 PM
zack closed D2212: typing: minimal changes to make a no-op mypy run pass.
Nov 4 2019, 12:17 PM
zack committed rDLDDEB14003757f8ed: typing: minimal changes to make a no-op mypy run pass (authored by zack).
typing: minimal changes to make a no-op mypy run pass
Nov 4 2019, 12:10 PM
zack closed D2209: typing: minimal changes to make a no-op mypy run pass.
Nov 4 2019, 12:10 PM
zack committed rDGRPHd211d38b66bc: binary maps: change type IDs on Python side, to be compatible with Java (authored by zack).
binary maps: change type IDs on Python side, to be compatible with Java
Nov 4 2019, 11:47 AM
zack closed T1944: use a compact, binary format for node ids mapping files as Resolved by committing rDGRPH7c40a7d2b722: switch Java map generation from CSV to binary format.
Nov 4 2019, 11:47 AM · Compressed graph service
zack committed rDGRPH14813086c3fd: cli.py: document configuration parameter and reorder args (authored by zack).
cli.py: document configuration parameter and reorder args
Nov 4 2019, 11:47 AM
zack committed rDGRPH7c40a7d2b722: switch Java map generation from CSV to binary format (authored by zack).
switch Java map generation from CSV to binary format
Nov 4 2019, 11:47 AM
zack closed T1944: use a compact, binary format for node ids mapping files, a subtask of T1950: Reduce RAM usage for generating mapping files, as Resolved.
Nov 4 2019, 11:47 AM · Compressed graph service
zack committed rDGRPHa2a2082b31d3: cli.py: update docstring doc about available compression steps (authored by zack).
cli.py: update docstring doc about available compression steps
Nov 4 2019, 11:47 AM
zack committed rDGRPH1845d7e82f59: remove cruft dir java/**/t/, committed by mistake (authored by zack).
remove cruft dir java/**/t/, committed by mistake
Nov 4 2019, 11:18 AM
zack committed rDGRPHe101b194cd0f: temporarily disable test_api_client.py due to T2055 (authored by zack).
temporarily disable test_api_client.py due to T2055
Nov 4 2019, 11:18 AM

Nov 3 2019

zack triaged T2055: swh-graph CI hangs badly when py4j doesn't find needed files as Unbreak Now! priority.
Nov 3 2019, 4:46 PM · Continuous Integration, Compressed graph service
zack committed rDGRPH677daca371fe: tox.ini: fix pytest ImportMismatchError (authored by zack).
tox.ini: fix pytest ImportMismatchError
Nov 3 2019, 4:25 PM
zack closed T2054: CI: ImportMismatchError when running on swh-graph as Resolved by committing rDGRPH677daca371fe: tox.ini: fix pytest ImportMismatchError.
Nov 3 2019, 4:25 PM · Compressed graph service, Continuous Integration
zack committed rDGRPH80fa555d655d: makefile: add convenience clean-java target (authored by zack).
makefile: add convenience clean-java target
Nov 3 2019, 4:25 PM
zack updated the task description for T2054: CI: ImportMismatchError when running on swh-graph.
Nov 3 2019, 4:12 PM · Compressed graph service, Continuous Integration
zack updated the task description for T2054: CI: ImportMismatchError when running on swh-graph.
Nov 3 2019, 4:07 PM · Compressed graph service, Continuous Integration
zack triaged T2054: CI: ImportMismatchError when running on swh-graph as High priority.
Nov 3 2019, 4:05 PM · Compressed graph service, Continuous Integration
zack committed rDGRPH8c77e0e9d69b: mypy.ini: ignore psutil, due to missing stubs (authored by zack).
mypy.ini: ignore psutil, due to missing stubs
Nov 3 2019, 3:51 PM
zack committed rDGRPHe72bdcf2baf5: test_cli.py: new test for CLI end-to-end compression (authored by zack).
test_cli.py: new test for CLI end-to-end compression
Nov 3 2019, 3:46 PM
zack committed rDGRPH29ae29bde9f3: java mapping Setup: make Usage message more standard-looking (authored by zack).
java mapping Setup: make Usage message more standard-looking
Nov 3 2019, 3:46 PM
zack committed rDGRPH545be725d34b: webgraph.py: autoatically generate mappings at the end of compression (authored by zack).
webgraph.py: autoatically generate mappings at the end of compression
Nov 3 2019, 3:46 PM
zack closed T1941: Automatically generate mapping files after compressing graph as Resolved by committing rDGRPH545be725d34b: webgraph.py: autoatically generate mappings at the end of compression.
Nov 3 2019, 3:46 PM · Compressed graph service
zack raised the priority of T1941: Automatically generate mapping files after compressing graph from Normal to High.
Nov 3 2019, 3:12 PM · Compressed graph service
zack raised the priority of T1944: use a compact, binary format for node ids mapping files from Normal to High.
Nov 3 2019, 3:11 PM · Compressed graph service
zack raised the priority of T1950: Reduce RAM usage for generating mapping files from Normal to High.
Nov 3 2019, 3:11 PM · Compressed graph service
zack closed T1930: swh-graph: ship swh-graph.jar in the docker container as Wontfix.

now that we have the swh graph compress CLI, we're moving away to using docker for automated compression, so this has become moot

Nov 3 2019, 3:10 PM · Compressed graph service

Nov 2 2019

zack committed rDGRPHe927f1c2e2a7: CLI: add one-stop shop compression "swh graph compress ...." (authored by zack).
CLI: add one-stop shop compression "swh graph compress ...."
Nov 2 2019, 11:12 PM
zack committed rDGRPHcc85a555111d: swh graph CLI: fix bogus "swh graph graph" nesting (authored by zack).
swh graph CLI: fix bogus "swh graph graph" nesting
Nov 2 2019, 11:12 PM
zack committed rDGRPH0ef506e4d1b3: java deps: bump fastutil dep to 8.3.0 (authored by zack).
java deps: bump fastutil dep to 8.3.0
Nov 2 2019, 11:12 PM

Nov 1 2019

zack committed rDWAPPSc3139a51630c: mypy: add django support (authored by zack).
mypy: add django support
Nov 1 2019, 4:29 PM
zack committed rDWAPPS6d3cee726d01: typing: initial mypy skeleton and ignores (authored by zack).
typing: initial mypy skeleton and ignores
Nov 1 2019, 3:58 PM
zack committed rDDOC3e00a17f62a4: disable type checking for swh-docs (authored by zack).
disable type checking for swh-docs
Nov 1 2019, 3:38 PM
zack planned changes to D2032: typing: minimal changes to make a no-op mypy run pass.
Nov 1 2019, 3:36 PM
zack updated the diff for D2032: typing: minimal changes to make a no-op mypy run pass.
  • typing: minimal changes to make a no-op mypy run pass
  • typing: convert leftover variable annotation to comments
  • init.py: switch to documented way of extending path
  • tox: add mypy environment
  • requirements-test.txt: add back deps from master
  • Makefile.local: sync typecheck target with global Makefile.python
  • mypy: ignore psycopg2
Nov 1 2019, 3:36 PM
zack created D2212: typing: minimal changes to make a no-op mypy run pass.
Nov 1 2019, 3:25 PM
zack updated the diff for D2211: typing: minimal changes to make a no-op mypy run pass.
  • MANIFEST.in: ship py.typed
Nov 1 2019, 3:23 PM
zack planned changes to D2032: typing: minimal changes to make a no-op mypy run pass.

removing this form the review queue for now, as it will need py3.7 in the CI to pass tests anyway

Nov 1 2019, 12:26 PM
zack created D2211: typing: minimal changes to make a no-op mypy run pass.
Nov 1 2019, 12:23 PM
zack committed rDMIRFOR3d83370ddba9: typing: minimal changes to make a no-op mypy run pass (authored by zack).
typing: minimal changes to make a no-op mypy run pass
Nov 1 2019, 12:18 PM
zack created D2210: typing: minimal changes to make a no-op mypy run pass.
Nov 1 2019, 12:10 PM
zack created D2209: typing: minimal changes to make a no-op mypy run pass.
Nov 1 2019, 12:05 PM

Oct 31 2019

zack committed rDGRPH8bedf808a687: server/app.py: make mypy pass again (authored by zack).
server/app.py: make mypy pass again
Oct 31 2019, 9:00 PM
zack committed rDGRPH84c3b8a7dac7: server/app.py: fix flake8 spacing (authored by zack).
server/app.py: fix flake8 spacing
Oct 31 2019, 8:42 PM
zack committed rDGRPH707c5141f9f4: java build: include LAW via Maven (authored by zack).
java build: include LAW via Maven
Oct 31 2019, 8:35 PM
zack triaged T2053: support graph export for the cassandra backend as Normal priority.
Oct 31 2019, 2:09 PM · Compressed graph service, Storage manager

Oct 30 2019

zack committed rDSTOCd8fa30bacfcc: storage.py: ignore typing of optional get_journal_writer import (authored by zack).
storage.py: ignore typing of optional get_journal_writer import
Oct 30 2019, 5:21 PM
zack committed rDSTOC490125912c65: mypy.ini: be less flaky w.r.t. the packages installed in tox (authored by zack).
mypy.ini: be less flaky w.r.t. the packages installed in tox
Oct 30 2019, 5:21 PM
zack committed rDSTOC7c92c3304593: tox: anticipate mypy run to just after flake8 (authored by zack).
tox: anticipate mypy run to just after flake8
Oct 30 2019, 5:21 PM
zack committed rDSTOC395e85deb570: typing: minimal changes to make a no-op mypy run pass (authored by zack).
typing: minimal changes to make a no-op mypy run pass
Oct 30 2019, 5:21 PM
zack committed rDSTOCef7d2efc3609: init.py: switch to documented way of extending path (authored by zack).
init.py: switch to documented way of extending path
Oct 30 2019, 5:21 PM
zack committed rDSTOC976aab39d118: mypy: ignore swh.journal to work-around dependency loop (authored by zack).
mypy: ignore swh.journal to work-around dependency loop
Oct 30 2019, 5:21 PM
zack committed rDSTOCbe3a025b52da: fix typos in docstrings and sample paths (authored by zack).
fix typos in docstrings and sample paths
Oct 30 2019, 5:21 PM
zack committed rDSTOCb2e8aa23936b: click "required" param wants bool, not int (authored by zack).
click "required" param wants bool, not int
Oct 30 2019, 5:21 PM

Oct 29 2019

zack committed rDSNIP0fc3a54b9a90: git2blobs: avoid re-writing objects over and over again (authored by zack).
git2blobs: avoid re-writing objects over and over again
Oct 29 2019, 5:34 PM
zack committed rDSNIP383a89931cd8: git2blobs: add support for tarring (+ deleting) object storage (authored by zack).
git2blobs: add support for tarring (+ deleting) object storage
Oct 29 2019, 5:34 PM
zack committed rDSNIP51714bb1234c: add GPL as default license, as one was missing (authored by zack).
add GPL as default license, as one was missing
Oct 29 2019, 5:02 PM
zack committed rDCIDX35296e8044a3: test_metadata typing: use type-specific mappings instead of cast (authored by zack).
test_metadata typing: use type-specific mappings instead of cast
Oct 29 2019, 4:40 PM
zack committed rDCIDXb8562b11eca1: typing: minimal changes to make a no-op mypy run pass (authored by zack).
typing: minimal changes to make a no-op mypy run pass
Oct 29 2019, 4:40 PM
zack committed rDCIDX36ec832a9643: storage/db.py: drop unused format arg regconfig from query (authored by zack).
storage/db.py: drop unused format arg regconfig from query
Oct 29 2019, 4:40 PM
zack closed D2144: typing: minimal changes to make a no-op mypy run pass.
Oct 29 2019, 4:40 PM
zack added a comment to D2144: typing: minimal changes to make a no-op mypy run pass.

Could you replace cast(GemspecMapping, MAPPINGS['GemspecMapping']) (and others) with an import of GemspecMapping from swh.indexer.metadata_dictionary.ruby?

Oct 29 2019, 3:03 PM
zack updated the diff for D2144: typing: minimal changes to make a no-op mypy run pass.
  • test_metadata typing: use type-specific mappings instead of cast
Oct 29 2019, 3:00 PM

Oct 28 2019

zack committed rDLS44bc4462e75d: CRAN lister: fix compute_package_url interpolation (authored by zack).
CRAN lister: fix compute_package_url interpolation
Oct 28 2019, 5:18 PM
zack committed rDLS6159faa2f5af: mypy: add typing annotations for novel lister abstractions (authored by zack).
mypy: add typing annotations for novel lister abstractions
Oct 28 2019, 5:18 PM
zack committed rDLS7dfd811e16de: CRAN lister: make shelling out decoding compatible with Python 3.5 (authored by zack).
CRAN lister: make shelling out decoding compatible with Python 3.5
Oct 28 2019, 5:18 PM
zack committed rDLS974f80f9663b: typing: minimal changes to make a no-op mypy run pass (authored by zack).
typing: minimal changes to make a no-op mypy run pass
Oct 28 2019, 5:18 PM
zack closed D2123: typing: minimal changes to make a no-op mypy run pass.
Oct 28 2019, 5:18 PM
zack updated the diff for D2123: typing: minimal changes to make a no-op mypy run pass.
  • CRAN lister: fix compute_package_url interpolation
Oct 28 2019, 3:50 PM
zack updated the diff for D2123: typing: minimal changes to make a no-op mypy run pass.
  • typing: minimal changes to make a no-op mypy run pass
  • CRAN lister: make shelling out decoding compatible with Python 3.5
  • mypy: add typing annotations for novel lister abstractions
  • CRAN lister: fix compute_package_url interpolation
Oct 28 2019, 3:41 PM

Oct 27 2019

zack committed rDSNIPa4bb8b7b852d: git ignore mypy cache dir (authored by zack).
git ignore mypy cache dir
Oct 27 2019, 6:08 PM
zack committed rDSNIP923c8f60ea69: git2filenames: new script to extract <sha1, filename> blob maps from git (authored by zack).
git2filenames: new script to extract <sha1, filename> blob maps from git
Oct 27 2019, 6:08 PM
zack committed rDSNIPe96958c40daa: git2blobs.py: add docstring (authored by zack).
git2blobs.py: add docstring
Oct 27 2019, 6:08 PM
zack committed rDSNIPc059c9b606dd: git2blobs: new script to extract blobs from git repos (authored by zack).
git2blobs: new script to extract blobs from git repos
Oct 27 2019, 2:26 PM

Oct 26 2019

zack added a comment to T1958: Performance tuning of zfs infrastructure.
In T1958#38009, @olasd wrote:

I think the bulk of the task here is done. We can open another task if we notice any specific pain point later.

Oct 26 2019, 1:10 PM · System administration

Oct 20 2019

zack committed rDMOD2e4558c0d2d5: test_cli.py: fill in valid snapshot ID (authored by zack).
test_cli.py: fill in valid snapshot ID
Oct 20 2019, 9:26 PM
zack committed rDMOD8e3ee395d3c7: test_cli.py: drop unused NoQA marker (authored by zack).
test_cli.py: drop unused NoQA marker
Oct 20 2019, 9:26 PM
zack committed rDMOD4a742055c299: swh identify -t snapshot: add support for symbolic refs (authored by zack).
swh identify -t snapshot: add support for symbolic refs
Oct 20 2019, 9:26 PM
zack closed D2081: swh identify -t snapshot: add support for symbolic refs.
Oct 20 2019, 9:26 PM
zack updated the diff for D2081: swh identify -t snapshot: add support for symbolic refs.
  • swh identify -t snapshot: add support for symbolic refs
  • test_cli.py: drop unused NoQA marker
  • test_cli.py: fill in valid snapshot ID
Oct 20 2019, 9:24 PM
zack added a comment to D2078: Add support for symbolic references.
In D2078#49823, @olasd wrote:
In D2078#48207, @zack wrote:

Then I've dumped the branch list just before it's hashed (with P542 in swh.model.identifiers) and what I get is this:

alias HEAD 726566732f68656164732f6d6173746572
revision refs/heads/master 9768d0b576dbaaecd80abedad6dfd0d72f1476da
revision refs/remotes/origin/master c3c588713233609f5bbbb2d9e7f3fb4a660f3f72

where HEAD is an alias, but it's pointing to an object ID (this is on the sample git repo used for tests on swh-load-git, but I obtain similar results on any other repo I've tried).

What am I missing here?

That's not an object id, that's a branch name (refs/heads/master, in ASCII).

Oct 20 2019, 9:23 PM
zack committed rDGRPH5230e55c0312: git2graph: fix snapshot ID computation, now compatible with swh identify (authored by zack).
git2graph: fix snapshot ID computation, now compatible with swh identify
Oct 20 2019, 9:21 PM
zack committed rDGRPHd93aeaf2a085: git2graph: remove left over, comment out debug prints (authored by zack).
git2graph: remove left over, comment out debug prints
Oct 20 2019, 9:21 PM
zack committed rDGRPH0c9a71a667d7: ParallelBFS benchmark: add support to optionally use transposed graph (authored by zack).
ParallelBFS benchmark: add support to optionally use transposed graph
Oct 20 2019, 2:15 PM
zack committed rDGRPH132e528ee737: benchmark.ParallelBFS: new class to time parallel BFS on the full graph (authored by zack).
benchmark.ParallelBFS: new class to time parallel BFS on the full graph
Oct 20 2019, 2:02 PM
zack committed rDGRPH7fcf2f1d0a49: Graph: add getBVGraph getter for underlying BVgraph(s) (authored by zack).
Graph: add getBVGraph getter for underlying BVgraph(s)
Oct 20 2019, 2:02 PM
zack updated the diff for D2081: swh identify -t snapshot: add support for symbolic refs.
  • test_cli.py: drop unused NoQA marker
  • test_cli.py: fill in valid snapshot ID
Oct 20 2019, 9:39 AM
zack updated the diff for D2144: typing: minimal changes to make a no-op mypy run pass.
  • storage/db.py: drop unused format arg regconfig from query
Oct 20 2019, 9:15 AM
zack added a reviewer for D2032: typing: minimal changes to make a no-op mypy run pass: ardumont.
Oct 20 2019, 9:09 AM
zack added a reviewer for D2144: typing: minimal changes to make a no-op mypy run pass: vlorentz.
Oct 20 2019, 9:08 AM
zack retitled D2144: typing: minimal changes to make a no-op mypy run pass from [WIP] typing: minimal changes to make a no-op mypy run pass to typing: minimal changes to make a no-op mypy run pass.
Oct 20 2019, 9:08 AM
zack updated the diff for D2144: typing: minimal changes to make a no-op mypy run pass.

Updating D2144: [WIP] typing: minimal changes to make a no-op mypy run pass

Oct 20 2019, 9:08 AM

Oct 19 2019

zack triaged T2045: add support for reverse lookup from swh:1:ori:... PIDs to origin URLs as Normal priority.
Oct 19 2019, 2:45 PM · Compressed graph service, Storage manager

Oct 18 2019

zack updated the diff for D2123: typing: minimal changes to make a no-op mypy run pass.
  • CRAN lister: fix compute_package_url interpolation
Oct 18 2019, 5:40 PM
zack updated the diff for D2123: typing: minimal changes to make a no-op mypy run pass.
  • typing: minimal changes to make a no-op mypy run pass
  • CRAN lister: make shelling out decoding compatible with Python 3.5
  • mypy: add typing annotations for novel lister abstractions
Oct 18 2019, 5:33 PM

Oct 15 2019

zack planned changes to D2144: typing: minimal changes to make a no-op mypy run pass.

doesn't actually pass mypy yet, will update

Oct 15 2019, 3:59 PM
zack created D2144: typing: minimal changes to make a no-op mypy run pass.
Oct 15 2019, 3:59 PM

Oct 11 2019

zack updated the diff for D2123: typing: minimal changes to make a no-op mypy run pass.
  • typing: minimal changes to make a no-op mypy run pass
  • CRAN lister: make shelling out decoding compatible with Python 3.5
Oct 11 2019, 3:28 PM
zack updated the diff for D2123: typing: minimal changes to make a no-op mypy run pass.

updating D2123

Oct 11 2019, 3:15 PM