Page MenuHomeSoftware Heritage
Feed Advanced Search

Jan 27 2022

zack added a comment to T3887: Storing multiple authors in Revisions and Releases.
In T3887#77949, @olasd wrote:

Now that I've written it out loud, of course, Releases don't have extra_headers so the package loaders can't make use of this workaround/hack for now.

Jan 27 2022, 5:40 PM · SWORD deposit, Data Model, BZR loader
zack committed rMSLD4bf5d5d41819: check in recent presentations (authored by zack).
check in recent presentations
Jan 27 2022, 10:13 AM

Jan 25 2022

zack triaged T3885: Filter rows of size >32MB from dataset export as Normal priority.
Jan 25 2022, 1:32 PM · Datasets

Jan 10 2022

zack committed R183:6b876e2ac76b: add several entries about reproducibility, FOSS geography, and diversity (authored by zack).
add several entries about reproducibility, FOSS geography, and diversity
Jan 10 2022, 7:58 PM

Jan 4 2022

zack closed T3260: publish swh.dataset to pypi as Resolved.
Jan 4 2022, 1:42 PM · Continuous Integration, Datasets
zack changed the status of T3768: Read compression input from ORC instead of the edges file from Open to Work in Progress.
Jan 4 2022, 1:35 PM · Compressed graph service

Jan 3 2022

zack added a comment to T3822: Update the fundraising banner.

@marla.dasilva @anlambert: let's go for "Until Jan 30th" then. (I'll also ping you about this in the chat, just in case.)

Jan 3 2022, 3:43 PM · Unknown Object (Project)
zack triaged T3822: Update the fundraising banner as High priority.

Thanks Marla, I also planned to raise this.

Jan 3 2022, 11:13 AM · Unknown Object (Project)

Dec 16 2021

zack triaged T3811: archive.s.o: change Debian tooltip to include derivatives as Low priority.
Dec 16 2021, 10:40 AM · Web app
zack renamed T2400: Ingest current and historical Ubuntu releases from Ingest current and history Ubuntu releases to Ingest current and historical Ubuntu releases.
Dec 16 2021, 10:36 AM · System administration, Debian loader, Package Loader, Archive coverage

Dec 14 2021

zack raised the priority of T3161: graph service: add anti-DoS limit on the number of edges traversed from Normal to High.
Dec 14 2021, 1:31 PM · Compressed graph service

Dec 6 2021

zack accepted D4821: Add LLP compression to the WebGraph pipeline.

Just to be sure: test_pipeline() from test_cli.py is now run with all new passes as well, and as such it also testes the LLP step(s), correct?
It seems that way to me because test_pipeline() seems to be running all passes, but I'd like this to be double-checked before landing.

Dec 6 2021, 6:08 PM

Dec 4 2021

zack committed rMSLDfc9bffe30c07: check-in slides for tech presentation at #swh5years sponsors meeting (authored by zack).
check-in slides for tech presentation at #swh5years sponsors meeting
Dec 4 2021, 10:22 AM

Dec 1 2021

zack moved T2595: Add a default configuration based on graph size (eg: batch_size) from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 5:00 PM · Compressed graph service
zack changed the status of T2113: swh-graph: add support to optionally resolve ori PIDs to origin URLs from Wontfix to Resolved.
Dec 1 2021, 5:00 PM · Compressed graph service
zack moved T2112: make "swh graph map lookup" accept lists of identifiers from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 5:00 PM · Compressed graph service
zack moved T2096: CNAME for graph service: graph.internal.softwareheritage.org (?) from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 5:00 PM · Compressed graph service, System administration
zack moved T2083: provide systemd service file for swh-graph from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 5:00 PM · Compressed graph service
zack changed the status of T2056: fix swh-graph sphinx table of content from Invalid to Resolved.
Dec 1 2021, 5:00 PM · Documentation, Compressed graph service
zack moved T1933: bad invocation of o.s.graph.backend.Setup in docker doc from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 5:00 PM · Compressed graph service
zack moved T1898: swh-graph: refactor algo implementations to not forcibly memoize results from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 5:00 PM · Compressed graph service
zack moved T1888: graph API documentation: clarify the relationship between directory=backward and edges= from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 5:00 PM · Documentation, Compressed graph service
zack moved T1884: python bindings for compressed graph access from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 5:00 PM · Compressed graph service
zack moved T1937: nicer landing page for the swh-graph REST API from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 5:00 PM · Compressed graph service
zack changed the status of T1968: existing graph endpoints should not return 404 upon missing arguments from Invalid to Resolved.
Dec 1 2021, 5:00 PM · Easy hack, Compressed graph service
zack moved T1936: integrate swh-graph into the docker environment from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:38 PM · Docker environment, Compressed graph service
zack changed the status of T1930: swh-graph: ship swh-graph.jar in the docker container from Wontfix to Resolved.
Dec 1 2021, 4:38 PM · Compressed graph service
zack moved T1887: publish swh-graph documentation at docs.s.o from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:38 PM · Documentation, Compressed graph service
zack moved T1851: Integrate graph-compression git repo in swh-environment from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:38 PM · Compressed graph service
zack moved T1877: Add contextual info to compression pipeline from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T1878: Write documentation on compression process from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T1879: Write documentation on compression Docker env from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T1889: graph API: add endpoint to return the leaves of a subgraph from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T1886: graph API: add endpoint to return the adjacency list of a node from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T1901: refactor graph API to use integer IDs in the kernel and translate to/from SWH PIDs on outer layers from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T1904: build developer documentation for swh-graph from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Documentation, Compressed graph service
zack moved T1920: graph service: add tests for the python client from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T1938: swh-graph: NullPointerException upon (wrong) /walk from cnt to snp from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T1945: Return timings instead of simply logging them from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T1952: Log raw datapoint in graph benchmarks from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T2053: support graph export for the cassandra backend from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service, Storage manager
zack moved T2072: common configuration file for swh graph rpc-serve, compress, … from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T2077: add random walk endpoint with limited retries from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T2084: swh-graph: add /last endpoint variants to the REST API from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Compressed graph service
zack moved T2114: swh-graph API: add ?limit=N method variants to return first N results from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:37 PM · Easy hack, Compressed graph service
zack moved T2589: expose swh-graph API at archive.s.o/api/1/graph/ from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · System administration, Web app, Compressed graph service
zack moved T2900: Public graph/ API does not handle streaming results from endpoints from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · System administration, Compressed graph service, Web app
zack moved T1862: Implement new graph API specifications from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1902: Use in-memory bitmap to store node->types relations in graph API from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1903: Add graph service README files from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1915: Add support for origin nodes in graph service API from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1867: compress Merkle DAG and origin nodes together from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1921: swh-graph: add logging of endpoint timing from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1922: swh-graph optimization: bypass edge restriction checks when edges=* from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1939: Measure memory needs for a swh-graph Azure VM from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1941: Automatically generate mapping files after compressing graph from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1944: use a compact, binary format for node ids mapping files from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:36 PM · Compressed graph service
zack moved T1950: Reduce RAM usage for generating mapping files from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:35 PM · Compressed graph service
zack moved T2045: add support for reverse lookup from swh:1:ori:... PIDs to origin URLs from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:35 PM · Compressed graph service, Storage manager
zack moved T1951: Reduce RAM usage in graph API endpoints from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:35 PM · Compressed graph service
zack moved T1885: benchmark swh-graph use cases on the full graph from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:35 PM · Compressed graph service
zack moved T2054: CI: ImportMismatchError when running on swh-graph from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:35 PM · Compressed graph service, Continuous Integration
zack moved T2055: swh-graph CI hangs badly when py4j doesn't find needed files from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:35 PM · Continuous Integration, Compressed graph service
zack moved T1868: refresh compressed representation of the archive from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:35 PM · Compressed graph service
zack moved T2492: swh-graph: loading maps fail when available memory is too low: Cannot allocate memory from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:35 PM · Compressed graph service
zack moved T2530: Write a simple "quick start" for swh-graph from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:35 PM · Documentation, Compressed graph service
zack moved T2642: swh-graph: fix CI from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:34 PM · Compressed graph service
zack moved T2768: unbreak swh-graph CI from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:34 PM · Continuous Integration, Compressed graph service
zack moved T3624: Update swh-graph from 0.3.0 to 0.5.0 on granet from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:34 PM · Compressed graph service, System administration
zack moved T3564: Puppetize graph service and add icinga alert from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:34 PM · System administration, Compressed graph service, Puppet recipes
zack raised the priority of T2647: add LLP support to graph compression pipeline from Normal to High.
Dec 1 2021, 4:27 PM · Compressed graph service
zack moved T1967: REST server hangs when loading entire graph from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:26 PM · Compressed graph service
zack moved T1943: Publish swh-graph to PyPI from Backlog to Deployed on the Compressed graph service board.
Dec 1 2021, 4:26 PM · Compressed graph service
zack moved T3740: swh-graph: Translate node IDs on the Java side, not Python side from In progress to Implemented on the Compressed graph service board.
Dec 1 2021, 4:26 PM · Compressed graph service
zack raised the priority of T2400: Ingest current and historical Ubuntu releases from Normal to High.
Dec 1 2021, 12:39 PM · System administration, Debian loader, Package Loader, Archive coverage

Nov 29 2021

zack changed the visibility for F5047255: sched.tar.gz.
Nov 29 2021, 1:16 PM
zack added a comment to T3755: misleading 100% known summary in sunburst rendering.

I've tried replacing the content of foo.txt with something unknown to the archive (random garbage) and the sunburst rendering still shows 100.0%.
So it could also be a rounding error instead.
Either way, it is misleading and should be fixed.

Nov 29 2021, 1:14 PM · Code scanner
zack triaged T3755: misleading 100% known summary in sunburst rendering as Low priority.
Nov 29 2021, 1:10 PM · Code scanner
zack triaged T3754: scanning sunburst rendering fail with "ValueError: Empty data passed with indices specified." as Normal priority.
Nov 29 2021, 1:04 PM · Code scanner
zack created T3754: scanning sunburst rendering fail with "ValueError: Empty data passed with indices specified.".
Nov 29 2021, 1:04 PM · Code scanner

Nov 23 2021

zack changed the status of T2647: add LLP support to graph compression pipeline from Open to Work in Progress.
Nov 23 2021, 1:47 PM · Compressed graph service
zack changed the status of T3161: graph service: add anti-DoS limit on the number of edges traversed, a subtask of T2220: swh-graph in production, from Open to Work in Progress.
Nov 23 2021, 1:38 PM · Roadmap 2022, meta-task, Roadmap 2021, Compressed graph service
zack changed the status of T3161: graph service: add anti-DoS limit on the number of edges traversed from Open to Work in Progress.
Nov 23 2021, 1:38 PM · Compressed graph service
zack changed the status of T3739: swh-graph: Remove SWHID -> Node ID mapping, use MPH instead from Open to Work in Progress.
Nov 23 2021, 1:36 PM · Compressed graph service
zack changed the status of T3740: swh-graph: Translate node IDs on the Java side, not Python side from Open to Work in Progress.
Nov 23 2021, 1:36 PM · Compressed graph service
zack changed the status of T3740: swh-graph: Translate node IDs on the Java side, not Python side, a subtask of T3739: swh-graph: Remove SWHID -> Node ID mapping, use MPH instead, from Open to Work in Progress.
Nov 23 2021, 1:36 PM · Compressed graph service

Nov 18 2021

zack accepted D6656: docs/index: Group swh.* entries under a top level "API reference" entry.

<3

Nov 18 2021, 2:15 PM
zack triaged T3737: documentation sidebar: group swh.* module doc under a "API reference" heading as Low priority.
Nov 18 2021, 10:54 AM · Documentation

Nov 15 2021

zack closed T3729: stuck vault cooking tasks as Resolved.

the cooking has completed now

Nov 15 2021, 11:36 AM · Vault, System administration
zack triaged T3729: stuck vault cooking tasks as Normal priority.
Nov 15 2021, 10:50 AM · Vault, System administration

Nov 11 2021

zack committed rMSLDe26c195f491d: check in slides for SFScon 2021 talk (authored by zack).
check in slides for SFScon 2021 talk
Nov 11 2021, 4:02 PM
zack committed rMSLDf100c34864bf: status module: update total size figure (authored by zack).
status module: update total size figure
Nov 11 2021, 4:02 PM

Nov 9 2021

zack added a comment to T1538: Add "forge" now.

Is the request for regular pulling?
If yes, it should be added for visibility on the home page of the archive.
If no, it should be clear this is a one shot thing.

Nov 9 2021, 9:48 AM · Add Forge Now , Roadmap 2022, meta-task, Roadmap 2021

Nov 8 2021

zack updated the task description for T3717: Ingest opam instance https://coq.inria.fr/opam/released/.
Nov 8 2021, 2:41 PM · System administration, Archive coverage, Opam
zack added a project to T3717: Ingest opam instance https://coq.inria.fr/opam/released/: Archive coverage.
Nov 8 2021, 2:41 PM · System administration, Archive coverage, Opam

Nov 3 2021

zack added a comment to T3621: Create a production read-only objstorage.

Where is the documentation on how to access the new read-only object storage?
(hint hint :-))

Nov 3 2021, 6:00 PM · System administration

Nov 2 2021

zack added a reviewer for D6594: Add parameter to load a single graph direction in memory.: seirl.
Nov 2 2021, 1:54 PM

Oct 31 2021

zack added a comment to T2983: graph service: allow loading in memory only one direction of the graph.

Nope, if loaded with only one direction, traversals will only be possible in the loaded direction.
This will essentially be a trade-off setting for people who cannot (or doesn't want to) load both direction.
It is fine to fail (gracefully, with an error) traversals requested in a direction that corresponds to a non-loaded graph.

Oct 31 2021, 7:10 PM · Compressed graph service
zack added a comment to T3627: Consider dropping pull request references from the git loader ingestion.

Thanks for the summaries @olasd, both here and on list.
I've followed up on list.

Oct 31 2021, 4:11 PM · Git loader

Oct 28 2021

zack closed D6567: FAQ: point to developer setup + minor fixes.
Oct 28 2021, 10:40 AM