Page MenuHomeSoftware Heritage
Feed Advanced Search

Nov 2 2020

zack added projects to T2748: [webapp] Deploy tag v0.0.265 in production: Web app, System administration.
Nov 2 2020, 1:44 PM · System administration, Web app

Oct 31 2020

zack committed rDFUSE34eed7bc9072: arch diagram: uniform arrow label texts (authored by zack).
arch diagram: uniform arrow label texts
Oct 31 2020, 9:56 PM
zack committed rDFUSEf25936c929d0: design doc: document branch escaping and drop sharding todo (authored by zack).
design doc: document branch escaping and drop sharding todo
Oct 31 2020, 7:58 PM
zack committed rDFUSE8e99c53dc8a4: docs: add architecture diagram(s) to design document (authored by zack).
docs: add architecture diagram(s) to design document
Oct 31 2020, 7:58 PM
zack created P844 Command-Line Input.
Oct 31 2020, 5:50 PM

Oct 28 2020

zack created P842 Command-Line Input.
Oct 28 2020, 10:52 AM
zack created P841 Command-Line Input.
Oct 28 2020, 10:49 AM
zack created P840 Command-Line Input.
Oct 28 2020, 8:57 AM

Oct 24 2020

zack triaged T2731: scanner: strip the path passed as argument from output as Low priority.
Oct 24 2020, 5:01 PM · Easy hack, Code scanner
zack updated the task description for T2730: scanner: should output the root SWHID as well.
Oct 24 2020, 4:58 PM · Easy hack, Code scanner
zack updated the task description for T2730: scanner: should output the root SWHID as well.
Oct 24 2020, 4:58 PM · Easy hack, Code scanner
zack triaged T2730: scanner: should output the root SWHID as well as Normal priority.
Oct 24 2020, 4:58 PM · Easy hack, Code scanner
zack added a reviewer for D4345: fuse: add cache on directories entries: seirl.
Oct 24 2020, 1:38 PM
zack requested changes to D4345: fuse: add cache on directories entries.

A few consideration on the open design questions:

Oct 24 2020, 1:37 PM

Oct 23 2020

zack committed rMSLDff2fca728275: deposit: fix typo in slide title (authored by zack).
deposit: fix typo in slide title
Oct 23 2020, 12:15 PM
zack committed rMSLD4e771e01812d: deposit: add oneslide CCS complicance use case (authored by zack).
deposit: add oneslide CCS complicance use case
Oct 23 2020, 10:13 AM
zack committed rMSLDd921547da7ba: deposit module: update and add a submit+status slide (authored by zack).
deposit module: update and add a submit+status slide
Oct 23 2020, 9:30 AM
zack committed rMSLDa277c441402a: modules: add oneslide version of swhid.org content (authored by zack).
modules: add oneslide version of swhid.org content
Oct 23 2020, 8:52 AM

Oct 22 2020

zack committed rMSLD43a93c448dfe: Web UI module: add 1-slide version and update source file image (authored by zack).
Web UI module: add 1-slide version and update source file image
Oct 22 2020, 4:37 PM
zack committed rDSNIP3827da16bd71: sf-ls-projects: drop obsolete commented out code (authored by zack).
sf-ls-projects: drop obsolete commented out code
Oct 22 2020, 2:10 PM
zack triaged T2728: Web API: make /graph accessible via the VPN (archive.internal.s.o), even without auth token as Low priority.
Oct 22 2020, 1:35 PM · Web app
zack updated the task description for T735: SourceForge lister.
Oct 22 2020, 1:22 PM · Origin-SourceForge
zack added a comment to T735: SourceForge lister.

Here's a blueprint for implementing a SourceForge lister, based on an exchange with a SourceForge tech contact:

Oct 22 2020, 1:21 PM · Origin-SourceForge
zack committed rDSNIP422bfaa8f889: add prototype SourceForge lister (authored by zack).
add prototype SourceForge lister
Oct 22 2020, 1:12 PM
zack updated the task description for T735: SourceForge lister.
Oct 22 2020, 1:07 PM · Origin-SourceForge
zack updated the title for P832 SourceForge "tools" and VCS count from SourceForge "tools" count to SourceForge "tools" and VCS count.
Oct 22 2020, 12:27 PM
zack edited P832 SourceForge "tools" and VCS count.
Oct 22 2020, 12:27 PM
zack edited P832 SourceForge "tools" and VCS count.
Oct 22 2020, 12:26 PM
zack edited P832 SourceForge "tools" and VCS count.
Oct 22 2020, 12:25 PM
zack updated the title for P832 SourceForge "tools" and VCS count from Command-Line Input to SourceForge "tools" count.
Oct 22 2020, 12:25 PM
zack created P832 SourceForge "tools" and VCS count.
Oct 22 2020, 12:23 PM

Oct 21 2020

zack resigned from D4309: Add flat commit view in a history/ virtual dir.
Oct 21 2020, 3:23 PM
zack requested changes to D4309: Add flat commit view in a history/ virtual dir.
Oct 21 2020, 3:23 PM
zack added a reviewer for D4316: cache: add missing aiosqlite commit call: seirl.
Oct 21 2020, 10:30 AM

Oct 20 2020

zack renamed T2724: fuse: populate history/ incrementally to avoid temporary hangs on huge repos from Realistic benchmark to explore the Linux kernel history to history/ dir browsing is too slow on big repos like the Linux kernel.
Oct 20 2020, 6:23 PM · Software Heritage filesystem
zack raised the priority of T2725: Concurrent access result in ERROR:aiosqlite:returning exception database is locked from Normal to High.
Oct 20 2020, 6:22 PM · Software Heritage filesystem
zack lowered the priority of T2635: web client: add async API from Normal to Wishlist.

lowering priority, as it's not needed for swh-fuse for now

Oct 20 2020, 6:19 PM · Web client
zack edited reviewers for D4309: Add flat commit view in a history/ virtual dir, added: seirl; removed: zack.
Oct 20 2020, 12:10 PM
zack resigned from D4309: Add flat commit view in a history/ virtual dir.
Oct 20 2020, 12:09 PM
zack committed rDFUSE3409bd8a092f: rename main CLI command from "swh fuse" to "swh fs" (authored by zack).
rename main CLI command from "swh fuse" to "swh fs"
Oct 20 2020, 10:39 AM
zack closed D4314: rename main CLI command from "swh fuse" to "swh fs".
Oct 20 2020, 10:39 AM
zack created D4314: rename main CLI command from "swh fuse" to "swh fs".
Oct 20 2020, 10:26 AM

Oct 19 2020

zack moved T2716: FUSE cli.py: move module-level import to function-level to reduce startup time from In progress to Done on the Software Heritage filesystem board.
Oct 19 2020, 2:23 PM · Easy hack, Software Heritage filesystem
zack accepted D4304: cli: move module-level import to function-level to reduce startup time.
Oct 19 2020, 2:20 PM
zack accepted D4303: fuse: use debug option only when loglevel is debug or higher.
Oct 19 2020, 2:19 PM
zack triaged T2718: Web UI: add a profile page as Normal priority.
Oct 19 2020, 1:37 PM · Web app
zack added a project to T2717: Write an end-user documentation on how to use the authenticated stack: Web app.
Oct 19 2020, 1:33 PM · Web app, Documentation
zack added a comment to T2717: Write an end-user documentation on how to use the authenticated stack.

Agreed. In terms of where this doc go, I think it should just be a new section under https://archive.softwareheritage.org/api/ , where we already have other stuff very similar in nature. And we will be able to link to the relevant section of it via # anchors, as we already do for, e.g., rate limit.

Oct 19 2020, 1:33 PM · Web app, Documentation
zack accepted D4302: cli: config: add error handling and logging when parsing.

This is fine. Two things to do before merge though:

  • make sure the commit message is mainly about non-ignoring user configuration, because this is the big change here, the logging is a nice addition but is marginal
  • consider turning this warning logging.warning("No swh.fuse stanza found in global configuration") into just pass, because, again, it would be entirely legitimate for the user to have a global.yml without that stanza, so it's not even something to be warned about (and the user will know from other messages that their conf is not being used)
Oct 19 2020, 1:32 PM
zack changed the status of T2711: Web API URL setting in configuration file ignored from Open to Work in Progress.
Oct 19 2020, 1:19 PM · Software Heritage filesystem
zack requested changes to D4302: cli: config: add error handling and logging when parsing.
Oct 19 2020, 1:17 PM
zack moved T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653' from In progress to Done on the Software Heritage filesystem board.
Oct 19 2020, 12:30 PM · Software Heritage filesystem
zack renamed T2716: FUSE cli.py: move module-level import to function-level to reduce startup time from FUSE cli.py: to FUSE cli.py: move module-level import to function-level to reduce startup time.
Oct 19 2020, 12:24 PM · Easy hack, Software Heritage filesystem
zack triaged T2716: FUSE cli.py: move module-level import to function-level to reduce startup time as Wishlist priority.
Oct 19 2020, 12:21 PM · Easy hack, Software Heritage filesystem
zack accepted D4301: fuse: add error handling around HTTP errors (web API).
Oct 19 2020, 12:16 PM
zack requested changes to D4301: fuse: add error handling around HTTP errors (web API).
Oct 19 2020, 11:15 AM
zack requested changes to D4301: fuse: add error handling around HTTP errors (web API).

just a style matter: instead of "error during BLA: {err}", please use something like "cannot BLA: {err}", which is more relevant for the user

Oct 19 2020, 11:06 AM
zack moved T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s) from Backlog to Done on the Software Heritage filesystem board.
Oct 19 2020, 10:09 AM · Software Heritage filesystem
zack moved T2702: FUSE: add an unmount command from In progress to Done on the Software Heritage filesystem board.
Oct 19 2020, 9:55 AM · Software Heritage filesystem
zack committed rDFUSE5c11efe3ae3b: CLI: add umount command, as a mnemonic alias for fusermount -u (authored by zack).
CLI: add umount command, as a mnemonic alias for fusermount -u
Oct 19 2020, 9:55 AM
zack closed T2702: FUSE: add an unmount command as Resolved by committing rDFUSE5c11efe3ae3b: CLI: add umount command, as a mnemonic alias for fusermount -u.
Oct 19 2020, 9:55 AM · Software Heritage filesystem
zack closed D4292: CLI: add umount command, as a mnemonic alias for fusermount -u.
Oct 19 2020, 9:55 AM
zack updated the diff for D4292: CLI: add umount command, as a mnemonic alias for fusermount -u.

rebase

Oct 19 2020, 9:44 AM

Oct 18 2020

zack triaged T2711: Web API URL setting in configuration file ignored as High priority.
Oct 18 2020, 8:33 PM · Software Heritage filesystem
zack added a comment to T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653'.

In a different test, I've seen the Too many request error, but only after Ctrl-C, with an asyncio issue:

ERROR:root:Unknown name during lookup: 'HEAD'
   unique: 1866, error: -2 (No such file or directory), outsize: 16
^CERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-45' coro=<_session_loop() done, defined at /home/zack/.virtualenvs/swh/lib/python3.8/site-packages/_pyfuse3.py:28> exception=HTTPError('429 Client Error: Too Many Requests for url: https://archive.softwareheritage.org/api/1/content/sha1_git:642be81e2679d4524e736bafe6c4576360a9fd5c/raw/')>

This might be the key issue here, a missing (async) exception handler somewhere.

Oct 18 2020, 8:26 PM · Software Heritage filesystem
zack updated the task description for T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653'.
Oct 18 2020, 2:33 PM · Software Heritage filesystem
zack added a hashtag to Software Heritage filesystem: #swhfs.
Oct 18 2020, 2:31 PM
zack planned changes to D4296: doc: add tutorial for end users.
Oct 18 2020, 2:11 PM
zack updated the diff for D4296: doc: add tutorial for end users.
  • doc: add tutorial for end users
  • tutorial: add directory listing examples
Oct 18 2020, 2:10 PM
zack abandoned D4297: doc: add tutorial for end users.

submitted by mistake, should've been an update of D4296

Oct 18 2020, 2:10 PM
zack added a revision to T2676: FUSE: write tutorial doc: D4297: doc: add tutorial for end users.
Oct 18 2020, 2:10 PM · Documentation, Software Heritage filesystem
zack created D4297: doc: add tutorial for end users.
Oct 18 2020, 2:10 PM
zack triaged T2710: swh-fuse: fails with "'TypeError: Cannot merge a <class 'dict'> with a <class 'NoneType'>" when conffile is empty or commented out as Normal priority.
Oct 18 2020, 1:53 PM · Software Heritage filesystem
zack added a comment to T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653'.

Looks like a lot of unrelated error are hidden behind "unknown SWHID". For instance, using the Web API via the VPN with the VPN down, results in that error too, with the syscall hanging.
There seems to be two things to fix about this:

  • more appropriate error reporting (rate limit exceeded, can't connect to API server, etc.)
  • the syscall should not hang, but rather return the user space an appropriate failure (note indeed that these hangs are bad, of the kind you can't Ctrl-C out of)
Oct 18 2020, 1:42 PM · Software Heritage filesystem
zack claimed T2676: FUSE: write tutorial doc.
Oct 18 2020, 1:34 PM · Documentation, Software Heritage filesystem
zack claimed T2702: FUSE: add an unmount command.
Oct 18 2020, 1:34 PM · Software Heritage filesystem
zack assigned T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653' to haltode.
Oct 18 2020, 1:34 PM · Software Heritage filesystem
zack updated the task description for T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653'.
Oct 18 2020, 1:33 PM · Software Heritage filesystem
zack triaged T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653' as High priority.
Oct 18 2020, 1:33 PM · Software Heritage filesystem
zack committed rDFUSE0819d4781ace: doc: fix dangling CLI ref in design doc (authored by zack).
doc: fix dangling CLI ref in design doc
Oct 18 2020, 9:43 AM
zack planned changes to D4296: doc: add tutorial for end users.
Oct 18 2020, 9:42 AM
zack added a revision to T2676: FUSE: write tutorial doc: D4296: doc: add tutorial for end users.
Oct 18 2020, 9:42 AM · Documentation, Software Heritage filesystem
zack created D4296: doc: add tutorial for end users.
Oct 18 2020, 9:42 AM
zack changed the status of T2676: FUSE: write tutorial doc from Open to Work in Progress.
Oct 18 2020, 9:39 AM · Documentation, Software Heritage filesystem
zack committed rDFUSE1ad41c849765: doc: uniform naming around SwhFS (authored by zack).
doc: uniform naming around SwhFS
Oct 18 2020, 9:13 AM

Oct 16 2020

zack updated the diff for D4292: CLI: add umount command, as a mnemonic alias for fusermount -u.

join strings that fit on a single line

Oct 16 2020, 5:56 PM
zack changed the status of T2702: FUSE: add an unmount command from Open to Work in Progress.
Oct 16 2020, 5:54 PM · Software Heritage filesystem
zack added a revision to T2702: FUSE: add an unmount command: D4292: CLI: add umount command, as a mnemonic alias for fusermount -u.
Oct 16 2020, 5:54 PM · Software Heritage filesystem
zack created D4292: CLI: add umount command, as a mnemonic alias for fusermount -u.
Oct 16 2020, 5:54 PM
zack committed rDFUSEbb963b44d0f9: CLI: add . at the end of short description (authored by zack).
CLI: add . at the end of short description
Oct 16 2020, 5:40 PM
zack committed rDFUSEa327bcfdf805: CLI: add clean command to remove on-disk cache files (authored by zack).
CLI: add clean command to remove on-disk cache files
Oct 16 2020, 5:30 PM
zack closed T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s) as Resolved by committing rDFUSEa327bcfdf805: CLI: add clean command to remove on-disk cache files.
Oct 16 2020, 5:30 PM · Software Heritage filesystem
zack closed D4291: CLI: add clean command to remove on-disk cache files.
Oct 16 2020, 5:30 PM
zack added a revision to T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s): D4291: CLI: add clean command to remove on-disk cache files.
Oct 16 2020, 4:47 PM · Software Heritage filesystem
zack created D4291: CLI: add clean command to remove on-disk cache files.
Oct 16 2020, 4:47 PM
zack triaged T2708: enable FUSE debug option only when loglevel is debug or higher as Low priority.
Oct 16 2020, 2:16 PM · Software Heritage filesystem
zack created P824 Command-Line Input.
Oct 16 2020, 2:05 PM
zack committed rDFUSE490c0de038f3: configuration doc: convert tabs to spaces (authored by zack).
configuration doc: convert tabs to spaces
Oct 16 2020, 2:02 PM
zack closed D4285: doc: document configuration file location.
Oct 16 2020, 2:02 PM
zack committed rDFUSE61afd3a428d1: doc: document configuration file location (authored by zack).
doc: document configuration file location
Oct 16 2020, 2:02 PM