Page MenuHomeSoftware Heritage
Feed Advanced Search

Oct 19 2020

haltode changed the status of T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653' from Open to Work in Progress.
Oct 19 2020, 11:03 AM · Software Heritage filesystem
haltode added a revision to T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653': D4301: fuse: add error handling around HTTP errors (web API).
Oct 19 2020, 11:01 AM · Software Heritage filesystem
zack moved T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s) from Backlog to Done on the Software Heritage filesystem board.
Oct 19 2020, 10:09 AM · Software Heritage filesystem
zack moved T2702: FUSE: add an unmount command from In progress to Done on the Software Heritage filesystem board.
Oct 19 2020, 9:55 AM · Software Heritage filesystem
zack closed T2702: FUSE: add an unmount command as Resolved by committing rDFUSE5c11efe3ae3b: CLI: add umount command, as a mnemonic alias for fusermount -u.
Oct 19 2020, 9:55 AM · Software Heritage filesystem
haltode added a comment to T2710: swh-fuse: fails with "'TypeError: Cannot merge a <class 'dict'> with a <class 'NoneType'>" when conffile is empty or commented out.

The configuration parsing/merging is currently done using swh.core.config and should be changed with T1410.

Oct 19 2020, 9:46 AM · Software Heritage filesystem

Oct 18 2020

zack triaged T2711: Web API URL setting in configuration file ignored as High priority.
Oct 18 2020, 8:33 PM · Software Heritage filesystem
zack added a comment to T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653'.

In a different test, I've seen the Too many request error, but only after Ctrl-C, with an asyncio issue:

ERROR:root:Unknown name during lookup: 'HEAD'
   unique: 1866, error: -2 (No such file or directory), outsize: 16
^CERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-45' coro=<_session_loop() done, defined at /home/zack/.virtualenvs/swh/lib/python3.8/site-packages/_pyfuse3.py:28> exception=HTTPError('429 Client Error: Too Many Requests for url: https://archive.softwareheritage.org/api/1/content/sha1_git:642be81e2679d4524e736bafe6c4576360a9fd5c/raw/')>

This might be the key issue here, a missing (async) exception handler somewhere.

Oct 18 2020, 8:26 PM · Software Heritage filesystem
zack updated the task description for T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653'.
Oct 18 2020, 2:33 PM · Software Heritage filesystem
zack added a hashtag to Software Heritage filesystem: #swhfs.
Oct 18 2020, 2:31 PM
zack added a revision to T2676: FUSE: write tutorial doc: D4297: doc: add tutorial for end users.
Oct 18 2020, 2:10 PM · Documentation, Software Heritage filesystem
zack triaged T2710: swh-fuse: fails with "'TypeError: Cannot merge a <class 'dict'> with a <class 'NoneType'>" when conffile is empty or commented out as Normal priority.
Oct 18 2020, 1:53 PM · Software Heritage filesystem
zack added a comment to T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653'.

Looks like a lot of unrelated error are hidden behind "unknown SWHID". For instance, using the Web API via the VPN with the VPN down, results in that error too, with the syscall hanging.
There seems to be two things to fix about this:

  • more appropriate error reporting (rate limit exceeded, can't connect to API server, etc.)
  • the syscall should not hang, but rather return the user space an appropriate failure (note indeed that these hangs are bad, of the kind you can't Ctrl-C out of)
Oct 18 2020, 1:42 PM · Software Heritage filesystem
zack claimed T2676: FUSE: write tutorial doc.
Oct 18 2020, 1:34 PM · Documentation, Software Heritage filesystem
zack claimed T2702: FUSE: add an unmount command.
Oct 18 2020, 1:34 PM · Software Heritage filesystem
zack assigned T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653' to haltode.
Oct 18 2020, 1:34 PM · Software Heritage filesystem
zack updated the task description for T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653'.
Oct 18 2020, 1:33 PM · Software Heritage filesystem
zack triaged T2709: hangs with: ERROR:root:Unknown SWHID: 'swh:1:dir:be2222d58268b1ba08e612ed57c21c703653e653' as High priority.
Oct 18 2020, 1:33 PM · Software Heritage filesystem
zack added a revision to T2676: FUSE: write tutorial doc: D4296: doc: add tutorial for end users.
Oct 18 2020, 9:42 AM · Documentation, Software Heritage filesystem
zack changed the status of T2676: FUSE: write tutorial doc from Open to Work in Progress.
Oct 18 2020, 9:39 AM · Documentation, Software Heritage filesystem

Oct 16 2020

zack changed the status of T2702: FUSE: add an unmount command from Open to Work in Progress.
Oct 16 2020, 5:54 PM · Software Heritage filesystem
zack added a revision to T2702: FUSE: add an unmount command: D4292: CLI: add umount command, as a mnemonic alias for fusermount -u.
Oct 16 2020, 5:54 PM · Software Heritage filesystem
zack closed T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s) as Resolved by committing rDFUSEa327bcfdf805: CLI: add clean command to remove on-disk cache files.
Oct 16 2020, 5:30 PM · Software Heritage filesystem
zack added a revision to T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s): D4291: CLI: add clean command to remove on-disk cache files.
Oct 16 2020, 4:47 PM · Software Heritage filesystem
haltode moved T2699: FUSE: mount fails silently when --foreground is not specified from In progress to Done on the Software Heritage filesystem board.
Oct 16 2020, 3:34 PM · Software Heritage filesystem
haltode closed T2699: FUSE: mount fails silently when --foreground is not specified as Resolved by committing rDFUSE6cf27dddec08: cli: fix daemon working directory.
Oct 16 2020, 3:31 PM · Software Heritage filesystem
haltode added a revision to T2699: FUSE: mount fails silently when --foreground is not specified: D4289: cli: fix daemon working directory.
Oct 16 2020, 3:27 PM · Software Heritage filesystem
zack triaged T2708: enable FUSE debug option only when loglevel is debug or higher as Low priority.
Oct 16 2020, 2:16 PM · Software Heritage filesystem
haltode added a watcher for Software Heritage filesystem: haltode.
Oct 16 2020, 1:17 PM
haltode changed the status of T2699: FUSE: mount fails silently when --foreground is not specified from Open to Work in Progress.
Oct 16 2020, 11:31 AM · Software Heritage filesystem
haltode moved T2654: modprobe fuse on the CI build machine from Backlog to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:30 AM · Software Heritage filesystem, System administration
haltode moved T2673: FUSE: document configuration file location and syntax from Backlog to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:30 AM · Documentation, Software Heritage filesystem
haltode moved T2663: FUSE: add support for revision objects from Backlog to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:29 AM · Software Heritage filesystem
haltode moved T2689: swh-fuse tests are failing on the DENV/tests jenkins job from Backlog to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:29 AM · Software Heritage filesystem, System administration
haltode moved T2665: FUSE: add support for snapshot objects from Backlog to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:29 AM · Software Heritage filesystem
haltode moved T2664: FUSE: add support for release objects from Backlog to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:29 AM · Software Heritage filesystem
haltode moved T2700: Lazily load symlinks from Backlog to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:29 AM · Software Heritage filesystem
haltode moved T2698: FUSE: mount fail when cache directory is missing from Backlog to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:29 AM · Software Heritage filesystem
haltode moved T2678: Have a default location for the configuration file from In progress to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:29 AM · Software Heritage filesystem
haltode moved T2705: FUSE CLI: move --config-file option up to "swh fuse" from In progress to Done on the Software Heritage filesystem board.
Oct 16 2020, 11:29 AM · Software Heritage filesystem
haltode closed T2678: Have a default location for the configuration file as Resolved by committing rDFUSEbbcd975f8c8e: cli: move configuration settings up to "swh fuse".
Oct 16 2020, 11:25 AM · Software Heritage filesystem
haltode closed T2705: FUSE CLI: move --config-file option up to "swh fuse", a subtask of T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s), as Resolved.
Oct 16 2020, 11:25 AM · Software Heritage filesystem
haltode closed T2705: FUSE CLI: move --config-file option up to "swh fuse" as Resolved by committing rDFUSEbbcd975f8c8e: cli: move configuration settings up to "swh fuse".
Oct 16 2020, 11:25 AM · Software Heritage filesystem
haltode added a revision to T2678: Have a default location for the configuration file: D4282: cli: move configuration settings up to "swh fuse".
Oct 16 2020, 10:42 AM · Software Heritage filesystem
haltode added a revision to T2705: FUSE CLI: move --config-file option up to "swh fuse": D4282: cli: move configuration settings up to "swh fuse".
Oct 16 2020, 10:42 AM · Software Heritage filesystem
haltode changed the status of T2678: Have a default location for the configuration file from Open to Work in Progress.
Oct 16 2020, 10:35 AM · Software Heritage filesystem
haltode changed the status of T2705: FUSE CLI: move --config-file option up to "swh fuse" from Open to Work in Progress.
Oct 16 2020, 10:34 AM · Software Heritage filesystem
haltode changed the status of T2705: FUSE CLI: move --config-file option up to "swh fuse", a subtask of T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s), from Open to Work in Progress.
Oct 16 2020, 10:34 AM · Software Heritage filesystem
haltode closed T2665: FUSE: add support for snapshot objects as Resolved by committing rDFUSEd3a4668ae819: fuse: add support for snapshot artifacts.
Oct 16 2020, 9:27 AM · Software Heritage filesystem
haltode closed T2665: FUSE: add support for snapshot objects, a subtask of T1926: FUSE filesystem to navigate the archive, as Resolved.
Oct 16 2020, 9:27 AM · Software Heritage filesystem
zack raised the priority of T2699: FUSE: mount fails silently when --foreground is not specified from Normal to High.
Oct 16 2020, 9:02 AM · Software Heritage filesystem
zack changed the status of T2697: FUSE: add logging mechanism from Open to Work in Progress.
Oct 16 2020, 9:02 AM · Software Heritage filesystem

Oct 15 2020

haltode added a revision to T2665: FUSE: add support for snapshot objects: D4274: fuse: add support for snapshot artifacts.
Oct 15 2020, 5:32 PM · Software Heritage filesystem
haltode closed T2700: Lazily load symlinks as Resolved.

Fixed in 259757e6fc0a.

Oct 15 2020, 3:45 PM · Software Heritage filesystem
zack triaged T2705: FUSE CLI: move --config-file option up to "swh fuse" as Normal priority.
Oct 15 2020, 11:06 AM · Software Heritage filesystem
haltode added a revision to T2697: FUSE: add logging mechanism: D4265: cli: redirect logging to user syslog when daemonized.
Oct 15 2020, 10:32 AM · Software Heritage filesystem
zack renamed T2697: FUSE: add logging mechanism from Output log to user journal when started as a daemon to FUSE: add logging mechanism.
Oct 15 2020, 9:19 AM · Software Heritage filesystem

Oct 14 2020

zack added a comment to T2702: FUSE: add an unmount command.

Also: what are the cons of adding a umount command with just wraps fusermount -u?

Oct 14 2020, 1:04 PM · Software Heritage filesystem
zack added a comment to T2702: FUSE: add an unmount command.

We can't expect all users to read the tutorial, that just doesn't happen in the real world :-)
If we don't add an explicit umount command, we should at least document how to unmount also in the doc of the mount command

Oct 14 2020, 12:41 PM · Software Heritage filesystem
haltode added a comment to T2702: FUSE: add an unmount command.

I think we should add a full example of using (+ unmounting) the FUSE VFS in the tutorial doc (T2676), and not an extra umount command.

Oct 14 2020, 12:02 PM · Software Heritage filesystem
zack added a comment to T2702: FUSE: add an unmount command.

To unmount you can use fusermount -u /mntdir

Oct 14 2020, 11:57 AM · Software Heritage filesystem
ardumont added a comment to T2702: FUSE: add an unmount command.

as @haltode suggests, that's how i unmount my sshfs mount point for example.

Oct 14 2020, 11:57 AM · Software Heritage filesystem
haltode added a comment to T2702: FUSE: add an unmount command.

To unmount you can use fusermount -u /mntdir

Oct 14 2020, 11:54 AM · Software Heritage filesystem
zack triaged T2702: FUSE: add an unmount command as Low priority.
Oct 14 2020, 11:53 AM · Software Heritage filesystem
haltode closed T2698: FUSE: mount fail when cache directory is missing as Resolved by committing rDFUSE51dddc366318: cache: make sure the cache directory exists.
Oct 14 2020, 9:33 AM · Software Heritage filesystem
haltode added a revision to T2698: FUSE: mount fail when cache directory is missing: D4253: cache: make sure the cache directory exists.
Oct 14 2020, 9:21 AM · Software Heritage filesystem

Oct 13 2020

haltode closed T2664: FUSE: add support for release objects as Resolved by committing rDFUSE7d59abcc8d07: fuse: add support for release artifacts.
Oct 13 2020, 5:19 PM · Software Heritage filesystem
haltode closed T2664: FUSE: add support for release objects, a subtask of T1926: FUSE filesystem to navigate the archive, as Resolved.
Oct 13 2020, 5:19 PM · Software Heritage filesystem
haltode added a revision to T2664: FUSE: add support for release objects: D4246: fuse: add support for release artifacts.
Oct 13 2020, 3:19 PM · Software Heritage filesystem
haltode triaged T2700: Lazily load symlinks as Normal priority.
Oct 13 2020, 2:50 PM · Software Heritage filesystem
zack triaged T2699: FUSE: mount fails silently when --foreground is not specified as Normal priority.
Oct 13 2020, 2:34 PM · Software Heritage filesystem
zack triaged T2698: FUSE: mount fail when cache directory is missing as High priority.
Oct 13 2020, 2:33 PM · Software Heritage filesystem
haltode triaged T2697: FUSE: add logging mechanism as Normal priority.
Oct 13 2020, 2:25 PM · Software Heritage filesystem
haltode added a comment to T2695: Cache directory entries to make readdir/lookup more efficient.

From the design doc:

Oct 13 2020, 12:56 PM · Software Heritage filesystem
seirl added a comment to T2695: Cache directory entries to make readdir/lookup more efficient.

lookup() should ideally be O(1).

Oct 13 2020, 12:13 PM · Software Heritage filesystem
haltode triaged T2695: Cache directory entries to make readdir/lookup more efficient as Normal priority.
Oct 13 2020, 12:13 PM · Software Heritage filesystem
haltode created T2695: Cache directory entries to make readdir/lookup more efficient.
Oct 13 2020, 12:12 PM · Software Heritage filesystem
zack triaged T2694: FUSE: add sharding support for top-level dirs as Low priority.
Oct 13 2020, 10:45 AM · Software Heritage filesystem

Oct 12 2020

olasd closed T2689: swh-fuse tests are failing on the DENV/tests jenkins job as Resolved.
[...]
17:20:25 swh/fuse/tests/test_revision.py ..                                       [100%]
17:20:25 
17:20:25 ============================== 6 passed in 1.79s ===============================
Oct 12 2020, 5:21 PM · Software Heritage filesystem, System administration
olasd added a comment to T2689: swh-fuse tests are failing on the DENV/tests jenkins job.

Job 188 indeed failed. Let's see if 189 is the right one!

Oct 12 2020, 5:12 PM · Software Heritage filesystem, System administration
olasd added a comment to T2689: swh-fuse tests are failing on the DENV/tests jenkins job.

FWIW, this is a bit trickier than the resolution for T2654 as this job uses docker via the plain "jenkins clouds" integration.

Oct 12 2020, 4:35 PM · Software Heritage filesystem, System administration
vsellier renamed T2689: swh-fuse tests are failing on the DENV/tests jenkins job from swh-fuse tests are failing on the DENV/tests jobs to swh-fuse tests are failing on the DENV/tests jenkins job.
Oct 12 2020, 4:07 PM · Software Heritage filesystem, System administration
vsellier triaged T2689: swh-fuse tests are failing on the DENV/tests jenkins job as High priority.
Oct 12 2020, 4:06 PM · Software Heritage filesystem, System administration

Oct 9 2020

zack assigned T2678: Have a default location for the configuration file to haltode.

While we wait for a proper solution, I think there is nothing stopping swh fuse from loading a configuration file by default if it exists and -C is not passed on the CLI.
We should just move -C up one level (from swh fuse mount to swh fuse) and do something similar to what swh-scanner does.
The location of the file might change in the future as a consequence of T1410, but meh, it's not a big deal.
We can stay on the safe side for now and only support ~/.config/swh/fuse/fuse.yml. And we will worry about merging with a global config file later.

Oct 9 2020, 6:24 PM · Software Heritage filesystem
haltode closed T2663: FUSE: add support for revision objects, a subtask of T1926: FUSE filesystem to navigate the archive, as Resolved.
Oct 9 2020, 3:38 PM · Software Heritage filesystem
haltode closed T2663: FUSE: add support for revision objects as Resolved by committing rDFUSE6fb734ddef3b: fuse: add support for revision artifacts.
Oct 9 2020, 3:38 PM · Software Heritage filesystem
haltode closed T2663: FUSE: add support for revision objects, a subtask of T2659: FUSE: add flat commit view to ease batch commit workloads, as Resolved.
Oct 9 2020, 3:38 PM · Software Heritage filesystem
haltode closed T2673: FUSE: document configuration file location and syntax as Resolved by committing rDFUSEd759b9b3b751: docs: add configuration.rst.
Oct 9 2020, 2:18 PM · Documentation, Software Heritage filesystem
haltode triaged T2678: Have a default location for the configuration file as Normal priority.
Oct 9 2020, 2:14 PM · Software Heritage filesystem
haltode added a revision to T2673: FUSE: document configuration file location and syntax: D4218: docs: add configuration.rst.
Oct 9 2020, 1:46 PM · Documentation, Software Heritage filesystem

Oct 8 2020

zack triaged T2676: FUSE: write tutorial doc as Low priority.
Oct 8 2020, 8:21 PM · Documentation, Software Heritage filesystem
haltode added a revision to T1926: FUSE filesystem to navigate the archive: D4200: Add support for revision artifacts.
Oct 8 2020, 2:26 PM · Software Heritage filesystem
haltode added a revision to T2663: FUSE: add support for revision objects: D4200: Add support for revision artifacts.
Oct 8 2020, 2:08 PM · Software Heritage filesystem
zack triaged T2673: FUSE: document configuration file location and syntax as Normal priority.
Oct 8 2020, 11:58 AM · Documentation, Software Heritage filesystem

Oct 7 2020

zack triaged T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s) as Low priority.
Oct 7 2020, 8:06 PM · Software Heritage filesystem

Oct 6 2020

zack closed T2654: modprobe fuse on the CI build machine, a subtask of T1926: FUSE filesystem to navigate the archive, as Resolved.
Oct 6 2020, 9:52 AM · Software Heritage filesystem
zack closed T2654: modprobe fuse on the CI build machine as Resolved.

Closed by @vsellier with rCJSWH5ade7a89a66b0ac6f12c00601f7e996f49e703e3

Oct 6 2020, 9:52 AM · Software Heritage filesystem, System administration

Oct 5 2020

vsellier added a comment to T2654: modprobe fuse on the CI build machine.

I failed to successfully execute mount on the container without the privileged option so I finally configured the swh-fuse job with this option.

Oct 5 2020, 4:06 PM · Software Heritage filesystem, System administration
vsellier added a revision to T2654: modprobe fuse on the CI build machine: D4144: swh-fuse: add privileged and device options to the docker container to allow mount to work.
Oct 5 2020, 4:04 PM · Software Heritage filesystem, System administration
zack added a subtask for T2659: FUSE: add flat commit view to ease batch commit workloads: T2663: FUSE: add support for revision objects.
Oct 5 2020, 3:50 PM · Software Heritage filesystem