Page MenuHomeSoftware Heritage
Feed Advanced Search

Oct 16 2020

zack updated the diff for D4285: doc: document configuration file location.
  • configuration doc: convert tabs to spaces
Oct 16 2020, 2:01 PM
zack updated the summary of D4285: doc: document configuration file location.
Oct 16 2020, 1:24 PM
zack updated the summary of D4285: doc: document configuration file location.
Oct 16 2020, 1:24 PM
zack updated the diff for D4285: doc: document configuration file location.

better commit message

Oct 16 2020, 1:24 PM
zack created D4285: doc: document configuration file location.
Oct 16 2020, 1:22 PM
zack accepted D4282: cli: move configuration settings up to "swh fuse".

Caveat: this does much more than just moving the -C option up one level, so please use a more appropriate commit message before landing.

Oct 16 2020, 11:07 AM
zack raised the priority of T2699: FUSE: mount fails silently when --foreground is not specified from Normal to High.
Oct 16 2020, 9:02 AM · Software Heritage filesystem
zack changed the status of T2697: FUSE: add logging mechanism from Open to Work in Progress.
Oct 16 2020, 9:02 AM · Software Heritage filesystem
zack accepted D4274: fuse: add support for snapshot artifacts.
Oct 16 2020, 9:01 AM

Oct 15 2020

zack committed rDFUSE9120d2963f1b: design doc: update to reflects current state (authored by zack).
design doc: update to reflects current state
Oct 15 2020, 3:48 PM
zack accepted D4271: fuse: use lazy symlinks for revisions parents.
Oct 15 2020, 3:41 PM
zack added inline comments to D4265: cli: redirect logging to user syslog when daemonized.
Oct 15 2020, 12:47 PM
zack added projects to T2706: Benchmark objstorage for mirror (uffizi vs. azure vs. s3): Mirror, Object storage.
Oct 15 2020, 12:44 PM · Object storage, Mirror
zack triaged T2705: FUSE CLI: move --config-file option up to "swh fuse" as Normal priority.
Oct 15 2020, 11:06 AM · Software Heritage filesystem
zack committed rDFUSEe8307fe54d19: CLI: add an exlicit --daemon flag (authored by zack).
CLI: add an exlicit --daemon flag
Oct 15 2020, 11:02 AM
zack closed D4264: CLI: add an exlicit --daemon flag.
Oct 15 2020, 11:02 AM
zack added reviewers for D4264: CLI: add an exlicit --daemon flag: seirl, haltode.
Oct 15 2020, 10:13 AM
zack created D4264: CLI: add an exlicit --daemon flag.
Oct 15 2020, 10:12 AM
zack renamed T2697: FUSE: add logging mechanism from Output log to user journal when started as a daemon to FUSE: add logging mechanism.
Oct 15 2020, 9:19 AM · Software Heritage filesystem

Oct 14 2020

zack added a comment to T2702: FUSE: add an unmount command.

Also: what are the cons of adding a umount command with just wraps fusermount -u?

Oct 14 2020, 1:04 PM · Software Heritage filesystem
zack added a comment to T2702: FUSE: add an unmount command.

We can't expect all users to read the tutorial, that just doesn't happen in the real world :-)
If we don't add an explicit umount command, we should at least document how to unmount also in the doc of the mount command

Oct 14 2020, 12:41 PM · Software Heritage filesystem
zack added a comment to T2702: FUSE: add an unmount command.

To unmount you can use fusermount -u /mntdir

Oct 14 2020, 11:57 AM · Software Heritage filesystem
zack triaged T2702: FUSE: add an unmount command as Low priority.
Oct 14 2020, 11:53 AM · Software Heritage filesystem
zack accepted D4253: cache: make sure the cache directory exists.

LGTM. (Please squash the 2 commits before merging though; 2 commits for a one-liner seem too much ;-))

Oct 14 2020, 9:31 AM

Oct 13 2020

zack accepted D4247: developer-setup: add FUSE dependencies.
Oct 13 2020, 4:23 PM
zack committed rDFUSE2312f6e9c713: mount: improve CLI help string and add usage example (authored by zack).
mount: improve CLI help string and add usage example
Oct 13 2020, 3:24 PM
zack closed D4243: mount: improve CLI help string and add usage example.
Oct 13 2020, 3:24 PM
zack added a reviewer for D4243: mount: improve CLI help string and add usage example: haltode.
Oct 13 2020, 2:40 PM
zack triaged T2699: FUSE: mount fails silently when --foreground is not specified as Normal priority.
Oct 13 2020, 2:34 PM · Software Heritage filesystem
zack triaged T2698: FUSE: mount fail when cache directory is missing as High priority.
Oct 13 2020, 2:33 PM · Software Heritage filesystem
zack created D4243: mount: improve CLI help string and add usage example.
Oct 13 2020, 2:30 PM
zack created P819 Command-Line Input.
Oct 13 2020, 2:18 PM
zack edited P818 Command-Line Input.
Oct 13 2020, 2:15 PM
zack created P818 Command-Line Input.
Oct 13 2020, 2:14 PM
zack created GitLab migration.
Oct 13 2020, 12:52 PM
zack triaged T2694: FUSE: add sharding support for top-level dirs as Low priority.
Oct 13 2020, 10:45 AM · Software Heritage filesystem
zack edited reviewers for D4240: fuse: allow mounting artifacts on the fly, added: seirl; removed: zack.
Oct 13 2020, 10:41 AM
zack updated the task description for T1740: fetch extrinsic origin metadata from GitHub.
Oct 13 2020, 10:21 AM · Metadata workflow, Origin-GitHub
zack updated the task description for T2693: fetch extrinsic origin metadata from GitLab instances.
Oct 13 2020, 10:21 AM · Extrinsic metadata, Origin-GitLab
zack updated the task description for T2693: fetch extrinsic origin metadata from GitLab instances.
Oct 13 2020, 10:20 AM · Extrinsic metadata, Origin-GitLab
zack updated the task description for T2693: fetch extrinsic origin metadata from GitLab instances.
Oct 13 2020, 10:20 AM · Extrinsic metadata, Origin-GitLab
zack updated subscribers of T2693: fetch extrinsic origin metadata from GitLab instances.
Oct 13 2020, 10:15 AM · Extrinsic metadata, Origin-GitLab
zack added a subtask for T2693: fetch extrinsic origin metadata from GitLab instances: T1739: Define an architecture to fetch extrinsic metadata outside listers and loaders.
Oct 13 2020, 10:15 AM · Extrinsic metadata, Origin-GitLab
zack added a parent task for T1739: Define an architecture to fetch extrinsic metadata outside listers and loaders: T2693: fetch extrinsic origin metadata from GitLab instances.
Oct 13 2020, 10:15 AM · Extrinsic metadata
zack triaged T2693: fetch extrinsic origin metadata from GitLab instances as Normal priority.
Oct 13 2020, 10:14 AM · Extrinsic metadata, Origin-GitLab
zack renamed T1740: fetch extrinsic origin metadata from GitHub from fetch extrinsic origin metadata from Github to fetch extrinsic origin metadata from GitHub.
Oct 13 2020, 10:10 AM · Metadata workflow, Origin-GitHub
zack updated the task description for T1740: fetch extrinsic origin metadata from GitHub.
Oct 13 2020, 10:08 AM · Metadata workflow, Origin-GitHub
zack renamed T1740: fetch extrinsic origin metadata from GitHub from Fetch extrinsic origin metadata from Github to fetch extrinsic origin metadata from Github.
Oct 13 2020, 10:08 AM · Metadata workflow, Origin-GitHub
zack accepted D4241: scanner: removed _iter_nodes_attr function in model (causes results duplication).

I confirm this fixes the issue. LGTM, thanks !

Oct 13 2020, 9:19 AM

Oct 12 2020

zack triaged T2679: Use the `swh.model` version of `extract_regex_objs` as Low priority.
Oct 12 2020, 6:59 PM · Code scanner
zack triaged T2690: swh scanner reports double results in ndjson format as Normal priority.
Oct 12 2020, 6:59 PM · Code scanner
zack requested changes to D4240: fuse: allow mounting artifacts on the fly.
Oct 12 2020, 6:01 PM

Oct 11 2020

zack triaged T2684: notify the swh-devel mailing list of build failures for daily CI jobs as High priority.
Oct 11 2020, 11:19 AM · Continuous Integration

Oct 9 2020

zack assigned T2678: Have a default location for the configuration file to haltode.

While we wait for a proper solution, I think there is nothing stopping swh fuse from loading a configuration file by default if it exists and -C is not passed on the CLI.
We should just move -C up one level (from swh fuse mount to swh fuse) and do something similar to what swh-scanner does.
The location of the file might change in the future as a consequence of T1410, but meh, it's not a big deal.
We can stay on the safe side for now and only support ~/.config/swh/fuse/fuse.yml. And we will worry about merging with a global config file later.

Oct 9 2020, 6:24 PM · Software Heritage filesystem
zack created P816 Command-Line Input.
Oct 9 2020, 4:47 PM
zack added a project to T2680: proxy support for swh scanner: Easy hack.
Oct 9 2020, 2:59 PM · Easy hack, Code scanner
zack triaged T2680: proxy support for swh scanner as Normal priority.
Oct 9 2020, 2:58 PM · Easy hack, Code scanner
zack accepted D4218: docs: add configuration.rst.

please just fix the inconsistency in quoting in the example before landing, it is not clear to me why url is quoted whereas path is not

Oct 9 2020, 2:17 PM
zack requested changes to D4218: docs: add configuration.rst.
  • can you please add info about where the configuration file is located? (both by default and w.r.t. XDG_CONFIG_HOME, if applicable)
  • as people might not be familiar with YAML quirks, having an example of a skeleton config file, in full concrete syntax, would be also useful
Oct 9 2020, 1:50 PM
zack added a comment to D4193: swh identify: add --exclude.

Changed --ignore-dir by --exclude

It supports glob, but had to copy extract_regex_objs from swh.scan.

Oct 9 2020, 12:55 PM
zack added inline comments to D4200: Add support for revision artifacts.
Oct 9 2020, 8:58 AM
zack committed rDDOCffbcb67c0041: add swh-fuse to documentation (authored by zack).
add swh-fuse to documentation
Oct 9 2020, 8:01 AM

Oct 8 2020

zack triaged T2676: FUSE: write tutorial doc as Low priority.
Oct 8 2020, 8:21 PM · Documentation, Software Heritage filesystem
zack added a comment to T2674: No login to the webapp.

It'd be useful to have a post-mortem on this, to make sure it doesn't happen again in the future.
Why wasn't @rdicosmo able to login? What has changed? The login URL? (if so, is the old URL still linked from anywhere?) Or were the old credentials temporarily unavailable? etc.

Oct 8 2020, 8:18 PM · Web app
zack created P810 Command-Line Input.
Oct 8 2020, 2:11 PM
zack committed rDENVc951baf68f3a: mrconfig: add swh-fuse repository (authored by haltode).
mrconfig: add swh-fuse repository
Oct 8 2020, 1:57 PM
zack closed D4199: mrconfig: add swh-fuse repository.
Oct 8 2020, 1:57 PM
zack accepted D4199: mrconfig: add swh-fuse repository.
Oct 8 2020, 1:53 PM
zack triaged T2673: FUSE: document configuration file location and syntax as Normal priority.
Oct 8 2020, 11:58 AM · Documentation, Software Heritage filesystem
zack resigned from D4193: swh identify: add --exclude.
Oct 8 2020, 11:46 AM
zack added inline comments to D4193: swh identify: add --exclude.
Oct 8 2020, 11:46 AM
zack triaged T2671: web client: add a CLI shim on top of the Python API as Low priority.
Oct 8 2020, 10:59 AM · Web client
zack triaged T2670: web client: expose web search as Low priority.
Oct 8 2020, 10:57 AM · Web client
zack requested changes to D4193: swh identify: add --exclude.

Thanks, even though this is a little bit disturbing discrepancy wrt swh-scanner exclusion mechanism, it's probably for the best, and arguably we should have something this simple also there.

Oct 8 2020, 9:09 AM

Oct 7 2020

zack triaged T2669: FUSE: add "swh fuse clean" CLI command to clean the cache(s) as Low priority.
Oct 7 2020, 8:06 PM · Software Heritage filesystem
zack accepted D4064: Early FUSE implementation, with support for blob and directory objects.
Oct 7 2020, 3:09 PM

Oct 6 2020

zack accepted D4160: client: make API response typify-ing optional.
Oct 6 2020, 4:45 PM
zack requested changes to D4160: client: make API response typify-ing optional.
Oct 6 2020, 4:36 PM
zack requested changes to D4160: client: make API response typify-ing optional.
Oct 6 2020, 4:24 PM
zack closed T2654: modprobe fuse on the CI build machine, a subtask of T1926: FUSE filesystem to navigate the archive, as Resolved.
Oct 6 2020, 9:52 AM · Software Heritage filesystem
zack closed T2654: modprobe fuse on the CI build machine as Resolved.

Closed by @vsellier with rCJSWH5ade7a89a66b0ac6f12c00601f7e996f49e703e3

Oct 6 2020, 9:52 AM · Software Heritage filesystem, System administration
zack added a comment to D4089: Add tests and fix behavior of scanner cli.

is this actually proposed for merging now? If so, please remove "WIP" from the diff title

Oct 6 2020, 9:20 AM
zack accepted D4028: Add Spotless formatting tool.
Oct 6 2020, 9:18 AM

Oct 5 2020

zack added a subtask for T2659: FUSE: add flat commit view to ease batch commit workloads: T2663: FUSE: add support for revision objects.
Oct 5 2020, 3:50 PM · Software Heritage filesystem
zack added a parent task for T2663: FUSE: add support for revision objects: T2659: FUSE: add flat commit view to ease batch commit workloads.
Oct 5 2020, 3:50 PM · Software Heritage filesystem
zack triaged T2665: FUSE: add support for snapshot objects as Normal priority.
Oct 5 2020, 2:13 PM · Software Heritage filesystem
zack triaged T2664: FUSE: add support for release objects as Normal priority.
Oct 5 2020, 2:12 PM · Software Heritage filesystem
zack triaged T2663: FUSE: add support for revision objects as Normal priority.
Oct 5 2020, 2:11 PM · Software Heritage filesystem
zack retitled D4064: Early FUSE implementation, with support for blob and directory objects from Early FUSE implementation to Early FUSE implementation, with support for blob and directory objects.
Oct 5 2020, 2:09 PM

Oct 4 2020

zack added a comment to T2656: Pause background ingestion until we get more local storage space.
In T2656#50089, @olasd wrote:

I've thought about it, but I don't know which component to mark as "degraded" currently. If anything, user-facing features like save code now and the vault should be going faster than usual :-)

Oct 4 2020, 3:38 PM · System administration
zack updated subscribers of T2656: Pause background ingestion until we get more local storage space.

When this is done (I guess in large part it is already, judging from the ingestion dashboard), please also update https://status.softwareheritage.org/ with a suitable message.

Oct 4 2020, 12:37 PM · System administration

Oct 2 2020

zack added a comment to D4064: Early FUSE implementation, with support for blob and directory objects.

maybe I'm doing something wrong, but with the current version of this diff applied locally I get the following traceback when trying to FUSE-mount the archive:

$ mkdir tmp-archive
$ swh fuse mount tmp-archive
Traceback (most recent call last):
  File "/home/zack/.virtualenvs/swh/bin/swh", line 33, in <module>
    sys.exit(load_entry_point('swh.core', 'console_scripts', 'swh')())
  File "/home/zack/dati/projects/sw-heritage/git/swh-environment/swh-core/swh/core/cli/__init__.py", line 135, in main
    return swh(auto_envvar_prefix="SWH")
  File "/home/zack/.virtualenvs/swh/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/zack/.virtualenvs/swh/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/zack/.virtualenvs/swh/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/zack/.virtualenvs/swh/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/zack/.virtualenvs/swh/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/zack/.virtualenvs/swh/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/zack/.virtualenvs/swh/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/zack/dati/projects/sw-heritage/git/swh-environment/swh-fuse/swh/fuse/cli.py", line 120, in mount
    stack.enter(DaemonContext())
AttributeError: 'ExitStack' object has no attribute 'enter'
Oct 2 2020, 10:46 PM
zack accepted D4130: Improve cli documentation.

tested and confirmed—yes, I like it! thanks

Oct 2 2020, 10:37 PM
zack accepted D4131: Remove parse_url helper that adds no real value.
Oct 2 2020, 10:36 PM
zack triaged T2659: FUSE: add flat commit view to ease batch commit workloads as Low priority.
Oct 2 2020, 6:22 PM · Software Heritage filesystem
zack added inline comments to D4064: Early FUSE implementation, with support for blob and directory objects.
Oct 2 2020, 5:05 PM
zack requested changes to D4064: Early FUSE implementation, with support for blob and directory objects.

Just a couple of minor things for me. I'll delegate to @seirl the remaining checks.
Good job !

Oct 2 2020, 4:57 PM
zack committed rDWCLI677e98565d61: tox.ini: pin black to the pre-commit version (19.10b0) to avoid flip-flops (authored by zack).
tox.ini: pin black to the pre-commit version (19.10b0) to avoid flip-flops
Oct 2 2020, 4:24 PM
zack committed rDVAUb0c50dd16868: tox.ini: pin black to the pre-commit version (19.10b0) to avoid flip-flops (authored by zack).
tox.ini: pin black to the pre-commit version (19.10b0) to avoid flip-flops
Oct 2 2020, 4:24 PM
zack committed rDSEAc3e5bf0a11d8: tox.ini: pin black to the pre-commit version (19.10b0) to avoid flip-flops (authored by zack).
tox.ini: pin black to the pre-commit version (19.10b0) to avoid flip-flops
Oct 2 2020, 4:24 PM