Page MenuHomeSoftware Heritage
Feed Advanced Search

Nov 13 2020

haltode committed rDFUSE604665ab40ec: cache: replace asizeof() call with simpler heuristic (authored by haltode).
cache: replace asizeof() call with simpler heuristic
Nov 13 2020, 4:06 PM
haltode committed rDFUSE2b1a725ad68d: fs: history: do not store history list in memory (authored by haltode).
fs: history: do not store history list in memory
Nov 13 2020, 4:06 PM
haltode committed rDFUSEdb41b75df1e3: fs: history: add by-hash/ sharded directory (authored by haltode).
fs: history: add by-hash/ sharded directory
Nov 13 2020, 4:06 PM
haltode closed D4416: fs: history: add by-hash/ sharded directory.
Nov 13 2020, 4:06 PM
haltode updated the diff for D4416: fs: history: add by-hash/ sharded directory.
  • Fix string construction
  • Rework commit messages
Nov 13 2020, 4:00 PM
haltode triaged T2775: Add top-level README to explain briefly archive/ and meta/ behavior as Normal priority.
Nov 13 2020, 12:46 PM · Software Heritage filesystem

Nov 10 2020

haltode abandoned D4371: WIP: archive + meta directory sharding.

Moved on to only sharding history/ directory instead of archive/ and meta/ (see D4416).

Nov 10 2020, 12:44 PM

Nov 6 2020

haltode updated the diff for D4416: fs: history: add by-hash/ sharded directory.
  • clean duplicate code
  • remove cache asizeof
Nov 6 2020, 3:56 PM
haltode updated the diff for D4416: fs: history: add by-hash/ sharded directory.

Do not store history in memory

Nov 6 2020, 10:42 AM

Nov 5 2020

haltode updated the diff for D4416: fs: history: add by-hash/ sharded directory.

Fix zack comments (cache outside of loop)

Nov 5 2020, 1:56 PM
haltode created D4416: fs: history: add by-hash/ sharded directory.
Nov 5 2020, 12:21 PM

Nov 4 2020

haltode moved T2695: Cache directory entries to make readdir/lookup more efficient from In progress to Done on the Software Heritage filesystem board.
Nov 4 2020, 5:10 PM · Software Heritage filesystem
haltode committed rDFUSE46a48a190722: fuse: add cache on directories entries (authored by haltode).
fuse: add cache on directories entries
Nov 4 2020, 5:10 PM
haltode closed T2695: Cache directory entries to make readdir/lookup more efficient as Resolved by committing rDFUSE46a48a190722: fuse: add cache on directories entries.
Nov 4 2020, 5:10 PM · Software Heritage filesystem
haltode closed D4345: fuse: add cache on directories entries.
Nov 4 2020, 5:10 PM
haltode retitled D4345: fuse: add cache on directories entries from WIP fuse: add cache on directories entries to fuse: add cache on directories entries.
Nov 4 2020, 5:09 PM
haltode abandoned D4389: fs: make FuseDirEntry slicable.

This has been integrated in D4345 since slicing cannot improve complexity on its own (the list is built everytime).

Nov 4 2020, 5:09 PM
haltode updated the diff for D4345: fuse: add cache on directories entries.

Squash

Nov 4 2020, 5:07 PM
haltode updated the diff for D4345: fuse: add cache on directories entries.

Use AsyncIterator in implementation and remove slicing

Nov 4 2020, 3:20 PM
haltode edited P851 (An Untitled Masterwork).
Nov 4 2020, 2:37 PM
haltode edited P851 (An Untitled Masterwork).
Nov 4 2020, 1:27 PM
haltode updated the diff for D4345: fuse: add cache on directories entries.

Use Sequence

Nov 4 2020, 11:57 AM
haltode updated the diff for D4345: fuse: add cache on directories entries.

aiter -> compute_entries

Nov 4 2020, 11:39 AM
haltode created P851 (An Untitled Masterwork).
Nov 4 2020, 10:25 AM

Nov 3 2020

haltode updated the diff for D4345: fuse: add cache on directories entries.

Remove unnecessary async

Nov 3 2020, 3:32 PM
haltode updated the diff for D4345: fuse: add cache on directories entries.

Add offset and generator for direntries.

Nov 3 2020, 3:28 PM
haltode updated the diff for D4345: fuse: add cache on directories entries.

Rebasing on master

Nov 3 2020, 2:02 PM
haltode updated the summary of D4389: fs: make FuseDirEntry slicable.
Nov 3 2020, 11:05 AM
haltode updated the diff for D4389: fs: make FuseDirEntry slicable.

Use generators instead of list.

Nov 3 2020, 11:04 AM
haltode created P847 (An Untitled Masterwork).
Nov 3 2020, 9:48 AM

Nov 2 2020

haltode updated the diff for D4389: fs: make FuseDirEntry slicable.

Fix typing and rework commit message

Nov 2 2020, 4:58 PM
haltode updated the diff for D4389: fs: make FuseDirEntry slicable.
  • Remove useless line
  • Rebase on master
Nov 2 2020, 4:56 PM
haltode updated the diff for D4389: fs: make FuseDirEntry slicable.

Replace aiter with get_entries().

Nov 2 2020, 4:52 PM

Oct 30 2020

haltode updated the diff for D4389: fs: make FuseDirEntry slicable.

Rework commit message: efficient => O(n²) to O(n)

Oct 30 2020, 10:29 AM
haltode created D4389: fs: make FuseDirEntry slicable.
Oct 30 2020, 10:12 AM

Oct 29 2020

haltode updated the summary of D4371: WIP: archive + meta directory sharding.
Oct 29 2020, 10:05 AM
haltode added a revision to T2694: FUSE: add sharding support for top-level dirs: D4371: WIP: archive + meta directory sharding.
Oct 29 2020, 10:05 AM · Software Heritage filesystem

Oct 28 2020

haltode updated the summary of D4371: WIP: archive + meta directory sharding.
Oct 28 2020, 2:20 PM
haltode updated the diff for D4371: WIP: archive + meta directory sharding.

Update commit message

Oct 28 2020, 1:46 PM
haltode created D4371: WIP: archive + meta directory sharding.
Oct 28 2020, 1:42 PM

Oct 27 2020

haltode changed the status of T2694: FUSE: add sharding support for top-level dirs from Open to Work in Progress.
Oct 27 2020, 4:12 PM · Software Heritage filesystem
haltode created P839 (An Untitled Masterwork).
Oct 27 2020, 10:07 AM

Oct 26 2020

haltode created P838 time ls -f history/.
Oct 26 2020, 5:15 PM
haltode updated the diff for D4345: fuse: add cache on directories entries.
  • Use RAM usage instead of number of dir entries for LRU cache policy
  • Parse % of RAM or B/KB/MB/GB in config file
Oct 26 2020, 3:41 PM
haltode created P835 (An Untitled Masterwork).
Oct 26 2020, 1:59 PM
haltode created P834 (An Untitled Masterwork).
Oct 26 2020, 1:08 PM

Oct 23 2020

haltode moved T2723: Change `logging.error` into `logging.debug` when returning ENOENT from In progress to Done on the Software Heritage filesystem board.
Oct 23 2020, 5:01 PM · Software Heritage filesystem
haltode committed rDFUSEf9de49aca704: fuse: use debug level instead of error when returning ENOENT (authored by haltode).
fuse: use debug level instead of error when returning ENOENT
Oct 23 2020, 5:01 PM
haltode closed T2723: Change `logging.error` into `logging.debug` when returning ENOENT as Resolved by committing rDFUSEf9de49aca704: fuse: use debug level instead of error when returning ENOENT.
Oct 23 2020, 5:01 PM · Software Heritage filesystem
haltode closed D4339: fuse: use debug level instead of error when returning ENOENT.
Oct 23 2020, 5:01 PM
haltode retitled D4345: fuse: add cache on directories entries from fuse: add cache on directories entries to WIP fuse: add cache on directories entries.
Oct 23 2020, 4:45 PM
haltode added a revision to T2695: Cache directory entries to make readdir/lookup more efficient: D4345: fuse: add cache on directories entries.
Oct 23 2020, 4:44 PM · Software Heritage filesystem
haltode created D4345: fuse: add cache on directories entries.
Oct 23 2020, 4:43 PM
haltode changed the status of T2695: Cache directory entries to make readdir/lookup more efficient from Open to Work in Progress.
Oct 23 2020, 2:03 PM · Software Heritage filesystem
haltode added a revision to T2723: Change `logging.error` into `logging.debug` when returning ENOENT: D4339: fuse: use debug level instead of error when returning ENOENT.
Oct 23 2020, 10:39 AM · Software Heritage filesystem
haltode created D4339: fuse: use debug level instead of error when returning ENOENT.
Oct 23 2020, 10:39 AM

Oct 22 2020

haltode moved T2659: FUSE: add flat commit view to ease batch commit workloads from In progress to Done on the Software Heritage filesystem board.
Oct 22 2020, 5:46 PM · Software Heritage filesystem
haltode committed rDFUSEe8cb3c6d7456: tests: add unit test for revision history (authored by haltode).
tests: add unit test for revision history
Oct 22 2020, 5:46 PM
haltode committed rDFUSEa8abee511524: fuse: add flat commit view in a 'history' virtual directory (authored by haltode).
fuse: add flat commit view in a 'history' virtual directory
Oct 22 2020, 5:46 PM
haltode closed T2659: FUSE: add flat commit view to ease batch commit workloads as Resolved by committing rDFUSEa8abee511524: fuse: add flat commit view in a 'history' virtual directory.
Oct 22 2020, 5:46 PM · Software Heritage filesystem
haltode closed D4309: Add flat commit view in a history/ virtual dir.
Oct 22 2020, 5:45 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.
  • Update docstrings
  • Remove RevisionHistory history inner field
Oct 22 2020, 5:44 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Add missing "if not exists" clause in sqlite index

Oct 22 2020, 3:35 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Fast graph sql query (union instead of union all).

Oct 22 2020, 9:34 AM

Oct 21 2020

haltode created P831 (An Untitled Masterwork).
Oct 21 2020, 4:51 PM
haltode added inline comments to D4309: Add flat commit view in a history/ virtual dir.
Oct 21 2020, 3:36 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Update unit tests for new history cache graph structure

Oct 21 2020, 3:18 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Represent ancestors using graph edges in sqlite.

Oct 21 2020, 2:55 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Rebasing on master.

Oct 21 2020, 1:26 PM
haltode moved T2725: Concurrent access result in ERROR:aiosqlite:returning exception database is locked from In progress to Done on the Software Heritage filesystem board.
Oct 21 2020, 1:23 PM · Software Heritage filesystem
haltode committed rDFUSE6a9b71965538: cache: add missing aiosqlite commit call (authored by haltode).
cache: add missing aiosqlite commit call
Oct 21 2020, 1:23 PM
haltode closed T2725: Concurrent access result in ERROR:aiosqlite:returning exception database is locked as Resolved by committing rDFUSE6a9b71965538: cache: add missing aiosqlite commit call.
Oct 21 2020, 1:23 PM · Software Heritage filesystem
haltode closed D4316: cache: add missing aiosqlite commit call.
Oct 21 2020, 1:23 PM
haltode updated the diff for D4316: cache: add missing aiosqlite commit call.

Rebase on master.

Oct 21 2020, 11:16 AM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Rebasing on master.

Oct 21 2020, 11:15 AM
haltode added inline comments to D4309: Add flat commit view in a history/ virtual dir.
Oct 21 2020, 11:14 AM
haltode added inline comments to D4309: Add flat commit view in a history/ virtual dir.
Oct 21 2020, 11:05 AM
haltode retitled D4316: cache: add missing aiosqlite commit call from cache: add missing commit aiosqlite call to cache: add missing aiosqlite commit call.
Oct 21 2020, 10:13 AM
haltode updated the diff for D4316: cache: add missing aiosqlite commit call.

Update commit message

Oct 21 2020, 10:13 AM
haltode added a revision to T2725: Concurrent access result in ERROR:aiosqlite:returning exception database is locked: D4316: cache: add missing aiosqlite commit call.
Oct 21 2020, 10:13 AM · Software Heritage filesystem
haltode created D4316: cache: add missing aiosqlite commit call.
Oct 21 2020, 10:13 AM

Oct 20 2020

haltode moved T1926: FUSE filesystem to navigate the archive from Backlog to Done on the Software Heritage filesystem board.
Oct 20 2020, 9:41 PM · Software Heritage filesystem
haltode closed T1926: FUSE filesystem to navigate the archive as Resolved.
Oct 20 2020, 9:41 PM · Software Heritage filesystem
haltode changed the status of T2723: Change `logging.error` into `logging.debug` when returning ENOENT from Open to Work in Progress.
Oct 20 2020, 4:38 PM · Software Heritage filesystem
haltode changed the status of T2725: Concurrent access result in ERROR:aiosqlite:returning exception database is locked from Open to Work in Progress.
Oct 20 2020, 4:38 PM · Software Heritage filesystem
haltode triaged T2725: Concurrent access result in ERROR:aiosqlite:returning exception database is locked as Normal priority.
Oct 20 2020, 4:31 PM · Software Heritage filesystem
haltode triaged T2724: fuse: populate history/ incrementally to avoid temporary hangs on huge repos as Normal priority.
Oct 20 2020, 4:04 PM · Software Heritage filesystem
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Request history at the latest once the history/ is accessed, not the revision
directory.

Oct 20 2020, 4:00 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.
  • tests: add unit test for revision history
Oct 20 2020, 3:40 PM
haltode added a revision to T2659: FUSE: add flat commit view to ease batch commit workloads: D4309: Add flat commit view in a history/ virtual dir.
Oct 20 2020, 1:38 PM · Software Heritage filesystem
haltode updated the summary of D4309: Add flat commit view in a history/ virtual dir.
Oct 20 2020, 1:38 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Commit message.

Oct 20 2020, 1:32 PM
haltode retitled D4309: Add flat commit view in a history/ virtual dir from WIP: history/ directory to Add flat commit view in a history/ virtual dir.
Oct 20 2020, 1:20 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Better description in design.md

Oct 20 2020, 1:16 PM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Do not raise error when missing entry in swh-graph

Oct 20 2020, 11:19 AM
haltode accepted D4314: rename main CLI command from "swh fuse" to "swh fs".
Oct 20 2020, 10:31 AM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Update fuse clean command to remove history cache.

Oct 20 2020, 10:17 AM
haltode triaged T2723: Change `logging.error` into `logging.debug` when returning ENOENT as Low priority.
Oct 20 2020, 9:58 AM · Software Heritage filesystem
haltode added inline comments to D4309: Add flat commit view in a history/ virtual dir.
Oct 20 2020, 9:54 AM
haltode updated the diff for D4309: Add flat commit view in a history/ virtual dir.

Add history cache and history dir in design.md

Oct 20 2020, 9:52 AM