Page MenuHomeSoftware Heritage
Differential D4389

fs: make FuseDirEntry slicable
AbandonedPublic
Actions

Authored by haltode on Oct 30 2020, 10:12 AM.

Details

Reviewers
None
Group Reviewers
Reviewers
Summary

This makes readdir() O(N) instead of O(N²) because we do not go through all
entries every time, but only those after the readdir() offset.

Diff Detail

Repository
rDFUSE FUSE virtual file system
Branch
feature/fuse-dir-slice
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 16789
Build 25885: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 25884: arc lint + arc unit

Event Timeline

haltode created this revision.Oct 30 2020, 10:12 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptOct 30 2020, 10:12 AM
swh-public-ci added a comment.Oct 30 2020, 10:13 AM

Build is green

Patch application report for D4389 (id=15518)

Rebasing onto f9de49aca7...

Current branch diff-target is up to date.
Changes applied before test
commit 7c180d5601f240c1b7a00d3c24f868b5b07ffe3a
Author: Thibault Allançon <haltode@gmail.com>
Date:   Fri Oct 30 10:10:02 2020 +0100

    fs: make FuseDirEntry slicable
    
    This makes readdir() more efficient because we do not go through all
    entries every time, but only those after the readdir() offset.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/132/ for more details.

Harbormaster completed remote builds in B16745: Diff 15518.Oct 30 2020, 10:13 AM
haltode updated this revision to Diff 15519.Oct 30 2020, 10:29 AM

Rework commit message: efficient => O(n²) to O(n)

swh-public-ci added a comment.Oct 30 2020, 10:30 AM

Build is green

Patch application report for D4389 (id=15519)

Rebasing onto f9de49aca7...

Current branch diff-target is up to date.
Changes applied before test
commit ca2e3160fb99caf8e056fd53876e6127147d7996
Author: Thibault Allançon <haltode@gmail.com>
Date:   Fri Oct 30 10:10:02 2020 +0100

    fs: make FuseDirEntry slicable
    
    This makes readdir() O(N) instead of O(N²) because we do not go through
    all entries every time, but only those after the readdir() offset.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/133/ for more details.

Harbormaster completed remote builds in B16746: Diff 15519.Oct 30 2020, 10:30 AM
haltode updated this revision to Diff 15559.Nov 2 2020, 4:52 PM

Replace aiter with get_entries().

swh-public-ci added a comment.Nov 2 2020, 4:53 PM

Build is green

Patch application report for D4389 (id=15559)

Rebasing onto 34eed7bc90...

First, rewinding head to replay your work on top of it...
Applying: fs: make FuseDirEntry slicable
Changes applied before test
commit 6bf9794dcbbb79160b66c363a56dd0105d81b6e3
Author: Thibault Allançon <haltode@gmail.com>
Date:   Fri Oct 30 10:10:02 2020 +0100

    fs: make FuseDirEntry slicable
    
    This makes readdir() O(N) instead of O(N²) because we do not go through
    all entries every time, but only those after the readdir() offset.
    
    Replace the iterator protocol `__aiter__() -> AsyncIterator[FuseEntry]`
    with a new method `get_entries(offset: int) -> List[FuseEntry]`.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/134/ for more details.

Harbormaster completed remote builds in B16788: Diff 15559.Nov 2 2020, 4:53 PM
haltode updated this revision to Diff 15560.Nov 2 2020, 4:56 PM
  • Remove useless line
  • Rebase on master
swh-public-ci added a comment.Nov 2 2020, 4:57 PM

Build has FAILED

Patch application report for D4389 (id=15560)

Rebasing onto 34eed7bc90...

Current branch diff-target is up to date.
Changes applied before test
commit a0511b010cc6e07d3ead40f69911a0e16e4058fb
Author: Thibault Allançon <haltode@gmail.com>
Date:   Fri Oct 30 10:10:02 2020 +0100

    fs: make FuseDirEntry slicable
    
    This makes readdir() O(N) instead of O(N²) because we do not go through
    all entries every time, but only those after the readdir() offset.
    
    Replace the iterator protocol `__aiter__() -> AsyncIterator[FuseEntry]`
    with a new method `get_entries(offset: int) -> List[FuseEntry]`.

Link to build: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/135/
See console output for more information: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/135/console

Harbormaster failed remote builds in B16789: Diff 15560!Nov 2 2020, 4:57 PM
haltode updated this revision to Diff 15561.Nov 2 2020, 4:58 PM

Fix typing and rework commit message

swh-public-ci added a comment.Nov 2 2020, 5:00 PM

Build is green

Patch application report for D4389 (id=15561)

Rebasing onto 34eed7bc90...

Current branch diff-target is up to date.
Changes applied before test
commit bed06c7588938f6e8d26a827f450a130f78dab0d
Author: Thibault Allançon <haltode@gmail.com>
Date:   Fri Oct 30 10:10:02 2020 +0100

    fs: replace __aiter__() with get_entries(offset: int)
    
    This makes readdir() O(N) instead of O(N²) because we do not go through
    all entries every time, but only those after the readdir() offset.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/136/ for more details.

Harbormaster completed remote builds in B16790: Diff 15561.Nov 2 2020, 5:00 PM
haltode updated this revision to Diff 15565.Nov 3 2020, 11:04 AM

Use generators instead of list.

swh-public-ci added a comment.Nov 3 2020, 11:05 AM

Build is green

Patch application report for D4389 (id=15565)

Rebasing onto 34eed7bc90...

Current branch diff-target is up to date.
Changes applied before test
commit 07a250e0452f170a05577cca0451a7dcc9e5ec32
Author: Thibault Allançon <haltode@gmail.com>
Date:   Fri Oct 30 10:10:02 2020 +0100

    fs: add get_entries(offset: int) in addition of __aiter__
    
    This makes readdir() O(N) instead of O(N²) because we do not go through
    all entries every time, but only those after the readdir() offset.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/137/ for more details.

Harbormaster completed remote builds in B16794: Diff 15565.Nov 3 2020, 11:05 AM
haltode edited the summary of this revision. (Show Details)Nov 3 2020, 11:05 AM
haltode abandoned this revision.Nov 4 2020, 5:09 PM

This has been integrated in D4345 since slicing cannot improve complexity on its own (the list is built everytime).

Revision Contents
Changeset List

PathSize
swh/
fuse/
3 lines
fs/
200 lines
6 lines
30 lines
7 lines
tests/
2 lines
2 lines
2 lines

Diff 15560

View Options

swh/fuse/cache.py

Loading...
View Options

swh/fuse/fs/artifact.py

Loading...
View Options

swh/fuse/fs/entry.py

Loading...
View Options

swh/fuse/fs/mountpoint.py

Loading...
View Options

swh/fuse/fuse.py

Loading...
View Options

swh/fuse/tests/test_meta.py

Loading...
View Options

swh/fuse/tests/test_release.py

Loading...
View Options

swh/fuse/tests/test_revision.py

Loading...
About · Developer information · Legal