Page MenuHomeSoftware Heritage

fs: history: clean sharded dir implementation
ClosedPublic

Authored by haltode on Wed, Nov 18, 3:26 PM.

Details

Summary
  • Remove manual direntry cache manipulation
  • Remove unnecessary by-hash/ sharding depth attribute
  • More extensive unit testing for by-date/

Diff Detail

Repository
rDFUSE FUSE virtual file system
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

haltode created this revision.Wed, Nov 18, 3:26 PM

Build is green

Patch application report for D4509 (id=15991)

Rebasing onto 430fa0e249...

Current branch diff-target is up to date.
Changes applied before test
commit b2cf00d0afc89f4cd1eaae3be4c76417714eb85f
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Nov 18 15:23:15 2020 +0100

    fs: history: clean sharded dir implementation
    
    - Remove manual direntry cache manipulation
    - Remove unnecessary by-hash/ sharding depth attribute
    - More extensive unit testing for by-date/

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/162/ for more details.

seirl accepted this revision.Wed, Nov 18, 5:57 PM
seirl added a subscriber: seirl.
seirl added inline comments.
swh/fuse/fs/artifact.py
356

Could you use None here?

This revision is now accepted and ready to land.Wed, Nov 18, 5:57 PM
haltode updated this revision to Diff 16011.Thu, Nov 19, 9:04 AM

Use None instead of -1.

Build is green

Patch application report for D4509 (id=16011)

Rebasing onto 430fa0e249...

Current branch diff-target is up to date.
Changes applied before test
commit b381f61806e68bea613138b0ed6c0eff892efaaf
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Nov 18 15:23:15 2020 +0100

    fs: history: clean sharded dir implementation
    
    - Remove manual direntry cache manipulation
    - Remove unnecessary by-hash/ sharding depth attribute
    - More extensive unit testing for by-date/

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/163/ for more details.

haltode updated this revision to Diff 16023.Thu, Nov 19, 11:44 AM

Add time optimization for by-date/ partial results by utilizing the metadata cache

Build is green

Patch application report for D4509 (id=16023)

Rebasing onto 430fa0e249...

Current branch diff-target is up to date.
Changes applied before test
commit 2fc39c5ba29c8705459ba51d81b391294242f117
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Nov 18 15:23:15 2020 +0100

    fs: history: clean sharded dir implementation
    
    - Remove manual direntry cache manipulation
    - Remove unnecessary by-hash/ sharding depth attribute
    - Time optimization for by-date/ partial results
    - More extensive unit testing for by-date/

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/164/ for more details.

haltode updated this revision to Diff 16024.Thu, Nov 19, 11:53 AM

Fix string formatting style

Build is green

Patch application report for D4509 (id=16024)

Rebasing onto 430fa0e249...

Current branch diff-target is up to date.
Changes applied before test
commit 88ea17fd20114b729c03d6f9eada40487fe00dff
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Nov 18 15:23:15 2020 +0100

    fs: history: clean sharded dir implementation
    
    - Remove manual direntry cache manipulation
    - Remove unnecessary by-hash/ sharding depth attribute
    - Time optimization for by-date/ partial results
    - More extensive unit testing for by-date/

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/165/ for more details.

haltode updated this revision to Diff 16037.Thu, Nov 19, 2:31 PM

ensure_cached -> get_cached_subset

Build is green

Patch application report for D4509 (id=16037)

Rebasing onto 430fa0e249...

Current branch diff-target is up to date.
Changes applied before test
commit 2ecfcf8bb4551ec5879255767d79d575d9aea779
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Nov 18 15:23:15 2020 +0100

    fs: history: clean sharded dir implementation
    
    - Remove manual direntry cache manipulation
    - Remove unnecessary by-hash/ sharding depth attribute
    - Time optimization for by-date/ partial results
    - More extensive unit testing for by-date/

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/166/ for more details.

seirl added inline comments.Thu, Nov 19, 2:39 PM
swh/fuse/cache.py
136–137

Use row/swhid to avoid reusing a variable name with a different type

swh/fuse/fs/artifact.py
305–306

Add parentheses to be easier to understand

haltode marked an inline comment as done.Thu, Nov 19, 2:42 PM
haltode updated this revision to Diff 16040.Thu, Nov 19, 2:45 PM

Fix name variable style

haltode marked 2 inline comments as done.Thu, Nov 19, 2:45 PM
haltode added inline comments.
swh/fuse/fs/artifact.py
305–306

black prevents parentheses here.

Build has FAILED

Patch application report for D4509 (id=16040)

Rebasing onto 430fa0e249...

Current branch diff-target is up to date.
Changes applied before test
commit 9decc23931740a940b7e698d7c416644b2bc8a45
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Nov 18 15:23:15 2020 +0100

    fs: history: clean sharded dir implementation
    
    - Remove manual direntry cache manipulation
    - Remove unnecessary by-hash/ sharding depth attribute
    - Time optimization for by-date/ partial results
    - More extensive unit testing for by-date/

Link to build: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/167/
See console output for more information: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/167/console

Build has FAILED

Patch application report for D4509 (id=16040)

Rebasing onto 430fa0e249...

Current branch diff-target is up to date.
Changes applied before test
commit 9decc23931740a940b7e698d7c416644b2bc8a45
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Nov 18 15:23:15 2020 +0100

    fs: history: clean sharded dir implementation
    
    - Remove manual direntry cache manipulation
    - Remove unnecessary by-hash/ sharding depth attribute
    - Time optimization for by-date/ partial results
    - More extensive unit testing for by-date/

Link to build: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/168/
See console output for more information: https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/168/console

haltode updated this revision to Diff 16087.Fri, Nov 20, 10:18 AM
haltode marked an inline comment as done.

Rebasing on master

Build is green

Patch application report for D4509 (id=16087)

Rebasing onto b5956b0fa6...

Current branch diff-target is up to date.
Changes applied before test
commit c4dd1c367078c903b31ac40f639f7b5a60657737
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Nov 18 15:23:15 2020 +0100

    fs: history: clean sharded dir implementation
    
    - Remove manual direntry cache manipulation
    - Remove unnecessary by-hash/ sharding depth attribute
    - Time optimization for by-date/ partial results
    - More extensive unit testing for by-date/

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/170/ for more details.

This revision was automatically updated to reflect the committed changes.