Page MenuHomeSoftware Heritage

fs: history: add by-date/ sharded directory
ClosedPublic

Authored by haltode on Nov 17 2020, 11:28 AM.

Diff Detail

Repository
rDFUSE FUSE virtual file system
Branch
feature/add-history-by-date-sharding
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 17181
Build 26526: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 26525: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D4489 (id=15926)

Rebasing onto 5fdda3d1cf...

Current branch diff-target is up to date.
Changes applied before test
commit 48fa448ffdb9980c3313d10c4a2ab2660418d4cd
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:58 2020 +0100

    fs: history: add by-date/ sharded directory
    
    Closes T2773.

commit 7a3ba79124a135ece42984168ebcd8cf439ba340
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:49 2020 +0100

    cache: fix exception when storing empty history (for root commit)

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/155/ for more details.

Build is green

Patch application report for D4489 (id=15927)

Rebasing onto 5fdda3d1cf...

Current branch diff-target is up to date.
Changes applied before test
commit d83cf2a30be560d1e3bc1dff6f8356f7ea7da242
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:58 2020 +0100

    fs: history: add by-date/ sharded directory
    
    Closes T2773.

commit 7a3ba79124a135ece42984168ebcd8cf439ba340
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:49 2020 +0100

    cache: fix exception when storing empty history (for root commit)

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/156/ for more details.

swh/fuse/fs/artifact.py
212–223

Hm, i realize now that starting the new thread here might mean multiple concurrent access to the direntry cache since we are modifying it as well a few lines down the code. So move this after by_hash and by_page? Also, put all the direntry cache filling in the separate cache?

  • Rebase on master
  • Use asyncio.create_task() instead of asyncio.run_coroutine_threadsafe

Build is green

Patch application report for D4489 (id=15936)

Rebasing onto 2be0aa2f2b...

Current branch diff-target is up to date.
Changes applied before test
commit 93d047340081b3f3602c701f3903cf27c8e42906
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:58 2020 +0100

    fs: history: add by-date/ sharded directory
    
    Closes T2773.

commit 7372b73ecc5ebeca39956203ada7831c4b5331b6
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:49 2020 +0100

    cache: fix exception when storing empty history (for root commit)

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/158/ for more details.

New implem using the metadata on-disk cache.

Build is green

Patch application report for D4489 (id=15958)

Rebasing onto 2b1c921089...

First, rewinding head to replay your work on top of it...
Applying: cache: fix exception when storing empty history (for root commit)
Applying: fs: history: add by-date/ sharded directory
Applying: WIP: new implem
Changes applied before test
commit af7b55c158a4657c529e587001d11a20fcb25ff1
Author: Thibault Allançon <haltode@gmail.com>
Date:   Wed Nov 18 10:17:07 2020 +0100

    WIP: new implem

commit 78ceb74596f24a81435cae77f347493b8184a0e8
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:58 2020 +0100

    fs: history: add by-date/ sharded directory
    
    Closes T2773.

commit 0c26a00611ab1c9d455cb15bb3f04ad44cb5e05b
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:49 2020 +0100

    cache: fix exception when storing empty history (for root commit)

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/159/ for more details.

This revision is now accepted and ready to land.Nov 18 2020, 12:49 PM
haltode retitled this revision from WIP: fs: history: add by-date/ sharded directory to fs: history: add by-date/ sharded directory.Nov 18 2020, 1:19 PM
haltode edited the summary of this revision. (Show Details)

Build is green

Patch application report for D4489 (id=15974)

Rebasing onto 2b1c921089...

First, rewinding head to replay your work on top of it...
Applying: cache: fix exception when storing empty history (for root commit)
Applying: fs: history: add by-date/ sharded directory
Changes applied before test
commit 8ef9eca7914854345a74ff6ceda2d6cf3e88ca03
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:58 2020 +0100

    fs: history: add by-date/ sharded directory
    
    Closes T2773.

commit d18e5e17661ef324f0935274a56a850990797561
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:49 2020 +0100

    cache: fix exception when storing empty history (for root commit)

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/160/ for more details.

Build is green

Patch application report for D4489 (id=15976)

Rebasing onto 2b1c921089...

Current branch diff-target is up to date.
Changes applied before test
commit 430fa0e24981bad59432f9ad5b1d9e8229176df8
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:58 2020 +0100

    fs: history: add by-date/ sharded directory
    
    Closes T2773.

commit 33909eb22572d84135d31fab6eda6745409c19cd
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Nov 17 10:46:49 2020 +0100

    cache: fix exception when storing empty history (for root commit)

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/161/ for more details.