Page MenuHomeSoftware Heritage

FUSE: cache: share sqlite connection between metadata/history cache
ClosedPublic

Authored by haltode on Jan 4 2021, 10:24 AM.

Details

Summary

Since metadata and history already share the same database, it makes
sense to share the connection as well, to prevent unnecessary locking on
the db file.

Related to T2830.

Diff Detail

Repository
rDFUSE FUSE virtual file system
Branch
fix-one-connection-metadatadb
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 18163
Build 28036: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 28035: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D4801 (id=16990)

Rebasing onto 3a76424ac8...

Current branch diff-target is up to date.
Changes applied before test
commit 32223da19ca87999645bba6905c9ed86eda3a0d6
Author: Thibault Allançon <haltode@gmail.com>
Date:   Mon Jan 4 10:20:06 2021 +0100

    cache: share sqlite connection between metadata/history cache
    
    Since metadata and history already share the same database, it makes
    sense to share the connection as well, to prevent unnecessary locking on
    the db file.
    
    Related to T2830.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/255/ for more details.

zack requested changes to this revision.Jan 4 2021, 10:49 AM
zack added a subscriber: zack.

I don't like this "asymmetric" approach much. I'd rather have both caches being equal and taking as class constructor input an optional sqlite connection. If it's None, they'll each create a connection (and not share it). If a connection is passed, they use it and do not close it on exit. Then, the init code in FuseCache takes care of initializing the shared sqlite connection, and passing it to both constructors. This will make the classes more reusable and the init code more clean.

This revision now requires changes to proceed.Jan 4 2021, 10:49 AM

Rework AbstractCache class and pass db connection instead of cache conf.

Build is green

Patch application report for D4801 (id=17024)

Rebasing onto 3a76424ac8...

Current branch diff-target is up to date.
Changes applied before test
commit e2c04fc05f590c68e579ae6809ecf5d7140b6f1b
Author: Thibault Allançon <haltode@gmail.com>
Date:   Mon Jan 4 10:20:06 2021 +0100

    cache: share sqlite connection between metadata/history cache
    
    Since metadata and history already share the same database, it makes
    sense to share the connection as well, to prevent unnecessary locking on
    the db file.
    
    Related to T2830.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/256/ for more details.

Use Union[Dict, Connection] to init cache.

Build is green

Patch application report for D4801 (id=17025)

Rebasing onto 3a76424ac8...

Current branch diff-target is up to date.
Changes applied before test
commit 75249aea1698d38d5543e5d814cd1785c0abe567
Author: Thibault Allançon <haltode@gmail.com>
Date:   Mon Jan 4 10:20:06 2021 +0100

    cache: share sqlite connection between metadata/history cache
    
    Since metadata and history already share the same database, it makes
    sense to share the connection as well, to prevent unnecessary locking on
    the db file.
    
    Related to T2830.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/257/ for more details.

Use Optional instead of Union

Build is green

Patch application report for D4801 (id=17026)

Rebasing onto 3a76424ac8...

Current branch diff-target is up to date.
Changes applied before test
commit b59cf2a1d12c607573313c41a798da5658250e8c
Author: Thibault Allançon <haltode@gmail.com>
Date:   Mon Jan 4 10:20:06 2021 +0100

    cache: share sqlite connection between metadata/history cache
    
    Since metadata and history already share the same database, it makes
    sense to share the connection as well, to prevent unnecessary locking on
    the db file.
    
    Related to T2830.

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/258/ for more details.

This revision is now accepted and ready to land.Jan 4 2021, 6:55 PM