Differential D4744

FUSE: cache: update cache with new origin visits
ClosedPublic
Actions

Authored by haltode on Dec 15 2020, 2:54 PM.

Details

Reviewers

zack

Group Reviewers

Reviewers

Maniphest Tasks

T2841: FUSE: update cache with new origin visits

Commits

rDFUSEa6cef6bad56e: cache: update cache with new origin visits
rDFUSEb6e8cf744f3e: cache: add primary key to db tables

Summary

Closes T2841.

Force a cache update on origin visits when last fetched more than one
day ago.

Diff Detail

Repository

rDFUSE FUSE virtual file system

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

haltode created this revision.Dec 15 2020, 2:54 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptDec 15 2020, 2:54 PM

Build is green

Patch application report for D4744 (id=16799)

Rebasing onto 7154e451ff...

Current branch diff-target is up to date.

Changes applied before test

commit a2ce5ab4fa491e4fd1865cc0b0deefef75d7eeaf
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Dec 15 14:20:48 2020 +0100

    cache: update cache with new origin visits
    
    Closes T2841.
    
    Force a cache update on origin visits when last fetched more than one
    day ago.

commit dd96749820b159fc85d4a4608a7862d2458d7d58
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Dec 15 14:47:51 2020 +0100

    cache: add primary key to db tables

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/238/ for more details.

Harbormaster completed remote builds in B17958: Diff 16799.Dec 15 2020, 2:58 PM

zack requested changes to this revision.Dec 16 2020, 12:12 PM

zack added a subscriber: zack.

zack added inline comments.

swh/fuse/cache.py
120–121	is this needed now that you're making the "swhid" column a primary key? please check, and remove it if not needed anymore
125	Minor naming issue: this is more like "fetch" than "last_fetched", because due to the primary key you will never have more than one entries for the same URL. All in all, something with "time" in the name would be more idiomatic, like "itime" (for "insertion time [in cache]").
125–126	ditto
145	not sure why you want to make timestamp checking optional (do we have a use case for that?), but if you really do make the default True, because that's the sane cache semantics

This revision now requires changes to proceed.Dec 16 2020, 12:12 PM

Rebase on master
Fix zack comments

swh/fuse/cache.py
120–121	Indeed, unique and primary keys already have an index associated (https://stackoverflow.com/questions/3379292/is-an-index-needed-for-a-primary-key-in-sqlite). Thanks for the catch!

Build is green

Patch application report for D4744 (id=16826)

Rebasing onto 9546ba2f7b...

Current branch diff-target is up to date.

Changes applied before test

commit a6cef6bad56e709fff28a1287031a481e37c097b
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Dec 15 14:20:48 2020 +0100

    cache: update cache with new origin visits
    
    Closes T2841.
    
    Force a cache update on origin visits when last fetched more than one
    day ago.

commit b6e8cf744f3e4c1e16fd1eca6d4ae98629d39613
Author: Thibault Allançon <haltode@gmail.com>
Date:   Tue Dec 15 14:47:51 2020 +0100

    cache: add primary key to db tables

See https://jenkins.softwareheritage.org/job/DFUSE/job/tests-on-diff/240/ for more details.

Harbormaster completed remote builds in B17982: Diff 16826.Dec 16 2020, 2:50 PM

zack accepted this revision.Dec 16 2020, 4:19 PM

This revision is now accepted and ready to land.Dec 16 2020, 4:19 PM

Closed by commit rDFUSEb6e8cf744f3e: cache: add primary key to db tables (authored by haltode). · Explain WhyDec 16 2020, 4:24 PM

This revision was automatically updated to reflect the committed changes.

haltode added a commit: rDFUSEb6e8cf744f3e: cache: add primary key to db tables.

haltode added a commit: rDFUSEa6cef6bad56e: cache: update cache with new origin visits.