Page MenuHomeSoftware Heritage

Add table sampling option to grab_next_visits
ClosedPublic

Authored by olasd on Aug 26 2021, 4:48 PM.

Details

Summary

Running common operations on all git origins is pretty intense. Using
table sampling gives us the opportunity to at least schedule some jobs
in (decently small) time.

Test Plan

smoke test added; not easy to test the *exact* table sampling behavior reliably

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D6145 (id=22244)

Rebasing onto cc76a573e7...

Current branch diff-target is up to date.
Changes applied before test
commit 5e8007fdbfeb612ea394f97eeba25a1c4e529b7e
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Jun 1 15:48:05 2021 +0200

    Add table sampling option to grab_next_visits
    
    Running common operations on all git origins is pretty intense. Using
    table sampling gives us the opportunity to at least schedule some jobs
    in (decently small) time.

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/446/ for more details.

olasd requested review of this revision.Aug 26 2021, 4:51 PM
This revision is now accepted and ready to land.Aug 26 2021, 4:51 PM