Page MenuHomeSoftware Heritage

lister_base: Split into chunks the tasks prior to creation
ClosedPublic

Authored by ardumont on Dec 19 2019, 10:55 AM.

Details

Summary

This should avoid the timeout-ing.

Trade off is that we create multiple transactions instead of one.

This will fix [1]

https://sentry.softwareheritage.org/share/issue/2858dec4b0794ee1a9029a080f6c7bb4/

Related to T2160

Test Plan
  • tox
  • prod: tested, the lister on the same task. It finished without any issue this time.

Diff Detail

Repository
rDLS Listers
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 9888
Build 14600: tox-on-jenkinsJenkins
Build 14599: arc lint + arc unit

Event Timeline

ardumont created this revision.Dec 19 2019, 10:55 AM
ardumont edited the test plan for this revision. (Show Details)Dec 19 2019, 10:56 AM
olasd accepted this revision.Dec 19 2019, 11:35 AM
This revision is now accepted and ready to land.Dec 19 2019, 11:35 AM
ardumont edited the test plan for this revision. (Show Details)Dec 19 2019, 11:48 AM
ardumont updated this revision to Diff 8795.Dec 19 2019, 11:49 AM

Plug to master branch