Page MenuHomeSoftware Heritage
Differential D612

swh.indexer: Let task reschedule itself through the scheduler
ClosedPublic
Actions

Authored by ardumont on Oct 26 2018, 8:07 PM.

Details

Reviewers
vlorentz
Commits
rDCIDXad22a05d7bc8: swh.indexer: Let task reschedule itself through the scheduler
rDCIDX796ff0a62594: swh.indexer.rehash: Let task reschedule itself
Summary

... Using the retries_left inherently associated to a scheduled task.

Related T1290

Depends on D609 (~> not really but rebase on it when it's merged ;)

Test Plan

tox

Diff Detail

Repository
rDCIDX Metadata indexer
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Oct 26 2018, 8:07 PM
Harbormaster completed remote builds in B1958: Diff 1870.Oct 26 2018, 8:07 PM
ardumont edited the summary of this revision. (Show Details)Oct 26 2018, 8:10 PM
ardumont added a parent revision: D609: Use swh.scheduler instead of celery in the MimetypeIndexer..
ardumont mentioned this in D614: swh-site: indexer/orchestrator configuration adaptations.Oct 26 2018, 9:23 PM
ardumont added a child revision: D614: swh-site: indexer/orchestrator configuration adaptations.Oct 26 2018, 9:24 PM
ardumont added a child revision: D615: sql/40-swh-data: Update new indexer task types for local db.Oct 26 2018, 9:34 PM
vlorentz requested changes to this revision.Oct 29 2018, 9:09 AM

rescheduling_task is still used in swh/indexer/rehash.py

This revision now requires changes to proceed.Oct 29 2018, 9:09 AM
ardumont updated this revision to Diff 1876.Oct 29 2018, 9:24 AM
  • swh.indexer.rehash: Let task reschedule itself
Harbormaster completed remote builds in B1964: Diff 1876.Oct 29 2018, 9:24 AM
vlorentz accepted this revision.Oct 29 2018, 9:25 AM
This revision is now accepted and ready to land.Oct 29 2018, 9:25 AM
ardumont updated this revision to Diff 1887.Oct 29 2018, 9:59 AM

Rebase

  • swh.indexer: Let task reschedule itself through the scheduler
  • swh.indexer.rehash: Let task reschedule itself
Harbormaster completed remote builds in B1973: Diff 1887.Oct 29 2018, 9:59 AM
Closed by commit rDCIDXad22a05d7bc8: swh.indexer: Let task reschedule itself through the scheduler (authored by ardumont). · Explain WhyOct 29 2018, 10:00 AM
This revision was automatically updated to reflect the committed changes.
ardumont mentioned this in rSPSITE77fd8e9102ad: data/defaults: Let task reschedule itself through the scheduler.Oct 29 2018, 10:08 AM

Revision Contents
Changeset List

PathSize
swh/
indexer/
19 lines
tests/
4 lines
8 lines
3 lines

Diff 1888

View Options

swh/indexer/indexer.py

Loading...
View Options

swh/indexer/tests/test_language.py

Loading...
View Options

swh/indexer/tests/test_metadata.py

Loading...
View Options

swh/indexer/tests/test_mimetype.py

Loading...
About · Developer information · Legal