Page MenuHomeSoftware Heritage

Replace task tests (in test_task.py) with pytest-based ones
ClosedPublic

Authored by douardda on Jan 15 2019, 11:36 AM.

Details

Summary

Add a 'scheduler' pytest fixture that created a scheduler usable for tests
which uses the pytest-postgresql package to manage the database life cycle.

Also remove now useless celery_testing.py and scheduler_testing.py files.

depends on D937

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

douardda created this revision.Jan 15 2019, 11:36 AM

Also remove now useless celery_testing.py and scheduler_testing.py files.

They are used in archiver and indexer tests.

douardda updated this revision to Diff 2990.Jan 15 2019, 2:42 PM

Rename the fixture as swh_scheduler

backend_tasks = run_ready_tasks(swh_scheduler, swh_app)
assert backend_tasks
for task in backend_tasks:
    # Make sure the task completed
    AsyncResult(id=task['backend_id']).get()

Looks like a recurring pattern. Could you write a wrapper of run_ready_tasks to do it? (or add an argument to run_ready_tasks)

douardda added a comment.EditedJan 15 2019, 4:31 PM
backend_tasks = run_ready_tasks(swh_scheduler, swh_app)
assert backend_tasks
for task in backend_tasks:
    # Make sure the task completed
    AsyncResult(id=task['backend_id']).get()

Looks like a recurring pattern. Could you write a wrapper of run_ready_tasks to do it? (or add an argument to run_ready_tasks)

Yes I have this on my todo list, but I was not sure what was really needed here.
I plan to have this done in a future refactoring, along with a more gemeric swh_scheduler fixture.

vlorentz accepted this revision.Jan 15 2019, 4:31 PM
This revision is now accepted and ready to land.Jan 15 2019, 4:31 PM
douardda updated this revision to Diff 3040.Jan 15 2019, 5:40 PM

typo in commit msg

This revision was automatically updated to reflect the committed changes.