Replace task tests (in test_task.py) with pytest-based ones
ClosedPublic
Actions

Authored by douardda on Jan 15 2019, 11:36 AM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Commits

rDSCHe1c25d2c0b5a: Replace task tests (in test_task.py) with pytest-based ones
rDSCHe741c936926a: Add a 'swh_scheduler' pytest fixture that creates a swh scheduler usable for…

Summary

Add a 'scheduler' pytest fixture that created a scheduler usable for tests
which uses the pytest-postgresql package to manage the database life cycle.

Also remove now useless celery_testing.py and scheduler_testing.py files.

depends on D937

Diff Detail

Repository

rDSCH Scheduling utilities

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

douardda created this revision.Jan 15 2019, 11:36 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptJan 15 2019, 11:36 AM

Harbormaster failed remote builds in B3481: Diff 2985!Jan 15 2019, 11:37 AM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tox/97/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tox/97/console

douardda mentioned this in T1394: Make swh/indexer/tests/test_origin_metadata.py run faster..Jan 15 2019, 11:42 AM

Also remove now useless celery_testing.py and scheduler_testing.py files.

They are used in archiver and indexer tests.

Rename the fixture as swh_scheduler

Harbormaster failed remote builds in B3487: Diff 2990!Jan 15 2019, 2:42 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tox/98/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tox/98/console

douardda added a child revision: D952: Rewrite test_origin_metadata.py using pytest's fixtures.Jan 15 2019, 3:21 PM

backend_tasks = run_ready_tasks(swh_scheduler, swh_app)
assert backend_tasks
for task in backend_tasks:
    # Make sure the task completed
    AsyncResult(id=task['backend_id']).get()

Looks like a recurring pattern. Could you write a wrapper of run_ready_tasks to do it? (or add an argument to run_ready_tasks)

In D948#20282, @vlorentz wrote:
backend_tasks = run_ready_tasks(swh_scheduler, swh_app)
assert backend_tasks
for task in backend_tasks:
    # Make sure the task completed
    AsyncResult(id=task['backend_id']).get()
Looks like a recurring pattern. Could you write a wrapper of run_ready_tasks to do it? (or add an argument to run_ready_tasks)

Yes I have this on my todo list, but I was not sure what was really needed here.
I plan to have this done in a future refactoring, along with a more gemeric swh_scheduler fixture.

vlorentz accepted this revision.Jan 15 2019, 4:31 PM

This revision is now accepted and ready to land.Jan 15 2019, 4:31 PM

typo in commit msg

Closed by commit rDSCHe741c936926a: Add a 'swh_scheduler' pytest fixture that creates a swh scheduler usable for… (authored by douardda). · Explain WhyJan 15 2019, 5:40 PM

This revision was automatically updated to reflect the committed changes.

Harbormaster failed remote builds in B3541: Diff 3040!Jan 15 2019, 5:41 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DSCH/job/tox/103/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCH/job/tox/103/console