Changeset View
Standalone View
swh/lister/launchpad/tests/test_lister.py
- This file was added.
from datetime import datetime | |||||
from unittest.mock import MagicMock | |||||
import json | |||||
def test_launchpad_lister(swh_listers): | |||||
class Collection: | |||||
entries = [] | |||||
ardumont: I did not fully realize earlier.
Does that test currently load a real launchpad instance?
(i… | |||||
Done Inline ActionsYes it does, what would you recommend to use to mock launchpadlib data ? legau: Yes it does, what would you recommend to use to mock launchpadlib data ? | |||||
Done Inline ActionsWe are using pytest and this allows to define fixtures like requests_mock_datadir. So as a first step, i'd look if there is any existing fixture around launchpadlib [1] If there is no fixture yet, you can try to define one which would mock the launchpadlib calls to return a fixed dataset of repositories to list. Just define a json file in the "tests/data/launchpadlib/dataset.json" and make the fixture returns it. Then in the tests, you can further ensure the data listed and the test dataset are consistent together. We use pytest-mocker for that, for example [2] [1] https://pytest.readthedocs.io/en/2.7.3/plugins_index/index.html ardumont: We are using pytest and this allows to define fixtures like requests_mock_datadir.
So as a… | |||||
def __init__(self, file): | |||||
self.entries = [Repo(r) for r in file] | |||||
def __getitem__(self, key): | |||||
return self.entries[key] | |||||
class Repo: | |||||
def __init__(self, d: dict): | |||||
for key in d.keys(): | |||||
if key == "date_last_modified": | |||||
setattr(self, key, datetime.fromisoformat(d[key])) | |||||
else: | |||||
setattr(self, key, d[key]) | |||||
def mock_lp_response(page) -> Collection: | |||||
with open('swh/lister/launchpad/tests/data/response%s.json' | |||||
Not Done Inline Actionswe have a datadir fixture [1] for that, "inject" datadir in the test signature and: response_filepath = os.path.join(datadir, f'response{page}.json') with open(response_filepath, encoding='utf-8') as f: ... ` [1] https://forge.softwareheritage.org/source/swh-core/browse/master/swh/core/pytest_plugin.py$0-121 ardumont: we have a `datadir` fixture [1] for that, "inject" datadir in the test signature and:
```… | |||||
% page, 'r', encoding='utf-8') as f: | |||||
return Collection(json.load(f)) | |||||
lister = swh_listers['launchpad'] | |||||
lister.launchpad.git_repositories.getRepositories = MagicMock( | |||||
side_effect=[mock_lp_response(i) for i in range(3)]) | |||||
lister.run() | |||||
assert len(lister.launchpad | |||||
.git_repositories.getRepositories.mock_calls) == 3 | |||||
r = lister.scheduler.search_tasks(task_type='load-git') | |||||
assert len(r) == 90 | |||||
for row in r: | |||||
assert row['type'] == 'load-git' | |||||
# arguments check | |||||
args = row['arguments']['args'] | |||||
assert len(args) == 0 | |||||
# kwargs | |||||
kwargs = row['arguments']['kwargs'] | |||||
assert set(kwargs.keys()) == {'url'} | |||||
url = kwargs['url'] | |||||
assert url.startswith('https://git.launchpad.net') | |||||
assert row['policy'] == 'recurring' | |||||
assert row['priority'] is None | |||||
assert row['retries_left'] == 0 |
I did not fully realize earlier.
Does that test currently load a real launchpad instance?
(i believe so and the diff description seems to agree ;)
I don't see the data test file.
requests_mock_datadir is a fixture to transform http requests into lookup within the ./data/<https_server-name>/remaining-file-named-after-remaining-http-query-parameters...