Changeset View
Standalone View
swh/lister/fedora/tests/test_lister.py
- This file was added.
# Copyright (C) 2022 The Software Heritage developers | ||||||||||||||||
# See the AUTHORS file at the top-level directory of this distribution | ||||||||||||||||
# License: GNU General Public License version 3, or any later version | ||||||||||||||||
# See top-level LICENSE file for more information | ||||||||||||||||
from pathlib import Path | ||||||||||||||||
from unittest.mock import MagicMock | ||||||||||||||||
import defusedxml | ||||||||||||||||
import pytest | ||||||||||||||||
from repomd import Repo | ||||||||||||||||
from requests_mock import Mocker | ||||||||||||||||
from swh.lister.fedora.lister import FedoraLister | ||||||||||||||||
from swh.scheduler.interface import SchedulerInterface | ||||||||||||||||
TRY_GOGS_URL = "https://try.gogs.io/api/v1/" | ||||||||||||||||
mirror_url = "https://eu.edge.kernel.org/fedora/releases" | ||||||||||||||||
@pytest.fixture | ||||||||||||||||
def repomd_mock(requests_mock, datadir, mocker): | ||||||||||||||||
primary_xml = Path(datadir, "https_eu.edge.kernel.org_fedora_36", "primary.xml") | ||||||||||||||||
metadata = defusedxml.lxml.fromstring(primary_xml.read_bytes()) | ||||||||||||||||
vlorentzUnsubmitted Not Done Inline Actions
vlorentz: | ||||||||||||||||
repomd_mock = mocker.patch("swh.lister.fedora.lister.repomd") | ||||||||||||||||
Not Done Inline Actions
vlorentz: | ||||||||||||||||
repomd_mock.load.return_value = Repo(mirror_url, metadata) | ||||||||||||||||
return repomd_mock | ||||||||||||||||
Not Done Inline Actionsplease add a docstring vlorentz: please add a docstring | ||||||||||||||||
def test_lister_fedora36_release( | ||||||||||||||||
swh_scheduler: SchedulerInterface, | ||||||||||||||||
Not Done Inline ActionsCan you mock the backend requests rather than repomd? This allows catching misuses of repomd in our code and/or incompatibility with some versions vlorentz: Can you mock the backend requests rather than repomd? This allows catching misuses of repomd in… | ||||||||||||||||
Done Inline ActionsI tried but stopped since it would be ugly since repomd uses urllib.request.urlopen instead of requests library (requests_mock doesn't work) If you think it's still worth the shot, I'll do it :) KShivendu: I tried but stopped since it would be ugly since `repomd` uses `urllib.request.urlopen` instead… | ||||||||||||||||
Not Done Inline Actionsyes, please. There are already examples of it in swh-loader-core/swh/loader/package/tests/test_utils.py and swh-storage/swh/storage/tests/migrate_extrinsic_metadata/test_pypi.py vlorentz: yes, please.
There are already examples of it in `swh-loader… | ||||||||||||||||
repomd_mock: MagicMock, | ||||||||||||||||
requests_mock: Mocker, | ||||||||||||||||
): | ||||||||||||||||
""" | ||||||||||||||||
Simulate a full listing of main component packages for fedora 36 release. | ||||||||||||||||
""" | ||||||||||||||||
lister = FedoraLister( | ||||||||||||||||
scheduler=swh_scheduler, | ||||||||||||||||
url=mirror_url, | ||||||||||||||||
releases=["36"], | ||||||||||||||||
) | ||||||||||||||||
stats = lister.run() | ||||||||||||||||
assert stats.pages == 1 # == no. of requested releases | ||||||||||||||||
assert stats.origins == 14 | ||||||||||||||||
Done Inline Actionsyou should test the actual values too vlorentz: you should test the actual values too | ||||||||||||||||
Not Done Inline ActionsI don't understand why changing pkg_versions affects the lister vlorentz: I don't understand why changing `pkg_versions` affects the lister | ||||||||||||||||
Done Inline ActionsChanging pkg._version_info.set("ver", "1.0.0") affected it (not pkg_versions) KShivendu: Changing `pkg._version_info.set("ver", "1.0.0")` affected it (not `pkg_versions`) | ||||||||||||||||
Not Done Inline ActionsAh, I see. Thanks for the extra comments vlorentz: Ah, I see. Thanks for the extra comments | ||||||||||||||||
Not Done Inline ActionsPlease test the full URLs; and make it a list equality to pytest can display diffs, instead of length mismatches. And you are missing assertions for other attributes of listed origins (esp. extra_loader_arguments) vlorentz: Please test the full URLs; and make it a list equality to pytest can display diffs, instead of… | ||||||||||||||||
Not Done Inline ActionsInstead of reusing the same lister instance, you should create a new one each time you want to execute a listing, def run_lister( swh_scheduler: SchedulerInterface, releases: List[Release], pkg_versions: dict, origin_count: int, ): """Runs the lister and tests that the listed origins are correct.""" lister = FedoraLister( scheduler=swh_scheduler, releases=releases, ) anlambert: Instead of reusing the same lister instance, you should create a new one each time you want to… |