Page MenuHomeSoftware Heritage

tests: Add test content generation util function
ClosedPublic

Authored by ardumont on Nov 9 2018, 2:55 PM.

Details

Summary

Tests back content_get and content_get_metadata as a first step. Plan
to use those for the new and upcoming content_get_range function.

To be rebased on the current work on storage.

Test Plan

tox

to test only those new tests though:

pytest -v -m property_based ./swh/storage/tests/test_storage.py --disable-warnings

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

@olasd Might take some time to execute indeed.

time pytest -v -m property_based ./swh/storage/tests/test_storage.py --disable-warnings
===================================================================================== test session starts =====================================================================================
platform linux -- Python 3.6.7, pytest-3.9.2, py-1.7.0, pluggy-0.8.0 -- /home/tony/work/inria/repo/swh/swh-environment/.venv/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/tony/work/inria/repo/swh/swh-environment/swh-storage/.hypothesis/examples')
rootdir: /home/tony/work/inria/repo/swh/swh-environment/swh-storage, inifile: pytest.ini
plugins: requests-mock-1.5.2, hypothesis-3.79.3, celery-4.2.1
collected 58 items / 56 deselected

swh/storage/tests/test_storage.py::PropBasedTestStorage::test_generate_content_get PASSED                                                                                               [ 50%]
swh/storage/tests/test_storage.py::PropBasedTestStorage::test_generate_content_get_metadata PASSED                                                                                      [100%]

==================================================================== 2 passed, 56 deselected, 100 warnings in 4.20 seconds ====================================================================
pytest -v -m property_based ./swh/storage/tests/test_storage.py   2.47s user 0.30s system 57% cpu 4.865 total
ardumont added inline comments.
swh/storage/tests/test_storage.py
1887

assertEqual here.

@olasd Might take some time to execute indeed.

This is addressed in D648

vlorentz added inline comments.
swh/storage/tests/__init__.py
37

I find this pretty unclear. Do the returned dicts have any relation with something defined in swh.model?

51

You can build content in a single statement, instead of .update()ing it.

swh/storage/tests/test_storage.py
1859

Nitpick: this code is equivalent and simpler:

expected_list = sorted(c[k] for c in expected_contents)
actual_list = sorted(c[k] for c in actual_contents)
1887

?

vlorentz added inline comments.
swh/storage/tests/test_storage.py
1894

I don't think this properly tests contents. What if contents[0] is an empty dict?

This revision now requires changes to proceed.Nov 14 2018, 2:30 PM
ardumont added inline comments.
swh/storage/tests/__init__.py
37

Yes, it's unclear. Will improve.
but it's definitely content defined per our model (swh.model is the implementation
It's just that there is nothing used from model here.

51

i do not see it, how?

swh/storage/tests/test_storage.py
1859

Right! Thanks.

1887

assertEquals is deprecated ;)
It's a warning from pytest.

1894

That should not be the case since it passes through the content_add, that'd break earlier.

swh/storage/tests/__init__.py
51
content = {
    'data': raw_content,
    'length': len(raw_content),
    'status': 'visible',
    **hashes,
}
ardumont marked 2 inline comments as done.
  • tests: Clarify gen_content docstring
  • tests: Build content generated in less steps
  • tests: Use sorted on list directly
swh/storage/tests/__init__.py
51

Right!

This revision is now accepted and ready to land.Nov 14 2018, 3:22 PM
This revision was automatically updated to reflect the committed changes.