Page MenuHomeSoftware Heritage

tox: split tests by 'extra' subpackage
ClosedPublic

Authored by douardda on May 7 2019, 10:43 AM.

Details

Summary

so each 'extra' subpackage is tested in a minimal venv (with minimal
dependencies installed).

This requires to remove extras-needed depencies from the main requirements.txt

Warning: this may break swh packages that depends on swh.core which have
not been updated to swh.core 0.0.60.

Test Plan

tox should still be happy

Diff Detail

Repository
rDCORE Foundations and core functionalities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

douardda created this revision.May 7 2019, 10:43 AM
vlorentz added a comment.EditedMay 7 2019, 11:40 AM

Instead of swh/core/X/tests/, you should put tests in swh/core/tests/X/ (where X is the sub-component), for uniformity with other repos (deposit/indexer/storage/web)

Build fails with:

_______________ ERROR collecting swh/core/db/tests/test_db.py _________________
swh/core/db/tests/test_db.py:14: in <module>
    from .db_testing import (
E   SystemError: Parent module '' not loaded, cannot perform relative import
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=

Wondering if we are not missing __init__.py files in the new subfolders created (db, api, server)?

douardda added a comment.EditedMay 7 2019, 2:34 PM

Build fails with:

_______________ ERROR collecting swh/core/db/tests/test_db.py _________________
swh/core/db/tests/test_db.py:14: in <module>
    from .db_testing import (
E   SystemError: Parent module '' not loaded, cannot perform relative import
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=

Wondering if we are not missing __init__.py files in the new subfolders created (db, api, server)?

Indeed I forget to git add a few of those

Instead of swh/core/X/tests/, you should put tests in swh/core/tests/X/ (where X is the sub-component), for uniformity with other repos (deposit/indexer/storage/web)

let me check that

douardda updated this revision to Diff 4723.May 7 2019, 2:38 PM

Add missing __init__.py files

Instead of swh/core/X/tests/, you should put tests in swh/core/tests/X/ (where X is the sub-component), for uniformity with other repos (deposit/indexer/storage/web)

let me check that

Actually it's not without a drawback: if I move these test dirs as proposed, I won't be able to easily execute basic swh.core tests alone.
With:

tests/
├── api
│   ├── __init__.py
│   ├── server_testing.py
│   ├── test_api.py
│   └── test_serializers.py
├── db
│   ├── conftest.py
│   ├── db_testing.py
│   ├── __init__.py
│   └── test_db.py
├── __init__.py
├── test_config.py
├── test_statsd.py
└── test_utils.py

I cannot easily write the testenv:py3-core tox section (and yes, I know the --ignore option for pytest). Which means I need to move 'core' tests (aka test_config.py, test_statsd.py and test_utils.py) in a sub directory, which I find quite clumsy.

To be honest, since we supply tests as part of the code (ie. tests are subpackages shipped with the code), I prefer the pattern:

packages/
├── tests/
├── subpackage1/
│   └── tests/
└── subpackage2
    └── tests/

than what we mainly use today. It keeps the tests close to the tested code and make a potential extraction/refactoring a bit easier. But meh/

ardumont accepted this revision.May 7 2019, 3:20 PM

Looks good to me

This revision is now accepted and ready to land.May 7 2019, 3:20 PM
This revision was landed with ongoing or failed builds.May 16 2019, 2:56 PM
This revision was automatically updated to reflect the committed changes.