Page MenuHomeSoftware Heritage

Make Storage, Db, and tests more modular.
AbandonedPublic

Authored by vlorentz on Nov 7 2018, 11:26 AM.

Details

Reviewers
None
Group Reviewers
Reviewers
Summary

To prepare for a new backend that doesn't use postgresql. First step toward T1306.

Test Plan

'tox' still runs the same tests, and doesn't fail.

Diff Detail

Repository
rDSTO Storage manager
Branch
modular-pg
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 2255
Build 2711: tox-on-jenkinsJenkins
Build 2710: arc lint + arc unit

Event Timeline

  • CommonTestStorage always needs DataTestStorage.
ardumont added inline comments.
swh/storage/db.py
177

That will need a bump in storage and migration in other modules:

./swh-vault/.tox/py3/lib/python3.6/site-packages/swh/storage/tests/test_db.py:13:from swh.storage.db import Db
./swh-indexer/.tox/py3/lib/python3.6/site-packages/swh/storage/tests/test_db.py:13:from swh.storage.db import Db
./swh-storage/swh/storage/tests/test_db.py:13:from swh.storage.db import Db
./swh-lister/.tox/py3/lib/python3.6/site-packages/swh/storage/tests/test_db.py:13:from swh.storage.db import Db
./swh-journal/.tox/py3/lib/python3.6/site-packages/swh/storage/tests/test_db.py:13:from swh.storage.db import Db
swh/storage/storage.py
270

(e.g file whose size is too large)

swh/storage/tests/storage_testing.py
16

Same as for the db modules.
This implies bump and migration.

  • Really move all pgsql-specific parts of Storage into PgStorage.
  • Apply @ardumont's comment
vlorentz added inline comments.
swh/storage/db.py
177

Do you want me to add an alias (with a DeprecationWarning)?

  • Move apply_options to BaseDb.
swh/storage/db.py
177

It was a jsyk basis.
It may be cleaner to do so though so let's say yes.

I expect the ci to go red fast when this hits master (maybe not though as we use pypi module now).

  • Add aliases with a DeprecationWarning for compatibility.

Abandonned in favor of D635.