Page MenuHomeSoftware Heritage

Make base indexers generic, with the result of index() as their type parameter.
ClosedPublic

Authored by vlorentz on Oct 6 2020, 3:44 PM.

Details

Summary

So the type of results can be statically checked, instead of needing to
assert it to please mypy.

Depends on D4158.

Diff Detail

Repository
rDCIDX Metadata indexer
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D4159 (id=14662)

Could not rebase; Attempt merge onto 51b10e891d...

Updating 51b10e8..44cee8f
Fast-forward
 swh/indexer/ctags.py                        |   2 +-
 swh/indexer/fossology_license.py            |   4 +-
 swh/indexer/indexer.py                      |  30 +++--
 swh/indexer/metadata.py                     |   6 +-
 swh/indexer/mimetype.py                     |  21 ++--
 swh/indexer/origin_head.py                  |   4 +-
 swh/indexer/storage/__init__.py             |  26 +++--
 swh/indexer/storage/api/client.py           |   3 +
 swh/indexer/storage/api/serializers.py      |  26 +++++
 swh/indexer/storage/api/server.py           |   9 +-
 swh/indexer/storage/db.py                   |  18 ++-
 swh/indexer/storage/in_memory.py            |  15 ++-
 swh/indexer/storage/interface.py            |  25 ++--
 swh/indexer/tests/storage/conftest.py       |   3 +-
 swh/indexer/tests/storage/test_storage.py   | 169 ++++++++++++++++++----------
 swh/indexer/tests/test_fossology_license.py |  16 +++
 swh/indexer/tests/test_mimetype.py          |   4 +-
 swh/indexer/tests/utils.py                  |  16 ++-
 18 files changed, 264 insertions(+), 133 deletions(-)
 create mode 100644 swh/indexer/storage/api/serializers.py
Changes applied before test
commit 44cee8f213f26e01aa63519a8c21962b2fca460b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 6 15:42:54 2020 +0200

    Make base indexers generic, with the result of index() as their type parameter.
    
    So the type of results can be statically checked, instead of needing to
    assert it to please mypy.

commit c3caf300830030f0cd1b3002d715e430582db8f1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 6 15:35:39 2020 +0200

    mimetype: use ContentMimetypeRow in the storage interface instead of dicts.
    
    This temporarily adds mess in the generic tests to support both rows and dicts,
    but I'll remove it once I migrated all endpoints.

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/68/ for more details.

ardumont added a subscriber: ardumont.

That's nice!

I don't like much TResult, RowResult may be better? In any case, i won't fight this.

Also you mention instead of "needing to assert to please mypy", but i don't see any assertion drop instructions here ;)

Anyway, thanks.

This revision is now accepted and ready to land.Oct 6 2020, 5:08 PM

I don't like much TResult, RowResult may be better? In any case, i won't fight this.

Except it's not always a row, it can also be a dict.

Also you mention instead of "needing to assert to please mypy", but i don't see any assertion drop instructions here ;)

Look better, there's one!

I don't like much TResult, RowResult may be better? In any case, i won't fight this.

Except it's not always a row, it can also be a dict.

But you aim at removing those, aren't you?
:D

(I imagine you could have planned to rename it appropriately later ;)

Also you mention instead of "needing to assert to please mypy", but i don't see any assertion drop instructions here ;)

Look better, there's one!

\o/