Depends on D4174.
Details
Details
- Reviewers
ardumont - Group Reviewers
Reviewers - Maniphest Tasks
- T2652: Make the indexer-storage interface use attr classes instead of dicts
- Commits
- rDCIDXaf3c220b14e2: use ContentLanguageRow in the storage interface instead of dicts.
Diff Detail
Diff Detail
- Repository
- rDCIDX Metadata indexer
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
Build is green
Patch application report for D4176 (id=14690)
Could not rebase; Attempt merge onto 51b10e891d...
Updating 51b10e8..af3c220 Fast-forward swh/indexer/ctags.py | 4 +- swh/indexer/fossology_license.py | 25 +- swh/indexer/indexer.py | 57 ++-- swh/indexer/metadata.py | 38 ++- swh/indexer/mimetype.py | 31 +- swh/indexer/origin_head.py | 15 +- swh/indexer/storage/__init__.py | 213 ++++++------ swh/indexer/storage/api/client.py | 3 + swh/indexer/storage/api/serializers.py | 26 ++ swh/indexer/storage/api/server.py | 9 +- swh/indexer/storage/converters.py | 15 +- swh/indexer/storage/db.py | 34 +- swh/indexer/storage/in_memory.py | 111 +++---- swh/indexer/storage/interface.py | 93 +++--- swh/indexer/tests/storage/conftest.py | 10 +- swh/indexer/tests/storage/test_converters.py | 17 +- swh/indexer/tests/storage/test_metrics.py | 8 +- swh/indexer/tests/storage/test_server.py | 14 +- swh/indexer/tests/storage/test_storage.py | 462 ++++++++++++++++----------- swh/indexer/tests/test_ctags.py | 25 +- swh/indexer/tests/test_fossology_license.py | 22 +- swh/indexer/tests/test_indexer.py | 4 +- swh/indexer/tests/test_mimetype.py | 46 +-- swh/indexer/tests/utils.py | 72 +---- 24 files changed, 749 insertions(+), 605 deletions(-) create mode 100644 swh/indexer/storage/api/serializers.py
Changes applied before test
commit af3c220b14e22ae270679511045f1f6563febe9b Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Oct 7 12:43:45 2020 +0200 use ContentLanguageRow in the storage interface instead of dicts. commit b4d084d675bb89f5d6803ddbf75bbe54c3e91c03 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Oct 7 11:25:05 2020 +0200 indexer.storage: Change return types from Iterable to List For consistency with the main storage. commit 570816fd87a1451e560fce8881a0da5626cf0ab4 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Oct 7 11:06:15 2020 +0200 license: use ContentLicenseRow in the storage interface instead of dicts. commit d346feffffaac7e8cc5c3bc0eedbffc66db1d37e Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Oct 7 11:43:05 2020 +0200 all indexers: make index() return a list of results instead of a single one. 1. it was wrongfully annotated as '-> TResult' even though some indexers can return None 2. in a future commit, the fossology indexer will need to return multiple results. commit 7fe4a89dbcf221a0125c474640cbcdc8b01b1df2 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Oct 7 10:54:35 2020 +0200 base indexers: add type annotation for self.{storage,idx_storage}. commit 486ee085f5ee999da38793815ae462c00ece4efb Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Oct 7 10:48:44 2020 +0200 indexer.storage: Update docstrings of mimetype-related endpoints. commit 4ec112337909cc364bec119b2ef6ae047a81b96f Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Oct 7 10:46:07 2020 +0200 indexer.storage: Change return type annotation from Iterator to Iterable. When going through the RPC, it's turned into a list. commit e8e94cf237471636e84ceac5d1740cfa7a982f90 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Oct 7 10:01:26 2020 +0200 tests: Enable type-checking on storage test functions. By adding a simple type annotation to the test functions' signature. commit 44cee8f213f26e01aa63519a8c21962b2fca460b Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Tue Oct 6 15:42:54 2020 +0200 Make base indexers generic, with the result of index() as their type parameter. So the type of results can be statically checked, instead of needing to assert it to please mypy. commit c3caf300830030f0cd1b3002d715e430582db8f1 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Tue Oct 6 15:35:39 2020 +0200 mimetype: use ContentMimetypeRow in the storage interface instead of dicts. This temporarily adds mess in the generic tests to support both rows and dicts, but I'll remove it once I migrated all endpoints.
See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/78/ for more details.