Page MenuHomeSoftware Heritage

swh.search: Define an interface for search backends and use it
ClosedPublic

Authored by ardumont on Aug 2 2020, 11:11 AM.

Details

Summary

This also:

  • fixes slight inconsistencies we had.
  • add tests to ensure signatures match (ala storage but in a dedicated tests because fixture hell)

Related to T2516

Test Plan

tox

Diff Detail

Repository
rDSEA Archive search
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Aug 2 2020, 11:11 AM
ardumont edited the summary of this revision. (Show Details)Aug 2 2020, 11:13 AM

Build is green

Patch application report for D3685 (id=12975)

Could not rebase; Attempt merge onto d285cd3f3b...

Updating d285cd3..2cbd837
Fast-forward
 requirements-test.txt               |  1 +
 swh/search/__init__.py              | 53 +++++++++++++++-------
 swh/search/api/client.py            |  6 +--
 swh/search/api/server.py            |  6 +--
 swh/search/elasticsearch.py         |  9 +---
 swh/search/in_memory.py             |  6 +--
 swh/search/interface.py             | 54 ++++++++++++++++++++++-
 swh/search/tests/conftest.py        |  2 +-
 swh/search/tests/test_api_client.py |  6 +--
 swh/search/tests/test_in_memory.py  |  4 +-
 swh/search/tests/test_init.py       | 88 +++++++++++++++++++++++++++++++++++++
 11 files changed, 194 insertions(+), 41 deletions(-)
 create mode 100644 swh/search/tests/test_init.py
Changes applied before test
commit 2cbd837e9a631478f68810528139980d41b5c3a9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Aug 2 11:09:33 2020 +0200

    swh.search: Define an interface for search backends and use it
    
    Related to T645

commit 58faacb5e95dc3bd9a55ec39c8bf06287b75879d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Aug 2 11:05:07 2020 +0200

    swh.search.get_search: Simplify instantiation
    
    This aligns with other get_* functions.
    This also adds tests to this part.
    
    This is preparatory work to improve and type properly this module without
    duplication.
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/31/ for more details.

vlorentz requested changes to this revision.Aug 3 2020, 9:17 AM
vlorentz added a subscriber: vlorentz.

You can remove docstrings from elasticsearch.py.

swh/search/tests/test_init.py
62–66

This once would really benefit from pytest.parametrize.

Or at least switch the two for loops, so we don't instantiate the same class for every method

This revision now requires changes to proceed.Aug 3 2020, 9:17 AM
ardumont updated this revision to Diff 12979.Aug 3 2020, 10:32 AM

Use pytest parametrize to simplify test

ardumont updated this revision to Diff 12980.Aug 3 2020, 10:33 AM

Drop elasticsearch docstrings which are now defined in the interface (I thought
I already did)

Build is green

Patch application report for D3685 (id=12979)

Could not rebase; Attempt merge onto d285cd3f3b...

Updating d285cd3..d040806
Fast-forward
 requirements-test.txt               |  1 +
 swh/search/__init__.py              | 53 ++++++++++++++++-------
 swh/search/api/client.py            |  6 +--
 swh/search/api/server.py            |  6 +--
 swh/search/elasticsearch.py         |  9 +---
 swh/search/in_memory.py             |  6 +--
 swh/search/interface.py             | 54 ++++++++++++++++++++++-
 swh/search/tests/conftest.py        |  2 +-
 swh/search/tests/test_api_client.py |  6 +--
 swh/search/tests/test_in_memory.py  |  4 +-
 swh/search/tests/test_init.py       | 86 +++++++++++++++++++++++++++++++++++++
 11 files changed, 192 insertions(+), 41 deletions(-)
 create mode 100644 swh/search/tests/test_init.py
Changes applied before test
commit d0408062a6889aa3db2eb55cce282da170ec5da3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Aug 2 11:09:33 2020 +0200

    swh.search: Define an interface for search backends and use it
    
    Related to T645

commit 46af49019c220ad4f398944bf43130bd26288cda
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Aug 2 11:05:07 2020 +0200

    swh.search.get_search: Simplify instantiation
    
    This aligns with other get_* functions.
    This also adds tests to this part.
    
    This is preparatory work to improve and type properly this module without
    duplication.
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/34/ for more details.

Build is green

Patch application report for D3685 (id=12980)

Could not rebase; Attempt merge onto d285cd3f3b...

Updating d285cd3..56f9458
Fast-forward
 requirements-test.txt               |  1 +
 swh/search/__init__.py              | 53 ++++++++++++++++-------
 swh/search/api/client.py            |  6 +--
 swh/search/api/server.py            |  6 +--
 swh/search/elasticsearch.py         | 27 +-----------
 swh/search/in_memory.py             |  6 +--
 swh/search/interface.py             | 54 ++++++++++++++++++++++-
 swh/search/tests/conftest.py        |  2 +-
 swh/search/tests/test_api_client.py |  6 +--
 swh/search/tests/test_in_memory.py  |  4 +-
 swh/search/tests/test_init.py       | 86 +++++++++++++++++++++++++++++++++++++
 11 files changed, 192 insertions(+), 59 deletions(-)
 create mode 100644 swh/search/tests/test_init.py
Changes applied before test
commit 56f94586de0683d28559be34b2db791466f5d76c
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Aug 2 11:09:33 2020 +0200

    swh.search: Define an interface for search backends and use it
    
    Related to T645

commit 46af49019c220ad4f398944bf43130bd26288cda
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Aug 2 11:05:07 2020 +0200

    swh.search.get_search: Simplify instantiation
    
    This aligns with other get_* functions.
    This also adds tests to this part.
    
    This is preparatory work to improve and type properly this module without
    duplication.
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/35/ for more details.

vlorentz accepted this revision.Aug 3 2020, 10:34 AM
This revision is now accepted and ready to land.Aug 3 2020, 10:34 AM