Page MenuHomeSoftware Heritage

test_journal_client: Migrate away from mocks
ClosedPublic

Authored by ardumont on Dec 1 2020, 5:57 PM.

Details

Summary

Related to T2821
Depends on D4638

Diff Detail

Repository
rDCIDX Metadata indexer
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D4640 (id=16469)

Could not rebase; Attempt merge onto 28ae49da34...

Updating 28ae49d..305b265
Fast-forward
 conftest.py                               |   6 +-
 requirements-swh.txt                      |   2 +-
 swh/indexer/storage/db.py                 |   4 +-
 swh/indexer/tests/conftest.py             |  76 +++++++-----
 swh/indexer/tests/test_cli.py             | 189 ++++++++++++++++++------------
 swh/indexer/tests/test_journal_client.py  | 172 ++++++++++++---------------
 swh/indexer/tests/test_origin_head.py     |  34 ++----
 swh/indexer/tests/test_origin_metadata.py |  89 +++++++++-----
 8 files changed, 321 insertions(+), 251 deletions(-)
Changes applied before test
commit 305b265359e503459b78ec6de43b64fb0b31a511
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Dec 1 17:55:36 2020 +0100

    test_journal_client: Migrate away from mocks
    
    Related to D4638

commit 394576b95688b867a86a5bcb4da30534640a0577
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Dec 1 15:40:01 2020 +0100

    tests: Use production backends within the indexer tests
    
    This detected some paper cuts within cli tests for example.
    
    The main goal is to decrease friction when actually deploying indexer related
    services (backend, indexers, ...).
    
    The pg backends tests should still be reasonably fast as it's using the
    swh.core.db.pytest_plugin (which truncate tables in between tests).
    
    Related to T2821

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/132/ for more details.

vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/indexer/tests/test_journal_client.py
17

this function doesn't compare; it only excludes keys from its result

This revision is now accepted and ready to land.Dec 2 2020, 5:23 PM
ardumont added inline comments.
swh/indexer/tests/test_journal_client.py
17

yes, to avoid the comparison which would not work (well for some fields at least [1]) for those fields so I just wanted it to make it clear...

any better name?

[1] id is generated, ...

ardumont marked an inline comment as done.

Rebase

Build is green

Patch application report for D4640 (id=16503)

Could not rebase; Attempt merge onto 28ae49da34...

Updating 28ae49d..f775fea
Fast-forward
 conftest.py                               |   8 +-
 requirements-swh.txt                      |   2 +-
 swh/indexer/storage/db.py                 |   4 +-
 swh/indexer/tests/conftest.py             |  76 +++++++-----
 swh/indexer/tests/test_cli.py             | 189 ++++++++++++++++++------------
 swh/indexer/tests/test_journal_client.py  | 172 ++++++++++++---------------
 swh/indexer/tests/test_origin_head.py     |  38 ++----
 swh/indexer/tests/test_origin_metadata.py |  89 +++++++++-----
 8 files changed, 322 insertions(+), 256 deletions(-)
Changes applied before test
commit f775fea12b828fcee622af4da86cb2598a41be30
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Dec 1 17:55:36 2020 +0100

    test_journal_client: Migrate away from mocks
    
    Related to D4638

commit 2e73f3be6a1cc9d66c88a37e817e042aa60db0f5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Dec 1 15:40:01 2020 +0100

    tests: Use production backends within the indexer tests
    
    This detected some paper cuts within cli tests for example.
    
    The main goal is to decrease friction when actually deploying indexer related
    services (backend, indexers, ...).
    
    The pg backends tests should still be reasonably fast as it's using the
    swh.core.db.pytest_plugin (which truncate tables in between tests).
    
    Related to T2821

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/135/ for more details.