Page MenuHomeSoftware Heritage
Differential D4300

Install postgresql_fact fixture for faster postgres tests
ClosedPublic
Actions

Authored by ardumont on Oct 19 2020, 10:04 AM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Commits
rDCORE0c01f4a09284: Install postgresql_fact fixture for faster postgres tests
Summary

This moves the SWHDatabaseJanitor from swh.storage to swh.core. This also adds
tests to it.

This will allow to share the faster behavior on other swh services with
postgres backends (vault, scheduler, etc...).

Test Plan

tox

Diff Detail

Event Timeline

ardumont created this revision.Oct 19 2020, 10:04 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptOct 19 2020, 10:04 AM
swh-public-ci added a comment.Oct 19 2020, 10:05 AM

Build is green

Patch application report for D4300 (id=15188)

Could not rebase; Attempt merge onto 1657fb827f...

Updating 1657fb8..4d11b30
Fast-forward
 mypy.ini                                           |   3 +
 requirements-db.txt                                |   1 +
 requirements-test-db.txt                           |   2 +-
 swh/core/api/asynchronous.py                       | 128 +++++++++++++++++---
 swh/core/api/tests/conftest.py                     |   3 +
 swh/core/api/tests/test_async.py                   |  38 +++---
 swh/core/api/tests/test_rpc_server_asynchronous.py | 132 +++++++++++++++++++++
 swh/core/db/db_utils.py                            | 104 +++++++++++++++-
 swh/core/db/pytest_plugin.py                       |  59 +++++++++
 swh/core/db/tests/conftest.py                      |   2 +
 swh/core/db/tests/data/0-schema.sql                |  13 ++
 swh/core/db/tests/data/1-data.sql                  |  10 ++
 swh/core/db/tests/test_db_utils.py                 |  56 +++++++++
 tox.ini                                            |   1 +
 14 files changed, 509 insertions(+), 43 deletions(-)
 create mode 100644 swh/core/api/tests/conftest.py
 create mode 100644 swh/core/api/tests/test_rpc_server_asynchronous.py
 create mode 100644 swh/core/db/pytest_plugin.py
 create mode 100644 swh/core/db/tests/data/0-schema.sql
 create mode 100644 swh/core/db/tests/data/1-data.sql
 create mode 100644 swh/core/db/tests/test_db_utils.py
Changes applied before test
commit 4d11b3055cfea7f9271d02dff9940c9cf13b67d5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:46:39 2020 +0200

    Install postgresql_fact fixture for faster postgres tests
    
    This moves the SWHDatabaseJanitor from swh.storage to swh.core. This also adds
    tests to it.
    
    This will allow to share the behavior on other swh services with postgres
    backends (vault, scheduler, etc...).

commit 604a08cbe80439b35ce0197805b6a4c536d96e62
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sat Oct 17 19:06:57 2020 +0200

    asynchronous.RPCServerApp: Align implementation with api.RPCServerApp
    
    This will allow to align the Vault's client/server code with the other rpc
    servers. And start adding one typed interface with more consistent checks in
    between implementations.

commit 7f928a07d91460efbbfbc6c5264a4fdd60e58be2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 10:25:01 2020 +0200

    api.tests.test_async: Simplify fixture setup

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/94/ for more details.

Harbormaster completed remote builds in B16425: Diff 15188.Oct 19 2020, 10:05 AM
vlorentz added a subscriber: vlorentz.Oct 19 2020, 12:36 PM
vlorentz added inline comments.
swh/core/db/db_utils.py
323

not covered, according to phab

ardumont added inline comments.Oct 19 2020, 4:10 PM
swh/core/db/db_utils.py
323

yes, i'm gonna look at how to add sequence in there.

ardumont updated this revision to Diff 15219.Oct 19 2020, 4:24 PM

Test the sequences are reset as well

swh-public-ci added a comment.Oct 19 2020, 4:25 PM

Build is green

Patch application report for D4300 (id=15219)

Rebasing onto 7f928a07d9...

Current branch diff-target is up to date.
Changes applied before test
commit 966a40142e23d8e6a258991055e65392e9acd3f7
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:46:39 2020 +0200

    Install postgresql_fact fixture for faster postgres tests
    
    This moves the SWHDatabaseJanitor from swh.storage to swh.core. This also adds
    tests to it.
    
    This will allow to share the behavior on other swh services with postgres
    backends (vault, scheduler, etc...).

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/97/ for more details.

Harbormaster completed remote builds in B16454: Diff 15219.Oct 19 2020, 4:25 PM
ardumont updated this revision to Diff 15234.Oct 19 2020, 6:32 PM
  • Fix typo in docstring
  • Add other dostrings
swh-public-ci added a comment.Oct 19 2020, 6:33 PM

Build is green

Patch application report for D4300 (id=15234)

Rebasing onto 7f928a07d9...

Current branch diff-target is up to date.
Changes applied before test
commit 6b311c8f7794a548c68c786e5adb79dff609843a
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:46:39 2020 +0200

    Install postgresql_fact fixture for faster postgres tests
    
    This moves the SWHDatabaseJanitor from swh.storage to swh.core. This also adds
    tests to it.
    
    This will allow to share the behavior on other swh services with postgres
    backends (vault, scheduler, etc...).

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/98/ for more details.

Harbormaster completed remote builds in B16467: Diff 15234.Oct 19 2020, 6:33 PM
ardumont mentioned this in D4310: vault.tests: Make postgresql fixture faster.Oct 19 2020, 7:05 PM
ardumont updated this revision to Diff 15287.Oct 21 2020, 6:04 PM

Add coverage and docstring

swh-public-ci added a comment.Oct 21 2020, 6:05 PM

Build is green

Patch application report for D4300 (id=15287)

Rebasing onto 7f928a07d9...

Current branch diff-target is up to date.
Changes applied before test
commit 0c01f4a09284b8843ac238f61f0a9f3ba3a0cd7a
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:46:39 2020 +0200

    Install postgresql_fact fixture for faster postgres tests
    
    This moves the SWHDatabaseJanitor from swh.storage to swh.core. This also adds
    tests to it.
    
    This will allow to share the behavior on other swh services with postgres
    backends (vault, scheduler, etc...).

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/101/ for more details.

Harbormaster completed remote builds in B16518: Diff 15287.Oct 21 2020, 6:05 PM
vlorentz accepted this revision.Oct 22 2020, 10:53 AM
This revision is now accepted and ready to land.Oct 22 2020, 10:53 AM
Closed by commit rDCORE0c01f4a09284: Install postgresql_fact fixture for faster postgres tests (authored by ardumont). · Explain WhyOct 22 2020, 11:09 AM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDCORE0c01f4a09284: Install postgresql_fact fixture for faster postgres tests.
ardumont mentioned this in D4332: requirements-swh: Bump requirements to fix webapp build.Oct 22 2020, 3:58 PM

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
3 lines--
1 line--
2 lines--
swh/
core/
db/
104 lines35%
Loading...
59 lines78%
Loading...
tests/
2 lines100%
Loading...
data/
13 lines--
10 lines--
56 lines100%
Loading...

Diff 15188

View Options

mypy.ini

Loading...
View Options

requirements-db.txt

Loading...
View Options

requirements-test-db.txt

Loading...
View Options

swh/core/db/db_utils.py

Loading...
View Options

swh/core/db/pytest_plugin.py

Loading...
View Options

swh/core/db/tests/conftest.py

Loading...
View Options

swh/core/db/tests/data/0-schema.sql

Loading...
View Options

swh/core/db/tests/data/1-data.sql

Loading...
View Options

swh/core/db/tests/test_db_utils.py

Loading...
About · Developer information · Legal