Page MenuHomeSoftware Heritage

vault.tests: Make postgresql fixture faster
ClosedPublic

Authored by ardumont on Oct 19 2020, 7:02 PM.

Details

Summary

This reuse the swh.core.db.db_utils postgresql_fact fixture (which inits the db
then truncate tables in between test calls). This makes the tests faster.

Basic benchmark:

|---------+--------------+-----------+-----------------------|
| machine | before (min) | after (s) | Using                 |
|---------+--------------+-----------+-----------------------|
| laptop  |      1:11.02 | 20.811    | `time tox`            |
| jenkins |         1:14 | 26        | jenkins output report |
|---------+--------------+-----------+-----------------------|
Test Plan

tox

Event Timeline

Build has FAILED

Patch application report for D4310 (id=15236)

Could not rebase; Attempt merge onto 8e4026a5e8...

Updating 8e4026a..b457d50
Fast-forward
 swh/vault/api/server.py        | 18 +++++++-----
 swh/vault/tests/conftest.py    | 67 ++++++++++++++++++++----------------------
 swh/vault/tests/test_server.py | 38 +++++++++++++++++++++++-
 3 files changed, 79 insertions(+), 44 deletions(-)
Changes applied before test
commit b457d50603989dd2203894060662f8ccd731ba32
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:54:44 2020 +0200

    vault.tests: Make postgresql fixture faster
    
    This reuse the swh.core.db.db_utils.SwhDatabasejanitor which allows to truncate
    tables in between test calls. This makes the tests faster.

commit a2ffe238d0e59def15ac5dd5a41717f6a5ff63e8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 20:06:00 2020 +0200

    api.server: Add types and tests on configuration checks

Link to build: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/27/
See console output for more information: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/27/console

ardumont edited the summary of this revision. (Show Details)

I'll rework the content of the commit to drop the fixture modifications introduced here (that's out of scope for this diff)

Drop unnecessary changes within the diff

Build has FAILED

Patch application report for D4310 (id=15238)

Could not rebase; Attempt merge onto 8e4026a5e8...

Updating 8e4026a..83308e9
Fast-forward
 swh/vault/api/server.py        | 18 +++++++++-------
 swh/vault/tests/conftest.py    | 49 +++++++++++++++++-------------------------
 swh/vault/tests/test_server.py | 38 +++++++++++++++++++++++++++++++-
 3 files changed, 67 insertions(+), 38 deletions(-)
Changes applied before test
commit 83308e9f7b5ca8de9a1a93d57c515898938a8434
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:54:44 2020 +0200

    vault.tests: Make postgresql fixture faster
    
    This reuse the swh.core.db.db_utils.SwhDatabasejanitor which allows to truncate
    tables in between test calls. This makes the tests faster.

commit a2ffe238d0e59def15ac5dd5a41717f6a5ff63e8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 20:06:00 2020 +0200

    api.server: Add types and tests on configuration checks

Link to build: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/28/
See console output for more information: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/28/console

This revision is now accepted and ready to land.Oct 22 2020, 10:54 AM

Build is green

Patch application report for D4310 (id=15238)

Could not rebase; Attempt merge onto 8e4026a5e8...

Updating 8e4026a..83308e9
Fast-forward
 swh/vault/api/server.py        | 18 +++++++++-------
 swh/vault/tests/conftest.py    | 49 +++++++++++++++++-------------------------
 swh/vault/tests/test_server.py | 38 +++++++++++++++++++++++++++++++-
 3 files changed, 67 insertions(+), 38 deletions(-)
Changes applied before test
commit 83308e9f7b5ca8de9a1a93d57c515898938a8434
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:54:44 2020 +0200

    vault.tests: Make postgresql fixture faster
    
    This reuse the swh.core.db.db_utils.SwhDatabasejanitor which allows to truncate
    tables in between test calls. This makes the tests faster.

commit a2ffe238d0e59def15ac5dd5a41717f6a5ff63e8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 20:06:00 2020 +0200

    api.server: Add types and tests on configuration checks

See https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/30/ for more details.

ardumont edited the summary of this revision. (Show Details)

Rebase

Build is green

Patch application report for D4310 (id=15295)

Rebasing onto 8e4026a5e8...

Current branch diff-target is up to date.
Changes applied before test
commit 97a860997b1144c6717424d11e696d50de3c4a1a
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:54:44 2020 +0200

    vault.tests: Make postgresql fixture faster
    
    This reuse the swh.core.db.db_utils.SwhDatabasejanitor which allows to truncate
    tables in between test calls. This makes the tests faster.

See https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/31/ for more details.

ardumont edited the test plan for this revision. (Show Details)
ardumont edited the summary of this revision. (Show Details)