Page MenuHomeSoftware Heritage
Differential D4310

vault.tests: Make postgresql fixture faster
ClosedPublic
Actions

Authored by ardumont on Oct 19 2020, 7:02 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Commits
rDVAU97a860997b11: vault.tests: Make postgresql fixture faster
Summary

This reuse the swh.core.db.db_utils postgresql_fact fixture (which inits the db
then truncate tables in between test calls). This makes the tests faster.

Basic benchmark:

|---------+--------------+-----------+-----------------------|
| machine | before (min) | after (s) | Using                 |
|---------+--------------+-----------+-----------------------|
| laptop  |      1:11.02 | 20.811    | `time tox`            |
| jenkins |         1:14 | 26        | jenkins output report |
|---------+--------------+-----------+-----------------------|
Test Plan

tox

Diff Detail

Repository
rDVAU Software Heritage Vault
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Oct 19 2020, 7:02 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptOct 19 2020, 7:02 PM
swh-public-ci added a comment.Oct 19 2020, 7:02 PM

Build has FAILED

Patch application report for D4310 (id=15236)

Could not rebase; Attempt merge onto 8e4026a5e8...

Updating 8e4026a..b457d50
Fast-forward
 swh/vault/api/server.py        | 18 +++++++-----
 swh/vault/tests/conftest.py    | 67 ++++++++++++++++++++----------------------
 swh/vault/tests/test_server.py | 38 +++++++++++++++++++++++-
 3 files changed, 79 insertions(+), 44 deletions(-)
Changes applied before test
commit b457d50603989dd2203894060662f8ccd731ba32
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:54:44 2020 +0200

    vault.tests: Make postgresql fixture faster
    
    This reuse the swh.core.db.db_utils.SwhDatabasejanitor which allows to truncate
    tables in between test calls. This makes the tests faster.

commit a2ffe238d0e59def15ac5dd5a41717f6a5ff63e8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 20:06:00 2020 +0200

    api.server: Add types and tests on configuration checks

Link to build: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/27/
See console output for more information: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/27/console

Harbormaster failed remote builds in B16469: Diff 15236!Oct 19 2020, 7:02 PM
ardumont edited the summary of this revision. (Show Details)Oct 19 2020, 7:04 PM
ardumont edited the summary of this revision. (Show Details)
ardumont added a comment.Oct 19 2020, 7:07 PM

I'll rework the content of the commit to drop the fixture modifications introduced here (that's out of scope for this diff)

ardumont updated this revision to Diff 15238.Oct 19 2020, 7:16 PM

Drop unnecessary changes within the diff

swh-public-ci added a comment.Oct 19 2020, 7:16 PM

Build has FAILED

Patch application report for D4310 (id=15238)

Could not rebase; Attempt merge onto 8e4026a5e8...

Updating 8e4026a..83308e9
Fast-forward
 swh/vault/api/server.py        | 18 +++++++++-------
 swh/vault/tests/conftest.py    | 49 +++++++++++++++++-------------------------
 swh/vault/tests/test_server.py | 38 +++++++++++++++++++++++++++++++-
 3 files changed, 67 insertions(+), 38 deletions(-)
Changes applied before test
commit 83308e9f7b5ca8de9a1a93d57c515898938a8434
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:54:44 2020 +0200

    vault.tests: Make postgresql fixture faster
    
    This reuse the swh.core.db.db_utils.SwhDatabasejanitor which allows to truncate
    tables in between test calls. This makes the tests faster.

commit a2ffe238d0e59def15ac5dd5a41717f6a5ff63e8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 20:06:00 2020 +0200

    api.server: Add types and tests on configuration checks

Link to build: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/28/
See console output for more information: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/28/console

Harbormaster failed remote builds in B16471: Diff 15238!Oct 19 2020, 7:17 PM
ardumont edited the summary of this revision. (Show Details)Oct 20 2020, 11:01 AM
ardumont edited the summary of this revision. (Show Details)Oct 20 2020, 2:06 PM
vlorentz accepted this revision.Oct 22 2020, 10:54 AM
This revision is now accepted and ready to land.Oct 22 2020, 10:54 AM
swh-public-ci added a comment.Oct 22 2020, 11:32 AM

Build is green

Patch application report for D4310 (id=15238)

Could not rebase; Attempt merge onto 8e4026a5e8...

Updating 8e4026a..83308e9
Fast-forward
 swh/vault/api/server.py        | 18 +++++++++-------
 swh/vault/tests/conftest.py    | 49 +++++++++++++++++-------------------------
 swh/vault/tests/test_server.py | 38 +++++++++++++++++++++++++++++++-
 3 files changed, 67 insertions(+), 38 deletions(-)
Changes applied before test
commit 83308e9f7b5ca8de9a1a93d57c515898938a8434
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:54:44 2020 +0200

    vault.tests: Make postgresql fixture faster
    
    This reuse the swh.core.db.db_utils.SwhDatabasejanitor which allows to truncate
    tables in between test calls. This makes the tests faster.

commit a2ffe238d0e59def15ac5dd5a41717f6a5ff63e8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 20:06:00 2020 +0200

    api.server: Add types and tests on configuration checks

See https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/30/ for more details.

Harbormaster completed remote builds in B16471: Diff 15238.Oct 22 2020, 11:32 AM
ardumont updated this revision to Diff 15295.Oct 22 2020, 11:43 AM
ardumont edited the summary of this revision. (Show Details)

Rebase

swh-public-ci added a comment.Oct 22 2020, 11:44 AM

Build is green

Patch application report for D4310 (id=15295)

Rebasing onto 8e4026a5e8...

Current branch diff-target is up to date.
Changes applied before test
commit 97a860997b1144c6717424d11e696d50de3c4a1a
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Oct 18 14:54:44 2020 +0200

    vault.tests: Make postgresql fixture faster
    
    This reuse the swh.core.db.db_utils.SwhDatabasejanitor which allows to truncate
    tables in between test calls. This makes the tests faster.

See https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/31/ for more details.

Harbormaster completed remote builds in B16523: Diff 15295.Oct 22 2020, 11:44 AM
ardumont edited the summary of this revision. (Show Details)Oct 22 2020, 11:47 AM
ardumont edited the test plan for this revision. (Show Details)
ardumont edited the summary of this revision. (Show Details)
Closed by commit rDVAU97a860997b11: vault.tests: Make postgresql fixture faster (authored by ardumont). · Explain WhyOct 22 2020, 11:51 AM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDVAU97a860997b11: vault.tests: Make postgresql fixture faster.

Revision Contents
Changeset List

PathSize
2 lines
swh/
vault/
tests/
49 lines

Diff 15296

View Options

requirements-swh.txt

Loading...
View Options

swh/vault/tests/conftest.py

Loading...
About · Developer information · Legal