Page MenuHomeSoftware Heritage

core.db.cli: Add coverage and ensure `swh db *` works as expected
ClosedPublic

Authored by ardumont on Oct 29 2020, 3:05 PM.

Details

Summary

This explicitely checks the different scenario possible with the current
subcommands (create, init-admin, init).

Using a subset of sql schema representation we have in our existing datasets.

Closes T2741

Test Plan

tox

Diff Detail

Event Timeline

Build is green

Patch application report for D4381 (id=15493)

Could not rebase; Attempt merge onto eefe1b6336...

Updating eefe1b6..3abd57f
Fast-forward
 requirements-db.txt                             |   2 +-
 requirements-test-db.txt                        |   2 +-
 swh/core/cli/db.py                              |  54 +++++-
 swh/core/db/pytest_plugin.py                    |  10 +-
 swh/core/db/tests/data/cli/0-superuser-init.sql |   1 +
 swh/core/db/tests/data/cli/1-schema.sql         |  13 ++
 swh/core/db/tests/data/cli/3-func.sql           |   6 +
 swh/core/db/tests/data/cli/4-data.sql           |   5 +
 swh/core/db/tests/test_cli.py                   | 218 +++++++++++++++++++++++-
 9 files changed, 291 insertions(+), 20 deletions(-)
 create mode 100644 swh/core/db/tests/data/cli/0-superuser-init.sql
 create mode 100644 swh/core/db/tests/data/cli/1-schema.sql
 create mode 100644 swh/core/db/tests/data/cli/3-func.sql
 create mode 100644 swh/core/db/tests/data/cli/4-data.sql
Changes applied before test
commit 3abd57f7268c5eab1c470b6c3123f2fb1f26da24
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes T2741

commit 272f91b939af9846dbc9c26172b1ead36885e156
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 09:20:32 2020 +0100

    db.pytest_plugin: Make pytest-postgresql an optional runtime dependency
    
    Currently, this pulls the postgresql installation when installing swh.core
    which is not something wanted. That bypasses the current production pinning
    done in puppet and installs the latest postgresql-13 for example (even on node
    without postgresql).
    
    That particular dependency should be installed alongside swh modules which
    needs the core db pytest-plugin for their tests.

commit 745448399ebda14f08a9286b08fb6dd008d9504b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 28 18:27:46 2020 +0100

    cli.db: Open init-admin subcmd to initialize superuser-level scripts
    
    In staging, the db is already created by puppet, the current create database
    stanza forced within the create-db cannot work.
    
    Ths subsequent steps of postgresql extensions installation is required though.
    
    This will allow it.
    
    Related to T2736

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/124/ for more details.

olasd added a subscriber: olasd.

Thanks!

This revision is now accepted and ready to land.Oct 30 2020, 3:45 PM

Build is green

Patch application report for D4381 (id=15537)

Could not rebase; Attempt merge onto 12fa6f263d...

Auto-merging swh/core/db/pytest_plugin.py
Merge made by the 'recursive' strategy.
 requirements-db.txt                             |   2 +-
 requirements-test-db.txt                        |   2 +-
 swh/core/db/pytest_plugin.py                    |  10 +-
 swh/core/db/tests/data/cli/0-superuser-init.sql |   1 +
 swh/core/db/tests/data/cli/1-schema.sql         |  13 ++
 swh/core/db/tests/data/cli/3-func.sql           |   6 +
 swh/core/db/tests/data/cli/4-data.sql           |   5 +
 swh/core/db/tests/test_cli.py                   | 212 +++++++++++++++++++++++-
 8 files changed, 240 insertions(+), 11 deletions(-)
 create mode 100644 swh/core/db/tests/data/cli/0-superuser-init.sql
 create mode 100644 swh/core/db/tests/data/cli/1-schema.sql
 create mode 100644 swh/core/db/tests/data/cli/3-func.sql
 create mode 100644 swh/core/db/tests/data/cli/4-data.sql
Changes applied before test
commit d4c5044c2bef39f4826bb6fb26c78b7e8b5f46f7
Merge: 12fa6f2 8438cfa
Author: Jenkins user <jenkins@localhost>
Date:   Fri Oct 30 14:55:28 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 8438cfa7ef0967d7c427d1fcc51816c1a1b5cab6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 09:20:32 2020 +0100

    db.pytest_plugin: Make pytest-postgresql an optional runtime dependency
    
    Currently, this pulls the postgresql installation when installing swh.core
    which is not something wanted. That bypasses the current production pinning
    done in puppet and installs the latest postgresql-13 for example (even on node
    without postgresql).
    
    That particular dependency should be installed alongside swh modules which
    needs the core db pytest-plugin for their tests.

commit e58c79a75316295d089312098119774034127f85
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes T2741

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/129/ for more details.

Rebase properly with the right commit

Build is green

Patch application report for D4381 (id=15538)

Rebasing onto 12fa6f263d...

First, rewinding head to replay your work on top of it...
Applying: core.db.cli: Add coverage and ensure `swh db *` works as expected
Changes applied before test
commit adb34076e7061b167f99d59d73ae49487290e538
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes T2741

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/130/ for more details.

Rebase (master got updated)

Build is green

Patch application report for D4381 (id=15540)

Rebasing onto 12fa6f263d...

Current branch diff-target is up to date.
Changes applied before test
commit 12b0e76defb15eebcc1fb2d79200842739e81fe6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes T2741

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/131/ for more details.