Page MenuHomeSoftware Heritage
Differential D4381

core.db.cli: Add coverage and ensure `swh db *` works as expected
ClosedPublic
Actions

Authored by ardumont on Oct 29 2020, 3:05 PM.

Details

Reviewers
olasd
Group Reviewers
Reviewers
Maniphest Tasks
T2741: swh.core.db.cli: Add tests on the db initialization cli
Commits
rDCORE12b0e76defb1: core.db.cli: Add coverage and ensure `swh db *` works as expected
Summary

This explicitely checks the different scenario possible with the current
subcommands (create, init-admin, init).

Using a subset of sql schema representation we have in our existing datasets.

Closes T2741

Test Plan

tox

Diff Detail

Repository
rDCORE Foundations and core functionalities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Oct 29 2020, 3:05 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptOct 29 2020, 3:05 PM
swh-public-ci added a comment.Oct 29 2020, 3:06 PM

Build is green

Patch application report for D4381 (id=15493)

Could not rebase; Attempt merge onto eefe1b6336...

Updating eefe1b6..3abd57f
Fast-forward
 requirements-db.txt                             |   2 +-
 requirements-test-db.txt                        |   2 +-
 swh/core/cli/db.py                              |  54 +++++-
 swh/core/db/pytest_plugin.py                    |  10 +-
 swh/core/db/tests/data/cli/0-superuser-init.sql |   1 +
 swh/core/db/tests/data/cli/1-schema.sql         |  13 ++
 swh/core/db/tests/data/cli/3-func.sql           |   6 +
 swh/core/db/tests/data/cli/4-data.sql           |   5 +
 swh/core/db/tests/test_cli.py                   | 218 +++++++++++++++++++++++-
 9 files changed, 291 insertions(+), 20 deletions(-)
 create mode 100644 swh/core/db/tests/data/cli/0-superuser-init.sql
 create mode 100644 swh/core/db/tests/data/cli/1-schema.sql
 create mode 100644 swh/core/db/tests/data/cli/3-func.sql
 create mode 100644 swh/core/db/tests/data/cli/4-data.sql
Changes applied before test
commit 3abd57f7268c5eab1c470b6c3123f2fb1f26da24
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes T2741

commit 272f91b939af9846dbc9c26172b1ead36885e156
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 09:20:32 2020 +0100

    db.pytest_plugin: Make pytest-postgresql an optional runtime dependency
    
    Currently, this pulls the postgresql installation when installing swh.core
    which is not something wanted. That bypasses the current production pinning
    done in puppet and installs the latest postgresql-13 for example (even on node
    without postgresql).
    
    That particular dependency should be installed alongside swh modules which
    needs the core db pytest-plugin for their tests.

commit 745448399ebda14f08a9286b08fb6dd008d9504b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Oct 28 18:27:46 2020 +0100

    cli.db: Open init-admin subcmd to initialize superuser-level scripts
    
    In staging, the db is already created by puppet, the current create database
    stanza forced within the create-db cannot work.
    
    Ths subsequent steps of postgresql extensions installation is required though.
    
    This will allow it.
    
    Related to T2736

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/124/ for more details.

Harbormaster completed remote builds in B16719: Diff 15493.Oct 29 2020, 3:07 PM
ardumont mentioned this in D4374: cli.db: Open init-admin subcmd to initialize superuser-level scripts.Oct 29 2020, 3:07 PM
olasd accepted this revision.Oct 30 2020, 3:45 PM
olasd added a subscriber: olasd.

Thanks!

This revision is now accepted and ready to land.Oct 30 2020, 3:45 PM
ardumont updated this revision to Diff 15537.Oct 30 2020, 3:55 PM

Rebase

swh-public-ci added a comment.Oct 30 2020, 3:56 PM

Build is green

Patch application report for D4381 (id=15537)

Could not rebase; Attempt merge onto 12fa6f263d...

Auto-merging swh/core/db/pytest_plugin.py
Merge made by the 'recursive' strategy.
 requirements-db.txt                             |   2 +-
 requirements-test-db.txt                        |   2 +-
 swh/core/db/pytest_plugin.py                    |  10 +-
 swh/core/db/tests/data/cli/0-superuser-init.sql |   1 +
 swh/core/db/tests/data/cli/1-schema.sql         |  13 ++
 swh/core/db/tests/data/cli/3-func.sql           |   6 +
 swh/core/db/tests/data/cli/4-data.sql           |   5 +
 swh/core/db/tests/test_cli.py                   | 212 +++++++++++++++++++++++-
 8 files changed, 240 insertions(+), 11 deletions(-)
 create mode 100644 swh/core/db/tests/data/cli/0-superuser-init.sql
 create mode 100644 swh/core/db/tests/data/cli/1-schema.sql
 create mode 100644 swh/core/db/tests/data/cli/3-func.sql
 create mode 100644 swh/core/db/tests/data/cli/4-data.sql
Changes applied before test
commit d4c5044c2bef39f4826bb6fb26c78b7e8b5f46f7
Merge: 12fa6f2 8438cfa
Author: Jenkins user <jenkins@localhost>
Date:   Fri Oct 30 14:55:28 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 8438cfa7ef0967d7c427d1fcc51816c1a1b5cab6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 09:20:32 2020 +0100

    db.pytest_plugin: Make pytest-postgresql an optional runtime dependency
    
    Currently, this pulls the postgresql installation when installing swh.core
    which is not something wanted. That bypasses the current production pinning
    done in puppet and installs the latest postgresql-13 for example (even on node
    without postgresql).
    
    That particular dependency should be installed alongside swh modules which
    needs the core db pytest-plugin for their tests.

commit e58c79a75316295d089312098119774034127f85
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes T2741

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/129/ for more details.

Harbormaster completed remote builds in B16763: Diff 15537.Oct 30 2020, 3:56 PM
ardumont updated this revision to Diff 15538.Oct 30 2020, 3:56 PM

Rebase properly with the right commit

swh-public-ci added a comment.Oct 30 2020, 3:57 PM

Build is green

Patch application report for D4381 (id=15538)

Rebasing onto 12fa6f263d...

First, rewinding head to replay your work on top of it...
Applying: core.db.cli: Add coverage and ensure `swh db *` works as expected
Changes applied before test
commit adb34076e7061b167f99d59d73ae49487290e538
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes T2741

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/130/ for more details.

Harbormaster completed remote builds in B16764: Diff 15538.Oct 30 2020, 3:57 PM
ardumont updated this revision to Diff 15540.Oct 30 2020, 4:01 PM

Rebase (master got updated)

swh-public-ci added a comment.Oct 30 2020, 4:02 PM

Build is green

Patch application report for D4381 (id=15540)

Rebasing onto 12fa6f263d...

Current branch diff-target is up to date.
Changes applied before test
commit 12b0e76defb15eebcc1fb2d79200842739e81fe6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 29 14:57:24 2020 +0100

    core.db.cli: Add coverage and ensure `swh db *` works as expected
    
    This explicitely checks the different scenario possible with the current
    subcommands (create, init-admin, init).
    
    Using a subset of sql schema representation we have in our existing datasets.
    
    Closes T2741

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/131/ for more details.

Harbormaster completed remote builds in B16766: Diff 15540.Oct 30 2020, 4:02 PM
Closed by commit rDCORE12b0e76defb1: core.db.cli: Add coverage and ensure `swh db *` works as expected (authored by ardumont). · Explain WhyOct 30 2020, 4:06 PM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDCORE12b0e76defb1: core.db.cli: Add coverage and ensure `swh db *` works as expected.

Revision Contents
Changeset List

PathSize
swh/
core/
db/
tests/
data/
cli/
1 line
13 lines
6 lines
5 lines
212 lines

Diff 15541

View Options

swh/core/db/tests/data/cli/0-superuser-init.sql

Loading...
View Options

swh/core/db/tests/data/cli/1-schema.sql

Loading...
View Options

swh/core/db/tests/data/cli/3-func.sql

Loading...
View Options

swh/core/db/tests/data/cli/4-data.sql

Loading...
View Options

swh/core/db/tests/test_cli.py

Loading...
About · Developer information · Legal