Make swh db init and swh db-init behavior much closer to one another
ClosedPublic
Actions

Authored by olasd on Sep 16 2020, 7:21 PM.

Details

Reviewers

douardda
ardumont

Commits

rDCORE6b983c28a3be: Make swh db init and swh db-init behavior much closer to one another
rDCORE327f98d18602: Ignore stderr in db_testing.swh_db_version instead of printing it out

Summary

While named similarly, these two CLI utilities used to take quite a different
approach to doing the same thing.

They now both call out the same initialization function for a given module. The
main difference is now the source of information for what databases to
initialize: swh db init uses the SWH_CONFIG_FILENAME configuration file, while
swh db-init only uses its own command line arguments.

(also sneak in a cosmetic change in a separate commit to db_testing.swh_db_version)

Test Plan

reviewed the use of swh db init and swh db-init in all modules and their

documentation; All documented usage is still valid.

ran the docker initialization with the new swh-core which worked fine

Diff Detail

Repository

rDCORE Foundations and core functionalities

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 15205
Build 23443: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 23442: arc lint + arc unit

Event Timeline

olasd created this revision.Sep 16 2020, 7:21 PM

Build has FAILED

Patch application report for D3970 (id=13992)

Rebasing onto 22e77212f1...

Current branch diff-target is up to date.

Changes applied before test

commit 2c086ddee9f249a0ba2ba8d47c7eff62faf24b98
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Wed Sep 16 19:13:51 2020 +0200

    Make swh db init and swh db-init behavior much closer to one another
    
    While named similarly, these two CLI utilities used to take quite a different
    approach to doing the same thing.
    
    They now both call out the same initialization function for a given module. The
    main difference is now the source of information for what databases to
    initialize: `swh db init` uses the SWH_CONFIG_FILENAME configuration file, while
    `swh db-init` only uses its own command line arguments.

commit 327f98d1860280c9cbde86af14addf183eff112a
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Wed Sep 16 18:57:11 2020 +0200

    Ignore stderr in db_testing.swh_db_version instead of printing it out

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/56/
See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/56/console

Harbormaster failed remote builds in B15204: Diff 13992!Sep 16 2020, 7:22 PM

Fix flake8 issue that snuck past pre-commit thanks to rebase shenanigans

Build is green

Patch application report for D3970 (id=13993)

Rebasing onto 22e77212f1...

Current branch diff-target is up to date.

Changes applied before test

commit 6b983c28a3bea0987281cbeb48004c164f55d07c
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Wed Sep 16 19:13:51 2020 +0200

    Make swh db init and swh db-init behavior much closer to one another
    
    While named similarly, these two CLI utilities used to take quite a different
    approach to doing the same thing.
    
    They now both call out the same initialization function for a given module. The
    main difference is now the source of information for what databases to
    initialize: `swh db init` uses the SWH_CONFIG_FILENAME configuration file, while
    `swh db-init` only uses its own command line arguments.

commit 327f98d1860280c9cbde86af14addf183eff112a
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Wed Sep 16 18:57:11 2020 +0200

    Ignore stderr in db_testing.swh_db_version instead of printing it out

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/57/ for more details.

Harbormaster completed remote builds in B15205: Diff 13993.Sep 16 2020, 7:25 PM

lgtm

one remark/question above.

Thanks.

swh/core/cli/db.py
78	i think we could also have the case of cfg.get("db") return something directly. Configuration can be either: storage: cls: local args: db: ... covered here. or with the "new" abilities from get_storage (iirc): storage: cls: local db: ...
88	for those wondering ;) https://click.palletsprojects.com/en/7.x/api/#click.secho