indexer.storage: Make server load and check explicit configuration
ClosedPublic
Actions

Authored by ardumont on Feb 21 2019, 2:19 PM.

Details

Reviewers

Group Reviewers

Commits

rDCIDX8de1d6a1b838: tests: Ignore wsgi module --doctest-modules forces module load
rDCIDXbd90485def56: swh.indexer.storage.api.wsgi: Open production wsgi entrypoint
rDCIDXd0789d98db7b: swh.indexer.cli: Move dev app entrypoint in dedicated cli
rDCIDX27f716490aad: indexer.storage: Make server load explicit configuration and check

Summary

swh.indexer.cli: Move dev app entrypoint in dedicated cli
swh.indexer.storage.api.wsgi: Open production wsgi entrypoint

Related T1533
Related D1167

Test Plan

tox

Diff Detail

Repository

rDCIDX Metadata indexer

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 4378
Build 5788: tox-on-jenkins	Jenkins
Build 5787: arc lint + arc unit

Event Timeline

ardumont created this revision.Feb 21 2019, 2:19 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/452/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/452/console

Harbormaster failed remote builds in B4378: Diff 3694!Feb 21 2019, 2:21 PM

Build has FAILED

In tox.ini, the --doctest-modules force the loading of all modules.
I don't want this behavior for the swh/indexer/storage/api/wsgi.py as this fails (since it's not meant to run in test context).
I need to find a way to exclude it.

tests: Ignore wsgi module --doctest-modules forces module load

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/453/ for more details.

Harbormaster completed remote builds in B4379: Diff 3695.Feb 21 2019, 2:46 PM

ardumont mentioned this in D1171: swh-idx-storage: Unify production and development start up.Feb 21 2019, 2:49 PM

ardumont added a child revision: D1171: swh-idx-storage: Unify production and development start up.Feb 21 2019, 2:52 PM

ardumont added a child revision: D1172: indexer-storage: Use new wsgi entrypoint.Feb 21 2019, 2:53 PM

vlorentz added inline comments.Feb 21 2019, 2:54 PM

swh/indexer/storage/api/server.py
44–95	That's almost the same code as in swh.storage. You should move at least the first half it to swh.core.config.

Fix exception type and rebase

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/454/ for more details.

Harbormaster completed remote builds in B4383: Diff 3698.Feb 21 2019, 3:03 PM

ardumont added a subscriber: douardda.Feb 21 2019, 3:36 PM

ardumont added inline comments.

swh/indexer/storage/api/server.py
44–95	Yes, but as discussed with @douardda, i'm focused on doing it right first and on time. Also, actually, for basic services, i agree (storage, indexer-storage, objstorage). for other services, that's no longer true (webapp, deposit), there are way much to check. So this refactoring will need more thinking first.

ardumont retitled this revision from indexer.storage: Make server load explicit configuration and check to indexer.storage: Make server load and check explicit configuration.Feb 21 2019, 3:51 PM

vlorentz accepted this revision.Feb 21 2019, 4:07 PM

This revision is now accepted and ready to land.Feb 21 2019, 4:07 PM

ardumont mentioned this in T1533: Make sure api server uses explicit configurations.Feb 21 2019, 8:34 PM

Closed by commit rDCIDX27f716490aad: indexer.storage: Make server load explicit configuration and check (authored by ardumont). · Explain WhyFeb 22 2019, 10:56 AM

This revision was automatically updated to reflect the committed changes.

ardumont mentioned this in rCDFD6d9720b41fcb: swh-idx-storage: Unify production and development start up.

ardumont mentioned this in D1182: swh.web.config: Allow configuration through environment variable.Feb 23 2019, 1:53 AM

ardumont mentioned this in rDENV6d9720b41fcb: swh-idx-storage: Unify production and development start up.Nov 26 2019, 1:35 PM