indexer.storage: Make server load and check explicit configuration
ClosedPublic
Actions

Authored by ardumont on Feb 21 2019, 2:19 PM.

Details

Reviewers

Group Reviewers

Commits

rDCIDX8de1d6a1b838: tests: Ignore wsgi module --doctest-modules forces module load
rDCIDXbd90485def56: swh.indexer.storage.api.wsgi: Open production wsgi entrypoint
rDCIDXd0789d98db7b: swh.indexer.cli: Move dev app entrypoint in dedicated cli
rDCIDX27f716490aad: indexer.storage: Make server load explicit configuration and check

Summary

swh.indexer.cli: Move dev app entrypoint in dedicated cli
swh.indexer.storage.api.wsgi: Open production wsgi entrypoint

Related T1533
Related D1167

Test Plan

tox

Diff Detail

Repository

rDCIDX Metadata indexer

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Feb 21 2019, 2:19 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/452/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/452/console

Harbormaster failed remote builds in B4378: Diff 3694!Feb 21 2019, 2:21 PM

Build has FAILED

In tox.ini, the --doctest-modules force the loading of all modules.
I don't want this behavior for the swh/indexer/storage/api/wsgi.py as this fails (since it's not meant to run in test context).
I need to find a way to exclude it.

tests: Ignore wsgi module --doctest-modules forces module load

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/453/ for more details.

Harbormaster completed remote builds in B4379: Diff 3695.Feb 21 2019, 2:46 PM

ardumont mentioned this in D1171: swh-idx-storage: Unify production and development start up.Feb 21 2019, 2:49 PM

ardumont added a child revision: D1171: swh-idx-storage: Unify production and development start up.Feb 21 2019, 2:52 PM

ardumont added a child revision: D1172: indexer-storage: Use new wsgi entrypoint.Feb 21 2019, 2:53 PM

vlorentz added inline comments.Feb 21 2019, 2:54 PM

swh/indexer/storage/api/server.py
44–95	That's almost the same code as in swh.storage. You should move at least the first half it to swh.core.config.

Fix exception type and rebase

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/454/ for more details.

Harbormaster completed remote builds in B4383: Diff 3698.Feb 21 2019, 3:03 PM

ardumont added a subscriber: douardda.Feb 21 2019, 3:36 PM

ardumont added inline comments.

swh/indexer/storage/api/server.py
44–95	Yes, but as discussed with @douardda, i'm focused on doing it right first and on time. Also, actually, for basic services, i agree (storage, indexer-storage, objstorage). for other services, that's no longer true (webapp, deposit), there are way much to check. So this refactoring will need more thinking first.

ardumont retitled this revision from indexer.storage: Make server load explicit configuration and check to indexer.storage: Make server load and check explicit configuration.Feb 21 2019, 3:51 PM

vlorentz accepted this revision.Feb 21 2019, 4:07 PM

This revision is now accepted and ready to land.Feb 21 2019, 4:07 PM

ardumont mentioned this in T1533: Make sure api server uses explicit configurations.Feb 21 2019, 8:34 PM

Closed by commit rDCIDX27f716490aad: indexer.storage: Make server load explicit configuration and check (authored by ardumont). · Explain WhyFeb 22 2019, 10:56 AM

This revision was automatically updated to reflect the committed changes.

ardumont mentioned this in rCDFD6d9720b41fcb: swh-idx-storage: Unify production and development start up.

ardumont mentioned this in D1182: swh.web.config: Allow configuration through environment variable.Feb 23 2019, 1:53 AM

ardumont mentioned this in rDENV6d9720b41fcb: swh-idx-storage: Unify production and development start up.Nov 26 2019, 1:35 PM