Page MenuHomeSoftware Heritage

settings.prod: Use SWH_CONFIG_FILENAME to load & check swh config
ClosedPublic

Authored by ardumont on Feb 20 2019, 2:03 PM.

Diff Detail

Repository
rDDEP Push deposit
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 4332
Build 5723: tox-on-jenkinsJenkins
Build 5722: arc lint + arc unit

Event Timeline

vlorentz added inline comments.
swh/deposit/wsgi.py
22

Shouldn't this be dropped?

ardumont added inline comments.
swh/deposit/wsgi.py
22

That's a good question. And i think you are right.

I'm unsure then about the config_file parameter of the function below though.

ardumont added inline comments.
swh/deposit/wsgi.py
22

I'm unsure then about the config_file parameter of the function below though.

I mean does that even make sense to open it up at all?
using an environment variable for the setup is still kinda implicit...


context: production setup is to use that module as entry point [1]

[1] https://forge.softwareheritage.org/source/puppet-swh-site/browse/production/site-modules/profile/manifests/swh/deploy/deposit.pp$89

From what I can tell, in your make_app_from_configfile function, the conf object is never used, only checked. That feels very brittle to me and I think the checks should happen in the swh.deposit.settings.production module directly.

It's standard practice in Django deployments to rely on environment variables set outside the WSGI process manager to load configuration; that's what the DJANGO_SETTINGS_MODULE environment variable is there for. I think it makes sense for our production settings module to also look at the SWH_CONFIG_FILENAME envvar to pull the settings that we don't want to publish in the main DJANGO_SETTINGS_MODULE (which is itself published in the swh.deposit module on PyPI).

I'm not too fussed about the wsgi module shipping a default value for DJANGO_SETTINGS_MODULE (as it's not supposed to be changed anyway), and frankly for SWH_CONFIG_FILENAME as well as it makes for easier deployment; but it's not too hard to set SWH_CONFIG_FILENAME explicitly on the gunicorn service declaration instead (via an Environment directive in the systemd unit).

swh/deposit/wsgi.py
22

When the gunicorn executable parameter is set to a plain module, it's going to look for an attribute called application.

It's also possible to explicitly set an explicit object to look for
(https://forge.softwareheritage.org/source/puppet-swh-site/browse/production/site-modules/profile/manifests/swh/deploy/storage.pp$9), and even to have a callable and pass it arguments.

olasd requested changes to this revision.Feb 20 2019, 3:52 PM
This revision now requires changes to proceed.Feb 20 2019, 3:52 PM

From what I can tell, in your make_app_from_configfile function, the conf object is never used, only checked.

True.

That feels very brittle to me and I think the checks should happen in the swh.deposit.settings.production module directly.

ah, yeah, makes sense, that's where it's used anyway!

It's standard practice in Django deployments to rely on environment variables set outside the WSGI process manager to load configuration; that's what the DJANGO_SETTINGS_MODULE environment variable is there for. I think it makes sense for our production settings module to also look at the SWH_CONFIG_FILENAME envvar to pull the settings that we don't want to publish in the main DJANGO_SETTINGS_MODULE (which is itself published in the swh.deposit module on PyPI).

Yes, right.
I think that's what i aimed at in D1161.

I'm not too fussed about the wsgi module shipping a default value for DJANGO_SETTINGS_MODULE (as it's not supposed to be changed anyway), and frankly for SWH_CONFIG_FILENAME as well as it makes for easier deployment; but it's not too hard to set SWH_CONFIG_FILENAME explicitly on the gunicorn service declaration instead (via an Environment directive in the systemd unit).

Right, well, i already did the change in the manifest so i'll keep that ;)

Thanks, that helps a lot.

ardumont retitled this revision from deposit.wsgi: Allow using SWH_CONFIG_FILENAME environment variable to settings.prod: Use SWH_CONFIG_FILENAME to load & check swh config.Feb 20 2019, 4:26 PM
This revision is now accepted and ready to land.Feb 20 2019, 4:33 PM
This revision was automatically updated to reflect the committed changes.