Page MenuHomeSoftware Heritage

Simplify the scheduler's database initialization
ClosedPublic

Authored by vsellier on Oct 29 2020, 10:00 AM.

Details

Summary

Use the new "init-admin" command to initialize the database and fallback to the standard postgresql initialization.
The database creation is delegated to the postgresql entrypoint, the extensions and the tables are configured respectively by the init-admin and init commands.

Using this, we can execute the "init-admin" command at each start which can be useful when new super-user migrations are added.

Example of a restart:

swh-scheduler_1                 | Creating extensions...
swh-scheduler_1                 | psql:/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/scheduler/sql/10-superuser-init.sql:1: NOTICE:  extension "uuid-ossp" already exists, skipping
swh-scheduler_1                 | Initializing the database...
swh-scheduler_1                 | DONE database for scheduler exists at version 17
swh-scheduler_1                 | Starting the swh-scheduler API server

Related to T2736
Depends on D4374

Test Plan

with the diff applied on core + the docker-compose.override.yml well configured to use the core module

docker-compose down -v
docker-compose up swh-scheduler-db
docker-compose down
docker-compose up swh-scheduler

Diff Detail

Repository
rDENV Development environment
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 16709
Build 25765: arc lint + arc unit

Event Timeline

This is a poc for the scheduler, all the databases initialization could be changed this way when the diff on swh-core will land.

Using this, we can execute the "init-admin" command at each start so new superuser scripts can be executed during each restart toUsing this, we can execute the "init-admin" command at each start which can be useful when new super-user migrations are added

Example of a restart:

swh-scheduler_1                 | Creating extensions...
swh-scheduler_1                 | psql:/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/scheduler/sql/10-superuser-init.sql:1: NOTICE:  extension "uuid-ossp" already exists, skipping
swh-scheduler_1                 | Initializing the database...
swh-scheduler_1                 | DONE database for scheduler exists at version 17
swh-scheduler_1                 | Starting the swh-scheduler API server
olasd added inline comments.
docker/services/swh-scheduler/entrypoint.sh
20

This needs to change back to wait_pgsql (to wait for the right database).

Check the right database availability

vsellier added inline comments.
docker/services/swh-scheduler/entrypoint.sh
20

yes right, it's fixed

ardumont added a subscriber: ardumont.

Thanks.

I really like this simplification ;)

This revision is now accepted and ready to land.Oct 29 2020, 12:46 PM
This revision was automatically updated to reflect the committed changes.
vsellier marked an inline comment as done.

I have landed this one as it's accepted. I will prepare another ones for the other databases