Page MenuHomeSoftware Heritage

docker-compose: Add journal-backfiller
ClosedPublic

Authored by ardumont on Apr 4 2019, 4:18 PM.

Diff Detail

Repository
rCDFD Dockerfiles for developers
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert added a subscriber: anlambert.

Looks good to me!

This revision is now accepted and ready to land.Apr 4 2019, 8:05 PM
douardda requested changes to this revision.Apr 5 2019, 4:51 PM

I'm not very fond adding this kind of service in the main docker-compose file.
I would prefer a dedicated docker-compose file to be used as an override, eg:

docker-compose -f docker-compose.yml -f docker-compose.backfiller.yml up

(with proper doc in the README).

This revision now requires changes to proceed.Apr 5 2019, 4:51 PM

I'm not very fond adding this kind of service in the main docker-compose file.
I would prefer a dedicated docker-compose file to be used as an override, eg:

docker-compose -f docker-compose.yml -f docker-compose.backfiller.yml up

(with proper doc in the README).

Sure!
It was to give access on how to test the D1345 at the time (not necessarily to merge it as is).

Rebase and plug diff to a branch

  • Ignore override docker-compose file
  • docker-compose: Add journal-backfiller
  • README: Update to demonstrate how to run the backfiller

few minor fixes to be make, then should be ok.

README.md
669 ↗(On Diff #4567)

please use the plain docker-compose command here

conf/journal_backfiller.yml
9 ↗(On Diff #4567)

Not to be fixed now, but this should be handled the generic way:
have a storage section to configure the storage to use (which should then be instantiated by the standard storage factory function swh.storage.get_storage).

services/swh-journal-backfiller/entrypoint.sh
7 ↗(On Diff #4567)

unless I'm mistaken, this is already done in the setup_pip command.

This revision now requires changes to proceed.Apr 15 2019, 12:44 PM
ardumont added inline comments.
services/swh-journal-backfiller/entrypoint.sh
7 ↗(On Diff #4567)

Indeed

ardumont marked an inline comment as done.

Adapt according to review

This revision is now accepted and ready to land.Apr 15 2019, 4:42 PM
This revision was automatically updated to reflect the committed changes.