Page MenuHomeSoftware Heritage
Differential D902

Add loader workers services
ClosedPublic
Actions

Authored by douardda on Jan 9 2019, 5:29 PM.

Details

Reviewers
vlorentz
ardumont
Group Reviewers
Reviewers
Commits
rDENV06f417d3eb88: task_queues configuration is not needed any more
rDENV91288bbf2de3: Fix loader config file name
rDENVaf2d06948b61: Add loader workers services
rCDFD91288bbf2de3: Fix loader config file name
rCDFD06f417d3eb88: task_queues configuration is not needed any more
rCDFDaf2d06948b61: Add loader workers services
Summary

Also

  • task_queues configuration is not needed any more
  • Fix loader config file name

We should really consider being more consistant here:

  • loaders: ~/.config/swh/loader/<name>.yml
  • listers: ~/.config/swh/lister-<name>.yml

Require D888, D900 and D901 to work properly.

Diff Detail

Repository
rCDFD Dockerfiles for developers
Branch
loaders
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 3320
Build 4283: arc lint + arc unit

Event Timeline

douardda created this revision.Jan 9 2019, 5:29 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptJan 9 2019, 5:29 PM
Harbormaster completed remote builds in B3320: Diff 2860.Jan 9 2019, 5:29 PM
vlorentz accepted this revision.Jan 9 2019, 5:52 PM
vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
dockerfiles/swh-loaders-worker/Dockerfile
9–10

indent

dockerfiles/swh-loaders-worker/entrypoint.sh
8

will

scheduler.env
6

Did you mean to commit this?

This revision is now accepted and ready to land.Jan 9 2019, 5:52 PM
ardumont accepted this revision.Jan 10 2019, 8:52 AM
ardumont added a subscriber: ardumont.

\m/

We should really consider being more consistant here...

Quite!

ardumont added a comment.Jan 10 2019, 8:54 AM

consistant

also, probably a typo, still, it's consist`e`nt ;)

douardda marked an inline comment as done.Jan 10 2019, 9:21 AM
douardda added inline comments.
scheduler.env
6

Defining the env var is needed here (according the entrypoint.sh file) and I don't mind running these dockerized env stuff in debug mode, but I'm fine with upgrading the log level to INFO here.

douardda updated this revision to Diff 2869.Jan 10 2019, 9:27 AM

Fix typos and raise log level to INFO

Harbormaster completed remote builds in B3344: Diff 2869.Jan 10 2019, 9:27 AM
douardda updated this revision to Diff 2872.Jan 10 2019, 9:34 AM

rebased

Harbormaster completed remote builds in B3347: Diff 2872.Jan 10 2019, 9:35 AM
douardda marked 3 inline comments as done.Jan 10 2019, 9:36 AM
douardda added a child revision: D909: Add missing swh-lister-bitbucket service.Jan 10 2019, 1:52 PM
Closed by commit rCDFDaf2d06948b61: Add loader workers services (authored by douardda). · Explain WhyJan 10 2019, 4:02 PM
This revision was automatically updated to reflect the committed changes.
douardda added a commit: rDENVaf2d06948b61: Add loader workers services.Nov 26 2019, 1:34 PM
douardda added a commit: rDENV06f417d3eb88: task_queues configuration is not needed any more.
douardda added a commit: rDENV91288bbf2de3: Fix loader config file name.

Revision Contents
Changeset List

PathSize
73 lines
dockerfiles/
swh-listers-worker/
1 line
2 lines
swh-loaders-worker/
25 lines
46 lines
swh-loaders-worker/
swh-listers-worker/
loader.yml
lister.yml
4 lines
swh-scheduler-worker/
16 lines
1 line

Diff 2860

View Options

docker-compose.yml

Loading...
View Options

dockerfiles/swh-listers-worker/entrypoint.sh

Loading...
View Options

dockerfiles/swh-listers-worker/lister.yml

Loading...
View Options

dockerfiles/swh-loaders-worker/Dockerfile

Loading...
View Options

dockerfiles/swh-loaders-worker/entrypoint.sh

Loading...
View Options

dockerfiles/swh-loaders-worker/loader.yml

Loading...
View Options

dockerfiles/swh-scheduler-worker/entrypoint.sh

Loading...
View Options

scheduler.env

Loading...
About · Developer information · Legal