Details

Reviewers

douardda
olasd

Group Reviewers

Reviewers

Commits

rCDFJb297622e655d: kafka/dockerfile: Name the building stage 0 kafka_fetcher
rCDFJb9128277908b: kafka/dockerfile: Fix permission in the runtime image
rCDFJa991bc6d18d7: kafka/Dockerfile: Use multi-stage build
rCDFJc80eae7798d9: kafka/Dockerfile: Use arguments to avoid repetition
rCDFJcc483aa7eb7e: Add a new unit test image based on swh-tox enriched with kafka

Summary

Use case is the journal tests needs a kafka that can be
started/stopped during the tests.

Related D1239

Remains:

D1239 needs adaptation to actually lift that, based on environment variable(s)

the build job needs to be parametrized to change the jenkins node for running tests. Defaulting to swh-tox and be able to run with swh-kafka (for swh-journal here)

Test Plan

# run a local registry
docker run -d -p 5000:5000 --name registry registry:2
# build the node using local registry
REGISTRY=localhost:5000 ; make kafka
# connect to the node and check everything is as expected
REGISTRY=localhost:5000; make kafka exec /bin/bash
$ ls -lh /opt
lrwxrwxrwx 1 jenkins jenkins   21 Mar 20 10:04 kafka -> /opt/kafka_2.12-2.1.1
drwxr-xr-x 1 jenkins jenkins 4.0K Feb  8 18:33 kafka_2.12-2.1.1

Diff Detail

Repository

rCDFJ Dockerfiles for Jenkins

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Mar 19 2019, 6:23 PM

Harbormaster completed remote builds in B4732: Diff 4016.Mar 19 2019, 6:23 PM

ardumont marked an inline comment as done.Mar 19 2019, 6:25 PM

ardumont added inline comments.

kafka/retrieve-kafka.sh
32 ↗	(On Diff #4016)	That should go away. Just to demonstrate i tried to have something that was less naive that what ends up in the Dockerfile. To run this script, i'd need more adaptation to do in the Makefile (mounting volume). I'm not sure that is worth the trouble though.

ardumont added a child revision: D1239: swh-journal: Add tests to the journal publisher using kafka instance.Mar 20 2019, 8:56 AM

olasd requested changes to this revision.Mar 20 2019, 9:32 AM

olasd added a subscriber: olasd.

olasd added inline comments.

kafka/Dockerfile
10–14	please collapse these in a single command, to avoid adding four useless layers in the docker cache

This revision now requires changes to proceed.Mar 20 2019, 9:32 AM

olasd added inline comments.Mar 20 2019, 9:34 AM

kafka/Dockerfile
10–14	you should probably also add the shasum check so we're sure what we've downloaded

vlorentz added a subscriber: vlorentz.Mar 20 2019, 9:34 AM

vlorentz added inline comments.

kafka/Dockerfile
10–14	and rm the `.tgz` in the same command.

Thanks people!
will do that.

Adapt according to reviews

Harbormaster completed remote builds in B4741: Diff 4021.Mar 20 2019, 9:53 AM

Remove the signature file as well

Harbormaster completed remote builds in B4742: Diff 4022.Mar 20 2019, 9:57 AM

olasd accepted this revision.Mar 20 2019, 9:59 AM

This revision is now accepted and ready to land.Mar 20 2019, 9:59 AM

kafka/Dockerfile: Use argument to avoid repetition
kafka/Dockerfile: Use multi-stage build

Harbormaster completed remote builds in B4744: Diff 4024.Mar 20 2019, 10:29 AM

kafka/Dockerfile: Use arguments to avoid repetition
kafka/Dockerfile: Use multi-stage build
kafka/dockerfile: Fix permission in the runtime image

Harbormaster completed remote builds in B4745: Diff 4025.Mar 20 2019, 11:07 AM

ardumont edited the test plan for this revision. (Show Details)Mar 20 2019, 11:08 AM

kafka/dockerfile: Name the building stage 0 kafka_fetcher

Harbormaster completed remote builds in B4746: Diff 4026.Mar 20 2019, 11:46 AM

ardumont added a child revision: D1282: Make swh-journal node run on swh-kafka node.Mar 20 2019, 12:39 PM

ardumont mentioned this in D1239: swh-journal: Add tests to the journal publisher using kafka instance.Mar 20 2019, 12:53 PM

Closed by commit rCDFJcc483aa7eb7e: Add a new unit test image based on swh-tox enriched with kafka (authored by ardumont). · Explain WhyMar 20 2019, 5:44 PM

This revision was automatically updated to reflect the committed changes.

Add a new unit test image based on swh-tox enriched with kafka
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 4039

kafka/Dockerfile

Add a new unit test image based on swh-tox enriched with kafkaClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 4039

kafka/Dockerfile

Add a new unit test image based on swh-tox enriched with kafka
ClosedPublic
Actions

Revision Contents
Changeset List