Page MenuHomeSoftware Heritage

client: allow to specify any cfg parameter to KafkaConsumer()
ClosedPublic

Authored by douardda on Jun 19 2019, 10:40 AM.

Details

Summary

directly from the config file.

This is needed because we want to be able to configure things like
max_poll_records or any other kafka tuning parameter.

also parallelize the copy of objects from a polled set in
content-replayer using a ThreadPoolExecutor.

Diff Detail

Repository
rDJNL Journal infrastructure
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 6311
Build 8741: tox-on-jenkinsJenkins
Build 8740: arc lint + arc unit