Page MenuHomeSoftware Heritage

cli: fix brokers argument handling (+ a pile of small refactorings)
ClosedPublic

Authored by douardda on Jun 12 2019, 10:56 AM.

Details

Summary
  • JournalClient: change the prefix init argument dafault value to None and give this None value the special meaning of 'use the default value', i.e. 'swh.journal.objects' (specifically, the value stored as swh.journal.DEFAULT_PREFIX).

    This is required to make it easier to handle cli arguments and config values read from the config file. Let the None value be the default (from the cli config/argument handling) as long as possible and decide only in the JournalClient what "default" really means.
  • replay: improve logging in process_replay_objects_content() catch exceptions and log human readable ids of failed objects.
  • cli: do only listen to required kafka topic in content-replay command

    Also make the JournalClient's constructor argument object_types default value resolved at execution time (to ease a bit default value handling from cli).
  • cli: fix handling of get_journal_client's 'brokers' argument

    it can be an empty tuple (when built from a click option with multiple=1).
  • Bump dependency on swh.storage to v0.0.141 It is required to make tests pass.

Diff Detail

Repository
rDJNL Journal infrastructure
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 6152
Build 8482: tox-on-jenkinsJenkins
Build 8481: arc lint + arc unit

Event Timeline

vlorentz added inline comments.
swh/journal/replay.py
61–62

logger.exception('Failed to copy %s', hash_to_hex(obj_id))

This revision now requires changes to proceed.Jun 12 2019, 11:00 AM

log an exception if process_replay_objects_content() fails

as asked by vlorentz.

Also reword the commit message of the impacted revision.

This revision is now accepted and ready to land.Jun 12 2019, 1:16 PM
This revision was automatically updated to reflect the committed changes.