Details

Reviewers

Group Reviewers

Commits

rDSTOCd19392c0fadb: Notify Kafka when an origin visit is update.
R65:d19392c0fadb: Notify Kafka when an origin visit is update.
rDSTOd19392c0fadb: Notify Kafka when an origin visit is update.

Summary

First step for T1465.

Note that it changes the value of topic_prefix.

Test Plan

There is a new test

Diff Detail

Repository

rDSTO Storage manager

Branch

notify-origin-visit-update

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 3394
Build 4377: tox-on-jenkins	Jenkins
Build 4376: arc lint + arc unit

Event Timeline

vlorentz created this revision.Jan 10 2019, 4:26 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJan 10 2019, 4:26 PM

vlorentz edited the summary of this revision. (Show Details)Jan 10 2019, 4:26 PM

vlorentz edited the test plan for this revision. (Show Details)

Build is green
See https://jenkins.softwareheritage.org/job/DSTO/job/tox/149/ for more details.

Harbormaster completed remote builds in B3388: Diff 2908.Jan 10 2019, 4:29 PM

vlorentz mentioned this in D916: Add support for object updates to the journal publisher..Jan 10 2019, 4:34 PM

I don't think there's any need to create the changed topics, or to add the mutable objects config : while its name doesn't reflect that, the new topic has always been designed as a way to notify the producer component of objects that need to be updated in the topic the actual clients subscribe to.

That's also apparent in D916 where the changed topic just ends up being used for feeding the same "final" topic.

To drive the point further : the journal hasn't been designed as a feed of all changes to the archive; it's currently designed as a way to get *the latest version* of all the objects that have been added to the archive, in a somewhat streaming fashion.

In the topics that the consumers are listening to, Kafka compaction rules will make messages about "old" versions of objects disappear when a new version (with an identical key) appears.