HomeSoftware Heritage

Add support for a priviledged "channel" of topics for non-anonymized objects

Description

Add support for a priviledged "channel" of topics for non-anonymized objects

The idea is to publish on a "public" prefix anonymized objects and have a
priviledged prefix for non-anonymized ones. Every model object which
".anonymize()" method returns a(n anonymized) model object will be sent
to both channels (anonymized version on the regular channel, and
original version on the privileged channel).

Currently only Release and Revision objects are anonymizable.

For anonymizable objects:

  • regular (non-anonymized) objects are sent to the topic

    "{prefix}_privileged.{object_type}"
  • anonymized version of these objects are sent to

    "{prefix}.{object_type}"

On the client side, a new boolean "privileged" config parameter is used to
select privileged topics if vivible on the Kafka broker at subscrition time.

Details

Provenance
douarddaAuthored on May 18 2020, 11:35 AM
douarddaPushed on May 25 2020, 10:34 AM
Differential Revision
D3172: Add support for a priviledged "channel" of topics for non-anonymized objects
Parents
rDJNL89cd8f7bea87: Fix test data: only create aware datetime objects
Branches
Unknown
Tags
Unknown
Build Status
Buildable 12473
Build 18932: test-and-buildJenkins console · Jenkins