Page MenuHomeSoftware Heritage

kafka: add more options to the user management script
ClosedPublic

Authored by vsellier on Jun 3 2022, 6:04 PM.

Details

Summary
  • Allow to specify the topic names
  • Allow to configure r/w and power users who can manage the topics under the prefixes

Related to T4265

Test Plan
  • getty
*** Running octocatalog-diff on host getty.internal.softwareheritage.org
I, [2022-06-03T18:02:52.796055 #90607]  INFO -- : Catalogs compiled for getty.internal.softwareheritage.org
I, [2022-06-03T18:02:53.704839 #90607]  INFO -- : Diffs computed for getty.internal.softwareheritage.org
diff origin/production/getty.internal.softwareheritage.org current/getty.internal.softwareheritage.org
*******************************************
  File[/usr/local/sbin/create_kafka_users_rocquencourt.sh] =>
   parameters =>
     content =>
      @@ -17,6 +17,10 @@
       fi
      _
      +topic_prefixes="swh.journal.objects. swh.journal.indexed."
      +privileged_topic_prefixes="swh.journal.objects_privileged."
      +
       privileged="unprivileged"
       cgrp_prefix=""
      +ops="READ DESCRIBE"
      _
       while (( $# )); do
      @@ -28,4 +32,18 @@
           shift
           shift
      +  elif [ $1 = "--topic_prefixes" ]; then
      +    topic_prefixes=$2
      +    shift
      +    shift
      +  elif [ $1 = "--privileged_topic_prefixes" ]; then
      +    privileged_topic_prefixes=$2
      +    shift
      +    shift
      +  elif [ $1 = "--rw" ]; then
      +    ops="${OPS} WRITE"
      +    shift
      +  elif [ $1 = "--admin" ]; then
      +    ops="${OPS} DELETE CREATE"
      +    shift
         else
           username=$1
      @@ -57,13 +75,11 @@
        --entity-name $username
      _
      -topic_prefixes="swh.journal.objects. swh.journal.indexed."
      -
       if [ $privileged = "privileged" ]; then
      - topic_prefixes="$topic_prefixes swh.journal.objects_privileged."
      + topic_prefixes="${topic_prefixes} ${privileged_topic_prefixes}"
       fi
      _
       for topic_prefix in $topic_prefixes; do
        echo "Granting access to topics $topic_prefix to $username"
      - for op in READ DESCRIBE; do
      + for op in ${OPS}; do
                /opt/kafka/bin/kafka-acls.sh --bootstrap-server $brokers --add --resource-pattern-type PREFIXED --topic $topic_prefix --allow-principal User:$username --operation $op
        done
*******************************************
  File[/usr/local/sbin/create_kafka_users_rocquencourt_staging.sh] =>
   parameters =>
     content =>
      @@ -17,6 +17,10 @@
       fi
      _
      +topic_prefixes="swh.journal.objects. swh.journal.indexed."
      +privileged_topic_prefixes="swh.journal.objects_privileged."
      +
       privileged="unprivileged"
       cgrp_prefix=""
      +ops="READ DESCRIBE"
      _
       while (( $# )); do
      @@ -28,4 +32,18 @@
           shift
           shift
      +  elif [ $1 = "--topic_prefixes" ]; then
      +    topic_prefixes=$2
      +    shift
      +    shift
      +  elif [ $1 = "--privileged_topic_prefixes" ]; then
      +    privileged_topic_prefixes=$2
      +    shift
      +    shift
      +  elif [ $1 = "--rw" ]; then
      +    ops="${OPS} WRITE"
      +    shift
      +  elif [ $1 = "--admin" ]; then
      +    ops="${OPS} DELETE CREATE"
      +    shift
         else
           username=$1
      @@ -57,13 +75,11 @@
        --entity-name $username
      _
      -topic_prefixes="swh.journal.objects. swh.journal.indexed."
      -
       if [ $privileged = "privileged" ]; then
      - topic_prefixes="$topic_prefixes swh.journal.objects_privileged."
      + topic_prefixes="${topic_prefixes} ${privileged_topic_prefixes}"
       fi
      _
       for topic_prefix in $topic_prefixes; do
        echo "Granting access to topics $topic_prefix to $username"
      - for op in READ DESCRIBE; do
      + for op in ${OPS}; do
                /opt/kafka/bin/kafka-acls.sh --bootstrap-server $brokers --add --resource-pattern-type PREFIXED --topic $topic_prefix --allow-principal User:$username --operation $op
        done
*******************************************
*** End octocatalog-diff on getty.internal.softwareheritage.org
`

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

update the parameter count limit according the new supported ones

ardumont added a subscriber: ardumont.

lgtm

The argument parsing is starting to get unconfortable to read.

So maybe next time, let's rewrite it to something more elegant (e.g lisp ;p, roh, ok python then ;)

This revision is now accepted and ready to land.Jun 7 2022, 2:43 PM