diff --git a/sysadm/mirror-operations/onboard.rst b/sysadm/mirror-operations/onboard.rst --- a/sysadm/mirror-operations/onboard.rst +++ b/sysadm/mirror-operations/onboard.rst @@ -3,6 +3,58 @@ How to onboard a mirror ======================= +.. admonition:: Intended audience + :class: important + + sysadm staff members + +A mirror needs credentials to access our journal. + +They are manually created by a software Heritage System Administrator and +different credentials must be provided for staging and production. + +The URLs to communicate to the mirror operator are defined in :ref:`service-url` +in the 'Public URLs' sections, 'Journal TLS' entries. + +How to create the credentials +----------------------------- + +Connect on one of the Kafka servers of the environment. + +Execute the following commands: + +.. code-block:: bash + + export ZOOKEEPER_SERVER=$(hostname -f):2181/kafka/softwareheritage + export BOOTSTRAP_SERVERS=$(hostname -f):9092 + export USERNAME=-[stg|prod] + export PASSWORD=XXXXX + + /opt/kafka/bin/kafka-configs.sh \ + --zookeeper ${ZOOKEEPER_SERVER}/kafka/softwareheritage \ + --alter \ + --add-config "SCRAM-SHA-256=[iterations=8192,password=${PASSWORD}],SCRAM-SHA-512=[password=${PASSWORD}]" \ + --entity-type users \ + --entity-name ${USERNAME} + + /opt/kafka/bin/kafka-acls.sh --bootstrap-server ${BOOTSTRAP_SERVERS} --add --resource-pattern-type PREFIXED --topic swh.journal.objects. --allow-principal User:${USERNAME} --operation READ + + /opt/kafka/bin/kafka-acls.sh --bootstrap-server ${BOOTSTRAP_SERVERS} --add --resource-pattern-type PREFIXED --topic swh.journal.objects. --allow-principal User:${USERNAME} --operation DESCRIBE + + /opt/kafka/bin/kafka-acls.sh --bootstrap-server ${BOOTSTRAP_SERVERS} --list --resource-pattern-type PREFIXED --topic swh.journal.objects. --principal User:${USERNAME} + + # Allow READ on consumer groups prefixed with `${USERNAME}-`. The '-' at the end of the group is important to disambiguate the prefix. + /opt/kafka/bin/kafka-acls.sh --bootstrap-server ${BOOTSTRAP_SERVERS} --add --resource-pattern-type PREFIXED --group ${USERNAME}- --allow-principal User:${USERNAME} --operation READ + + +The above commands give an access to the anonymized topics (names and emails replaced by their hashes). For an access to the non anonymized, execute this additional command: + +.. code-block:: bash + + /opt/kafka/bin/kafka-acls.sh --bootstrap-server ${BOOTSTRAP_SERVERS} --add --resource-pattern-type PREFIXED --topic swh.journal.objects_privileged. --allow-principal User:${USERNAME} --operation READ + + /opt/kafka/bin/kafka-acls.sh --bootstrap-server ${BOOTSTRAP_SERVERS} --add --resource-pattern-type PREFIXED --topic swh.journal.objects_privileged. --allow-principal User:${USERNAME} --operation DESCRIBE + + /opt/kafka/bin/kafka-acls.sh --bootstrap-server ${BOOTSTRAP_SERVERS} --list --resource-pattern-type PREFIXED --topic swh.journal.objects_privileged. --principal User:${USERNAME} + -.. todo:: - This page is a work in progress.