Differential D6497

staging/kafka: decommission journal0
ClosedPublic
Actions

Authored by vsellier on Oct 18 2021, 3:14 PM.

Details

Reviewers

ardumont

Group Reviewers

System administrators

Maniphest Tasks

T3630: staging - journal0 needs more space

Commits

rSPSITE4516257094e0: staging/kafka: decommission journal0

Summary

Remove journal0 in site.pp
Remove now unused role swh_journal_allinone
Remove journal0 from the zookeeper and broker lists
use journal1 instead of storage1 in the journal client configurations

Related to T3630

Test Plan

counters0 (client part):

diff origin/production/counters0.internal.staging.swh.network current/counters0.internal.staging.swh.network
*******************************************
  File[/etc/softwareheritage/counters/journal_client.yml] =>
   parameters =>
     content =>
      @@ -5,6 +5,5 @@
       journal:
         brokers:
      -  - journal0.internal.staging.swh.network
      -  - storage1.internal.staging.swh.network
      +  - journal1.internal.staging.swh.network
         group_id: swh.counters.journal_client
         prefix: swh.journal.objects
*******************************************
*** End octocatalog-diff on counters0.internal.staging.swh.network

storage1:

diff origin/production/storage1.internal.staging.swh.network current/storage1.internal.staging.swh.network
*******************************************
  File[/etc/softwareheritage/journal/backfill.yml] =>
   parameters =>
     content =>
      @@ -9,6 +9,5 @@
         args:
           brokers:
      -    - journal0.internal.staging.swh.network
      -    - storage1.internal.staging.swh.network
      +    - journal1.internal.staging.swh.network
           prefix: swh.journal.objects
           client_id: swh.storage.journal_writer.storage1
*******************************************
  File[/etc/softwareheritage/storage/indexer.yml] =>
   parameters =>
     content =>
      @@ -7,6 +7,5 @@
           cls: kafka
           brokers:
      -    - journal0.internal.staging.swh.network
      -    - storage1.internal.staging.swh.network
      +    - journal1.internal.staging.swh.network
           prefix: swh.journal.indexed
           client_id: swh.idx_storage.journal_writer.storage1
*******************************************
  File[/etc/softwareheritage/storage/storage.yml] =>
   parameters =>
     content =>
      @@ -13,6 +13,5 @@
             args:
               brokers:
      -        - journal0.internal.staging.swh.network
      -        - storage1.internal.staging.swh.network
      +        - journal1.internal.staging.swh.network
               prefix: swh.journal.objects
               client_id: swh.storage.journal_writer.storage1
*******************************************
  File[/etc/zookeeper/conf/zoo.cfg] =>
   parameters =>
     content =>
      @@ -27,5 +27,4 @@
       #server.2=zookeeper2:2888:3888
       #server.3=zookeeper3:2888:3888
      -server.1=journal0.internal.staging.swh.network:2888:3888
       server.2=storage1.internal.staging.swh.network:2888:3888
      _
*******************************************
  File[/opt/kafka/config/server.properties] =>
   parameters =>
     content =>
      @@ -25,5 +25,5 @@
       ssl.keystore.location=/opt/kafka/config/broker.ks
       ssl.keystore.password=LBXrf6suU4cedMtM
      -super.users=User:broker-journal0.internal.staging.swh.network;User:broker-storage1.internal.staging.swh.network;User:swh-admin-olasd;User:ANONYMOUS
      -zookeeper.connect=journal0.internal.staging.swh.network:2181,journal1.internal.staging.swh.network:2181/kafka/softwareheritage
      +super.users=User:broker-storage1.internal.staging.swh.network;User:swh-admin-olasd;User:ANONYMOUS
      +zookeeper.connect=journal1.internal.staging.swh.network:2181/kafka/softwareheritage
       zookeeper.session.timeout.ms=18000
*******************************************
*** End octocatalog-diff on storage1.internal.staging.swh.network