Page MenuHomeSoftware Heritage

staging: Fix internal webapp and deposit communication
ClosedPublic

Authored by ardumont on Nov 10 2020, 8:19 PM.

Details

Summary

Use only internal address instead of using public one

It fixes problems to reach the public ip from the internal network.

Related to T2747

Test Plan

octocatalog diff:

  • moma:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff moma
Found host moma.softwareheritage.org
Cloning into '/tmp/swh-ocd.qk7L3P6B/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.qk7L3P6B/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host moma.softwareheritage.org
I, [2020-11-10T20:11:05.731726 #19793]  INFO -- : Catalogs compiled for moma.softwareheritage.org
I, [2020-11-10T20:11:07.230655 #19793]  INFO -- : Diffs computed for moma.softwareheritage.org
I, [2020-11-10T20:11:07.230688 #19793]  INFO -- : No differences
*** End octocatalog-diff on moma.softwareheritage.org
  • worker01:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff worker01
Found host worker01.softwareheritage.org
Cloning into '/tmp/swh-ocd.vj59cQWZ/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.vj59cQWZ/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host worker01.softwareheritage.org
I, [2020-11-10T20:11:23.613764 #24774]  INFO -- : Catalogs compiled for worker01.softwareheritage.org
I, [2020-11-10T20:11:24.611490 #24774]  INFO -- : Diffs computed for worker01.softwareheritage.org
I, [2020-11-10T20:11:24.611525 #24774]  INFO -- : No differences
  • webapp.staging:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff webapp.internal.staging.swh.network         
Found host webapp.internal.staging.swh.network
Cloning into '/tmp/swh-ocd.54J9e2XS/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.54J9e2XS/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host webapp.internal.staging.swh.network
I, [2020-11-10T20:13:19.996182 #30174]  INFO -- : Catalogs compiled for webapp.internal.staging.swh.network
I, [2020-11-10T20:13:21.207548 #30174]  INFO -- : Diffs computed for webapp.internal.staging.swh.network
diff origin/production/webapp.internal.staging.swh.network current/webapp.internal.staging.swh.network
*******************************************
  File[/etc/softwareheritage/web/web.yml] =>
   parameters =>
     content =>
      @@ -58,5 +58,5 @@
       es_workers_index_url: http://esnode1.internal.softwareheritage.org:9200/swh_workers-*
       deposit:
      -  private_api_url: http://deposit.internal.staging.swh.network/1/private/
      +  private_api_url: https://deposit-rp.internal.staging.swh.network/1/private/
         private_api_user: deposit_basic_auth_swhworker_username
         private_api_password: deposit_basic_auth_swhworker_password
*******************************************
*** End octocatalog-diff on webapp.internal.staging.swh.network
  • deposit.staging:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff deposit.internal.staging.swh.network   
Found host deposit.internal.staging.swh.network
Cloning into '/tmp/swh-ocd.qIZIcMvk/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.qIZIcMvk/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host deposit.internal.staging.swh.network
I, [2020-11-10T20:14:23.566176 #13282]  INFO -- : Catalogs compiled for deposit.internal.staging.swh.network
I, [2020-11-10T20:14:24.802902 #13282]  INFO -- : Diffs computed for deposit.internal.staging.swh.network
I, [2020-11-10T20:14:24.802936 #13282]  INFO -- : No differences
*** End octocatalog-diff on deposit.internal.staging.swh.network
  • worker0.staging:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff worker0.internal.staging.swh.network
Found host worker0.internal.staging.swh.network
Cloning into '/tmp/swh-ocd.tbynahLj/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.tbynahLj/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host worker0.internal.staging.swh.network
I, [2020-11-10T20:13:38.612655 #3083]  INFO -- : Catalogs compiled for worker0.internal.staging.swh.network
I, [2020-11-10T20:13:39.569152 #3083]  INFO -- : Diffs computed for worker0.internal.staging.swh.network
diff origin/production/worker0.internal.staging.swh.network current/worker0.internal.staging.swh.network
*******************************************
  File[/etc/softwareheritage/checker_deposit.yml] =>
   parameters =>
     content =>
      @@ -12,5 +12,5 @@
         - swh.deposit.loader.tasks.ChecksDepositTsk
       deposit:
      -  url: http://deposit.internal.staging.swh.network/1/private/
      +  url: https://deposit-rp.internal.staging.swh.network/1/private/
         auth:
           username: deposit_basic_auth_swhworker_username
*******************************************
  File[/etc/softwareheritage/loader_deposit.yml] =>
   parameters =>
     content =>
      @@ -22,5 +22,5 @@
         - swh.loader.package.deposit.tasks.LoadDeposit
       deposit:
      -  url: http://deposit.internal.staging.swh.network/1/private/
      +  url: https://deposit-rp.internal.staging.swh.network/1/private/
         auth:
           username: deposit_basic_auth_swhworker_username
*******************************************
*** End octocatalog-diff on worker0.internal.staging.swh.network
  • rp0.staging:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff rp0.internal.staging.swh.network   
Found host rp0.internal.staging.swh.network
Cloning into '/tmp/swh-ocd.zloi5Das/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.zloi5Das/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host rp0.internal.staging.swh.network
I, [2020-11-10T20:14:38.252981 #18477]  INFO -- : Catalogs compiled for rp0.internal.staging.swh.network
I, [2020-11-10T20:14:39.205483 #18477]  INFO -- : Diffs computed for rp0.internal.staging.swh.network
I, [2020-11-10T20:14:39.205517 #18477]  INFO -- : No differences
*** End octocatalog-diff on rp0.internal.staging.swh.network

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

It fixes problems to reach the public ip from the internal network.
Feel free to land it if it looks good to you

ardumont edited the summary of this revision. (Show Details)
ardumont added a subscriber: ardumont.

makes sense.

This revision is now accepted and ready to land.Nov 11 2020, 10:26 AM
ardumont edited reviewers, added: vsellier; removed: ardumont.
This revision now requires review to proceed.Nov 12 2020, 3:48 PM

Just owning the patch so when pushed, it will get closed alongside.

Doing it because i'm gonna deploy it, it will indeed fix issues on at least
deposit related workers

This revision was not accepted when it landed; it landed in state Needs Review.Nov 12 2020, 3:55 PM
This revision was automatically updated to reflect the committed changes.