Page MenuHomeSoftware Heritage
Differential D4460

staging: Fix internal webapp and deposit communication
ClosedPublic
Actions

Authored by ardumont on Nov 10 2020, 8:19 PM.

Details

Reviewers
vsellier
Group Reviewers
System administrators
Maniphest Tasks
T2747: Create the reverse proxy to expose the staging services publicly
Commits
rSPSITEdd8b01b8ab5d: staging: Fix internal webapp and deposit communication
Summary

Use only internal address instead of using public one

It fixes problems to reach the public ip from the internal network.

Related to T2747

Test Plan

octocatalog diff:

  • moma:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff moma
Found host moma.softwareheritage.org
Cloning into '/tmp/swh-ocd.qk7L3P6B/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.qk7L3P6B/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host moma.softwareheritage.org
I, [2020-11-10T20:11:05.731726 #19793]  INFO -- : Catalogs compiled for moma.softwareheritage.org
I, [2020-11-10T20:11:07.230655 #19793]  INFO -- : Diffs computed for moma.softwareheritage.org
I, [2020-11-10T20:11:07.230688 #19793]  INFO -- : No differences
*** End octocatalog-diff on moma.softwareheritage.org
  • worker01:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff worker01
Found host worker01.softwareheritage.org
Cloning into '/tmp/swh-ocd.vj59cQWZ/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.vj59cQWZ/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host worker01.softwareheritage.org
I, [2020-11-10T20:11:23.613764 #24774]  INFO -- : Catalogs compiled for worker01.softwareheritage.org
I, [2020-11-10T20:11:24.611490 #24774]  INFO -- : Diffs computed for worker01.softwareheritage.org
I, [2020-11-10T20:11:24.611525 #24774]  INFO -- : No differences
  • webapp.staging:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff webapp.internal.staging.swh.network         
Found host webapp.internal.staging.swh.network
Cloning into '/tmp/swh-ocd.54J9e2XS/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.54J9e2XS/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host webapp.internal.staging.swh.network
I, [2020-11-10T20:13:19.996182 #30174]  INFO -- : Catalogs compiled for webapp.internal.staging.swh.network
I, [2020-11-10T20:13:21.207548 #30174]  INFO -- : Diffs computed for webapp.internal.staging.swh.network
diff origin/production/webapp.internal.staging.swh.network current/webapp.internal.staging.swh.network
*******************************************
  File[/etc/softwareheritage/web/web.yml] =>
   parameters =>
     content =>
      @@ -58,5 +58,5 @@
       es_workers_index_url: http://esnode1.internal.softwareheritage.org:9200/swh_workers-*
       deposit:
      -  private_api_url: http://deposit.internal.staging.swh.network/1/private/
      +  private_api_url: https://deposit-rp.internal.staging.swh.network/1/private/
         private_api_user: deposit_basic_auth_swhworker_username
         private_api_password: deposit_basic_auth_swhworker_password
*******************************************
*** End octocatalog-diff on webapp.internal.staging.swh.network
  • deposit.staging:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff deposit.internal.staging.swh.network   
Found host deposit.internal.staging.swh.network
Cloning into '/tmp/swh-ocd.qIZIcMvk/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.qIZIcMvk/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host deposit.internal.staging.swh.network
I, [2020-11-10T20:14:23.566176 #13282]  INFO -- : Catalogs compiled for deposit.internal.staging.swh.network
I, [2020-11-10T20:14:24.802902 #13282]  INFO -- : Diffs computed for deposit.internal.staging.swh.network
I, [2020-11-10T20:14:24.802936 #13282]  INFO -- : No differences
*** End octocatalog-diff on deposit.internal.staging.swh.network
  • worker0.staging:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff worker0.internal.staging.swh.network
Found host worker0.internal.staging.swh.network
Cloning into '/tmp/swh-ocd.tbynahLj/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.tbynahLj/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host worker0.internal.staging.swh.network
I, [2020-11-10T20:13:38.612655 #3083]  INFO -- : Catalogs compiled for worker0.internal.staging.swh.network
I, [2020-11-10T20:13:39.569152 #3083]  INFO -- : Diffs computed for worker0.internal.staging.swh.network
diff origin/production/worker0.internal.staging.swh.network current/worker0.internal.staging.swh.network
*******************************************
  File[/etc/softwareheritage/checker_deposit.yml] =>
   parameters =>
     content =>
      @@ -12,5 +12,5 @@
         - swh.deposit.loader.tasks.ChecksDepositTsk
       deposit:
      -  url: http://deposit.internal.staging.swh.network/1/private/
      +  url: https://deposit-rp.internal.staging.swh.network/1/private/
         auth:
           username: deposit_basic_auth_swhworker_username
*******************************************
  File[/etc/softwareheritage/loader_deposit.yml] =>
   parameters =>
     content =>
      @@ -22,5 +22,5 @@
         - swh.loader.package.deposit.tasks.LoadDeposit
       deposit:
      -  url: http://deposit.internal.staging.swh.network/1/private/
      +  url: https://deposit-rp.internal.staging.swh.network/1/private/
         auth:
           username: deposit_basic_auth_swhworker_username
*******************************************
*** End octocatalog-diff on worker0.internal.staging.swh.network
  • rp0.staging:
➜  puppet-environment git:(master) ✗ bin/octocatalog-diff rp0.internal.staging.swh.network   
Found host rp0.internal.staging.swh.network
Cloning into '/tmp/swh-ocd.zloi5Das/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.zloi5Das/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host rp0.internal.staging.swh.network
I, [2020-11-10T20:14:38.252981 #18477]  INFO -- : Catalogs compiled for rp0.internal.staging.swh.network
I, [2020-11-10T20:14:39.205483 #18477]  INFO -- : Diffs computed for rp0.internal.staging.swh.network
I, [2020-11-10T20:14:39.205517 #18477]  INFO -- : No differences
*** End octocatalog-diff on rp0.internal.staging.swh.network

Diff Detail

Repository
rSPSITE puppet-swh-site
Branch
staging
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 17024
Build 26270: arc lint + arc unit

Event Timeline

vsellier created this revision.Nov 10 2020, 8:19 PM
Harbormaster completed remote builds in B17023: Diff 15813.Nov 10 2020, 8:19 PM
vsellier added a comment.Nov 10 2020, 8:21 PM

It fixes problems to reach the public ip from the internal network.
Feel free to land it if it looks good to you

vsellier updated this revision to Diff 15814.Nov 10 2020, 8:23 PM

use https

Harbormaster completed remote builds in B17024: Diff 15814.Nov 10 2020, 8:23 PM
ardumont accepted this revision.Nov 11 2020, 10:26 AM
ardumont edited the summary of this revision. (Show Details)
ardumont added a subscriber: ardumont.

makes sense.

This revision is now accepted and ready to land.Nov 11 2020, 10:26 AM
ardumont commandeered this revision.Nov 12 2020, 3:48 PM
ardumont edited reviewers, added: vsellier; removed: ardumont.
This revision now requires review to proceed.Nov 12 2020, 3:48 PM
ardumont updated this revision to Diff 15857.EditedNov 12 2020, 3:53 PM

Just owning the patch so when pushed, it will get closed alongside.

Doing it because i'm gonna deploy it, it will indeed fix issues on at least
deposit related workers

Harbormaster completed remote builds in B17067: Diff 15857.Nov 12 2020, 3:53 PM
This revision was not accepted when it landed; it landed in state Needs Review.Nov 12 2020, 3:55 PM
Closed by commit rSPSITEdd8b01b8ab5d: staging: Fix internal webapp and deposit communication (authored by vsellier, committed by ardumont). · Explain Why
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rSPSITEdd8b01b8ab5d: staging: Fix internal webapp and deposit communication.

Revision Contents
Changeset List

PathSize
data/
common/
5 lines
deployments/
staging/
3 lines

Diff 15814

View Options

data/common/common.yaml

Loading...
View Options

data/deployments/staging/common.yaml

Loading...
About · Developer information · Legal