diff --git a/docs/infrastructure/index.rst b/docs/infrastructure/index.rst --- a/docs/infrastructure/index.rst +++ b/docs/infrastructure/index.rst @@ -3,7 +3,7 @@ Infrastructure ############## -.. keep this in sync with the 'sysadm' section in swh-docs/docs/index.rst +.. keep this in sync with the 'infrastructure' section in swh-docs/docs/index.rst This section regroups the knowledge base and procedures relative to the |swh| infrastructure management. @@ -11,4 +11,5 @@ :maxdepth: 2 :titlesonly: + service-urls network diff --git a/docs/infrastructure/network.rst b/docs/infrastructure/network.rst --- a/docs/infrastructure/network.rst +++ b/docs/infrastructure/network.rst @@ -1,7 +1,7 @@ Network documentation ##################### -.. keep this in sync with the 'sysadm' section in swh-docs/docs/index.rst +.. keep this in sync with the 'infrastructure' section in swh-docs/docs/index.rst This section regroups the knowledge base for our network components. diff --git a/docs/infrastructure/service-urls.rst b/docs/infrastructure/service-urls.rst new file mode 100644 --- /dev/null +++ b/docs/infrastructure/service-urls.rst @@ -0,0 +1,191 @@ +Service urls +##################### + + +This section regroups the urls of the services + + +.. toctree:: + :maxdepth: 2 + :titlesonly: + +Staging +------- + +Try to use the staging environment as far as possible for your tests + +Public urls +~~~~~~~~~~~ + ++---------------------------------------+-------------------------------------------+ +| Service | URL | ++=======================================+===========================================+ +| swh-web | https://webapp.staging.swh.network | ++---------------------------------------+-------------------------------------------+ +| swh-deposit | https://deposit.staging.swh.network | ++---------------------------------------+-------------------------------------------+ +| swh-objstorage read-only (for mirror) | https://objstorage.staging.swh.network | ++---------------------------------------+-------------------------------------------+ +| Journal TLS | broker1.journal.softwareheritage.org:9093 | ++---------------------------------------+-------------------------------------------+ + +Internal services +~~~~~~~~~~~~~~~~~ + ++--------------------------+------------------------------------------------------+--------+------------+ +| Service | URL | VPN[1] | Private[2] | ++==========================+======================================================+========+============+ +| swh-storage | http://storage1.internal.staging.swh.network:5002 | | X | ++--------------------------+------------------------------------------------------+--------+------------+ +| swh-storage read-only | http://webapp.internal.staging.swh.network:5002 | X | | ++--------------------------+------------------------------------------------------+--------+------------+ +| swh-objstorage | http://storage1.internal.staging.swh.network:5003 | | X | ++--------------------------+------------------------------------------------------+--------+------------+ +| swh-objstorage read-only | http://objstorage0.internal.staging.swh.network:5003 | X | | ++--------------------------+------------------------------------------------------+--------+------------+ +| swh-scheduler | http://scheduler0.internal.staging.swh.network:5008 | X | | ++--------------------------+------------------------------------------------------+--------+------------+ +| swh-counters | http://counters0.internal.staging.swh.network:5011 | X | | ++--------------------------+------------------------------------------------------+--------+------------+ +| swh-search | http://webapp.internal.staging.swh.network:5010 | X | | ++--------------------------+------------------------------------------------------+--------+------------+ +| swh-search | http://search0.internal.staging.swh.network:5010 | | X | ++--------------------------+------------------------------------------------------+--------+------------+ +| swh-vault | http://vault.internal.staging.swh.network:5005 | | X | ++--------------------------+------------------------------------------------------+--------+------------+ +| Journal plaintext | journal0.internal.staging.swh.network:9092 | | X | ++--------------------------+------------------------------------------------------+--------+------------+ +| Journal internal TLS | journal0.internal.staging.swh.network:9094 | | X | ++--------------------------+------------------------------------------------------+--------+------------+ + +SWH backends +~~~~~~~~~~~~ + ++--------------------+---------------------------------------------------------+--------+------------+ +| Backend | URL | VPN[1] | Private[2] | ++====================+=========================================================+========+============+ +| RabbitMq GUI | http://scheduler0.internal.staging.swh.network:15672 | X | | ++--------------------+---------------------------------------------------------+--------+------------+ +| archive database | db1.internal.staging.swh.network:5432/swh | X | | ++--------------------+---------------------------------------------------------+--------+------------+ +| webapp database | db1.internal.staging.swh.network:5432/swh-web | X | | ++--------------------+---------------------------------------------------------+--------+------------+ +| deposit database | db1.internal.staging.swh.network:5432/swh-deposit | X | | ++--------------------+---------------------------------------------------------+--------+------------+ +| vault database | db1.internal.staging.swh.network:5432/swh-vault | X | | ++--------------------+---------------------------------------------------------+--------+------------+ +| scheduler database | db1.internal.staging.swh.network:5432/swh-scheduler | X | | ++--------------------+---------------------------------------------------------+--------+------------+ +| lister database | db1.internal.staging.swh.network:5432/swh-lister | X | | ++--------------------+---------------------------------------------------------+--------+------------+ +| swh-search ES | http://search-esnode0.internal.staging.swh.network:9200 | | X | ++--------------------+---------------------------------------------------------+--------+------------+ +| Counters redis | counters0.internal.staging.swh.network:6379 | | X | ++--------------------+---------------------------------------------------------+--------+------------+ + +Production +---------- + +.. _public-urls-1: + +Public urls +~~~~~~~~~~~ + ++---------------------------------------+-----------------------------------------------+ +| Service | URL | ++=======================================+===============================================+ +| swh-web | https://archive.softwareheritage.org | ++---------------------------------------+-----------------------------------------------+ +| swh-deposit | https://deposit.softwareheritage.org | ++---------------------------------------+-----------------------------------------------+ +| swh-objstorage read-only (for mirror) | N/A | ++---------------------------------------+-----------------------------------------------+ +| Journal TLS | broker[1-4].journal.softwareheritage.org:9093 | ++---------------------------------------+-----------------------------------------------+ + +.. _internal-services-1: + +Internal services +~~~~~~~~~~~~~~~~~ + ++--------------------------+----------------------------------------------------------------+--------+------------+ +| Service | URL | VPN[1] | Private[2] | ++==========================+================================================================+========+============+ +| swh-web test/validation | https://webapp1.internal.softwareheritage.org | X | | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-storage | http://saam.internal.softwareheritage.org:5002 | | X | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-storage read-only | http://webapp1.internal.softwareheritage.org:5002 | X | | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-storage read-only | http://moma.internal.softwareheritage.org:5002 | X | | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-objstorage | http://saam.internal.softwareheritage.org:5003 | | X | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-objstorage read-only | N/A | | | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-scheduler | http://saatchi.internal.softwareheritage.org:5008 | X | | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-counters | http://counters1.internal.softwareheritage.org:5011 | X | | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-search | http://webapp1.internal.softwareheritage.org:5010 | X | | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-search | http://moma.internal.softwareheritage.org:5010 | X | | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-search | http://search1.internal.softwareheritage.org:5010 | | X | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| swh-vault | http://vangogh.euwest.azure.internal.softwareheritage.org:5005 | | X | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| Journal plaintext | kafka[1-4].internal.softwareheritage.org:9092 | | X | ++--------------------------+----------------------------------------------------------------+--------+------------+ +| Journal internal TLS | kafka[1-4].internal.softwareheritage.org:9094 | X | | ++--------------------------+----------------------------------------------------------------+--------+------------+ + +.. _swh-backends-1: + +SWH backends +~~~~~~~~~~~~ + ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| Backend | URL | VPN[1] | Private[2] | ++==========================+=======================================================================+========+============+ +| RabbitMq GUI | http://saatchi.internal.softwareheritage.org:15672 | X | | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| archive database replica | somerset.internal.softwareheritage.org:5432/softwareheritage | X | | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| archive database main | belvedere.internal.softwareheritage.org:5432/softwareheritage | X | | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| webapp database main | belvedere.internal.softwareheritage.org:5432/swh-web | X | | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| scheduler database | belvedere.internal.softwareheritage.org:5432/swh-scheduler | X | | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| lister database | belvedere.internal.softwareheritage.org:5432/swh-lister | X | | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| deposit database | belvedere.internal.softwareheritage.org:5432/softwareheritage-deposit | X | | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| vault database | belvedere.internal.softwareheritage.org:5432/swh-vault | X | | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| swh-search ES | http://search-esnode[1-3].internal.softwareheritage.org:9200 | | X | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ +| Counters redis | counters1.internal.softwareheritage.org:6379 | | X | ++--------------------------+-----------------------------------------------------------------------+--------+------------+ + +Other tools +----------- + ++-------------------+-------------------------------------------------------+--------------------+--------+------------+ +| Tool | URL | Public | VPN[1] | Private[2] | ++===================+=======================================================+====================+========+============+ +| grafana | https://grafana.softwareheritage.org | X | | | ++-------------------+-------------------------------------------------------+--------------------+--------+------------+ +| Kibana | http://kibana0.internal.softwareheritage.org:5601 | | X | | ++-------------------+-------------------------------------------------------+--------------------+--------+------------+ +| Log Elasticsearch | http://search[1-3].internal.softwareheritage.org:9200 | | X | | ++-------------------+-------------------------------------------------------+--------------------+--------+------------+ +| C.M.A.K. | http://getty.internal.softwareheritage.org:9000 | | X | | ++-------------------+-------------------------------------------------------+--------------------+--------+------------+ +| Sentry | https://sentry.softwareheritage.org | X (authentication) | | | ++-------------------+-------------------------------------------------------+--------------------+--------+------------+ + +[1] VPN: URL only accessible when connected to the SoftwareHeritage VPN + +[2] Private: URL only accessible from the internal network, i.e nor public neither accessible through the VPN.