Page MenuHomeSoftware Heritage

azure/terraform: Add azure vault vm using terraform
ClosedPublic

Authored by ardumont on May 20 2019, 6:31 PM.

Details

Summary

Related T1716

Test Plan
($ terraform init)
$ terraform plan

Output:

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.azurerm_subnet.default: Refreshing state...
data.azurerm_network_security_group.worker-nsg: Refreshing state...

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + azurerm_network_interface.vault-servers_interfaces
      id:                                                                 <computed>
      applied_dns_servers.#:                                              <computed>
      dns_servers.#:                                                      <computed>
      enable_accelerated_networking:                                      "false"
      enable_ip_forwarding:                                               "false"
      internal_dns_name_label:                                            <computed>
      internal_fqdn:                                                      <computed>
      ip_configuration.#:                                                 "1"
      ip_configuration.0.application_gateway_backend_address_pools_ids.#: <computed>
      ip_configuration.0.application_security_group_ids.#:                <computed>
      ip_configuration.0.load_balancer_backend_address_pools_ids.#:       <computed>
      ip_configuration.0.load_balancer_inbound_nat_rules_ids.#:           <computed>
      ip_configuration.0.name:                                            "vaultNicConfiguration"
      ip_configuration.0.primary:                                         <computed>
      ip_configuration.0.private_ip_address_allocation:                   "dynamic"
      ip_configuration.0.private_ip_address_version:                      "IPv4"
      ip_configuration.0.subnet_id:                                       "/subscriptions/<redacted>/resourceGroups/swh-resource/providers/Microsoft.Network/virtualNetworks/swh-vnet/subnets/default"
      location:                                                           "westeurope"
      mac_address:                                                        <computed>
      name:                                                               "vault-server-0-interface"
      network_security_group_id:                                          "/subscriptions/<redacted>/resourceGroups/swh-resource/providers/Microsoft.Network/networkSecurityGroups/worker-nsg"
      private_ip_address:                                                 <computed>
      private_ip_addresses.#:                                             <computed>
      resource_group_name:                                                "euwest-vault"
      tags.%:                                                             <computed>
      virtual_machine_id:                                                 <computed>

  + azurerm_resource_group.euwest-vault
      id:                                                                 <computed>
      location:                                                           "westeurope"
      name:                                                               "euwest-vault"
      tags.%:                                                             "1"
      tags.environment:                                                   "SWH Vault"

  + azurerm_storage_account.vault-storage
      id:                                                                 <computed>
      access_tier:                                                        "Cool"
      account_encryption_source:                                          "Microsoft.Storage"
      account_kind:                                                       "BlobStorage"
      account_replication_type:                                           "LRS"
      account_tier:                                                       "Standard"
      enable_blob_encryption:                                             "false"
      enable_file_encryption:                                             "false"
      identity.#:                                                         <computed>
      is_hns_enabled:                                                     "false"
      location:                                                           "westeurope"
      name:                                                               "vaultstorage"
      primary_access_key:                                                 <computed>
      primary_blob_connection_string:                                     <computed>
      primary_blob_endpoint:                                              <computed>
      primary_blob_host:                                                  <computed>
      primary_connection_string:                                          <computed>
      primary_dfs_endpoint:                                               <computed>
      primary_dfs_host:                                                   <computed>
      primary_file_endpoint:                                              <computed>
      primary_file_host:                                                  <computed>
      primary_location:                                                   <computed>
      primary_queue_endpoint:                                             <computed>
      primary_queue_host:                                                 <computed>
      primary_table_endpoint:                                             <computed>
      primary_table_host:                                                 <computed>
      primary_web_endpoint:                                               <computed>
      primary_web_host:                                                   <computed>
      resource_group_name:                                                "euwest-vault"
      secondary_access_key:                                               <computed>
      secondary_blob_connection_string:                                   <computed>
      secondary_blob_endpoint:                                            <computed>
      secondary_blob_host:                                                <computed>
      secondary_connection_string:                                        <computed>
      secondary_dfs_endpoint:                                             <computed>
      secondary_dfs_host:                                                 <computed>
      secondary_file_endpoint:                                            <computed>
      secondary_file_host:                                                <computed>
      secondary_location:                                                 <computed>
      secondary_queue_endpoint:                                           <computed>
      secondary_queue_host:                                               <computed>
      secondary_table_endpoint:                                           <computed>
      secondary_table_host:                                               <computed>
      secondary_web_endpoint:                                             <computed>
      secondary_web_host:                                                 <computed>
      tags.%:                                                             "1"
      tags.environment:                                                   "SWH Vault"

  + azurerm_virtual_machine.vault-servers
      id:                                                                 <computed>
      availability_set_id:                                                <computed>
      delete_data_disks_on_termination:                                   "false"
      delete_os_disk_on_termination:                                      "false"
      identity.#:                                                         <computed>
      location:                                                           "westeurope"
      name:                                                               "vatican"
      network_interface_ids.#:                                            <computed>
      os_profile.#:                                                       "1"
      os_profile.3272332376.admin_password:                               <sensitive>
      os_profile.3272332376.admin_username:                               "ardumont"
      os_profile.3272332376.computer_name:                                "vatican"
      os_profile.3272332376.custom_data:                                  <computed>
      os_profile_linux_config.#:                                          "1"
      os_profile_linux_config.69840937.disable_password_authentication:   "true"
      os_profile_linux_config.69840937.ssh_keys.#:                        "1"
      os_profile_linux_config.69840937.ssh_keys.0.key_data:               "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZarzgHrzUYspvrgSI6fszrALo92BDys7QOkJgUfZa9t9m4g7dUANNtwBiqIbqijAQPmB1zKgG6QTZC5rJkRy6KqXCW/+Qeedw/FWIbuI7jOD5WxnglbEQgvPkkB8kf1xIF7icRfWcQmK2je/3sFd9yS4/+jftNMPPXkBCxYm74onMenyllA1akA8FLyujLu6MNA1D8iLLXvz6pBDTT4GZ5/bm3vSE6Go8Xbuyu4SCtYZSHaHC2lXZ6Hhi6dbli4d3OwkUWz+YhFGaEra5Fx45Iig4UCL6kXPkvL/oSc9KGerpT//Xj9qz1K7p/IrBS8+eA4X69bHYYV0UZKDADZSn ardumont@bespin"
      os_profile_linux_config.69840937.ssh_keys.0.path:                   "/home/ardumont/.ssh/authorized_keys"
      resource_group_name:                                                "euwest-vault"
      storage_data_disk.#:                                                <computed>
      storage_image_reference.#:                                          "1"
      storage_image_reference.1202893792.id:                              ""
      storage_image_reference.1202893792.offer:                           "Debian"
      storage_image_reference.1202893792.publisher:                       "credativ"
      storage_image_reference.1202893792.sku:                             "9"
      storage_image_reference.1202893792.version:                         "latest"
      storage_os_disk.#:                                                  "1"
      storage_os_disk.0.caching:                                          "ReadWrite"
      storage_os_disk.0.create_option:                                    "FromImage"
      storage_os_disk.0.disk_size_gb:                                     <computed>
      storage_os_disk.0.managed_disk_id:                                  <computed>
      storage_os_disk.0.managed_disk_type:                                "Premium_LRS"
      storage_os_disk.0.name:                                             "vault-server-0_osdisk"
      storage_os_disk.0.write_accelerator_enabled:                        "false"
      tags.%:                                                             "1"
      tags.environment:                                                   "SWH Vault"
      vm_size:                                                            "Standard_DS2_v2"


Plan: 4 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

Diff Detail

Repository
rSPRE sysadm-provisioning
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 5835
Build 7993: arc lint + arc unit

Event Timeline

.gitignore
1

As far as i understood it, this is filled with needed plugins installed by terraform init.

azure/terraform/README.md
29

Maybe we should have a generic admin user with a key in the swh password-store and set that as default here.

azure/terraform/variables.tf
8

Here would be the use case for the generic admin user and its associated public key.
(Maybe we already have that somewhere, i don't remember).

lgtm

azure/terraform/vault.tf
69

s/servers/workers/ ?

77

What about a B2ms or B4ms? They are roughly the same price, but you get more "burst" CPU power, which is good for a worker

azure/terraform/vault.tf
69

It's for the vault's api and the objstorage's api, that's why i called it servers.

The cooking workers are already set up (and not part of this, but could be? <- outside of the diff's scope ;)

77

I did not get into that much details yet.
Thanks for reminding me. I kept the initial reference of the worker machine.

The use case here:

  • 2 apis (objstorage, cooking) running on the same machine.
  • discussion with a remote db (cache on prado)
  • discussion with a remote storage (running on azure)

Comparison:

|-----------+----------+-----------------+-------+------+-----------+----------+--------------+----------------|
| vm size   | offering | family          | vcpus | rams | data disk | max iops | temp storage | cost/month (€) |
|-----------+----------+-----------------+-------+------+-----------+----------+--------------+----------------|
| B2ms      | Standard | General purpose |     2 |    8 |         4 |     2400 | 16 GB        |          52.20 |
| B4ms      | Standard | General purpose |     4 |   16 |         8 |     3600 | 32 GB        |         104.15 |
| DS2_v2    | Standard | General purpose |     2 |    7 |         8 |     6400 | 14 GB        |          91.60 |
| D2s_v3    | Standard | General purpose |     2 |    8 |         4 |     3200 | 16 GB        |          60.23 |
|-----------+----------+-----------------+-------+------+-----------+----------+--------------+----------------|
| orangerie | X        | X               |     2 |    2 |         X |        ? |              |                |
|-----------+----------+-----------------+-------+------+-----------+----------+--------------+----------------|

The actual machine running the vault is orangerie.

Use B2ms as default vm size machine

Rename vaultstorage blobstorage to swhvaultstorage

As otherwise, terraform complains it already exists.

Add missing container for the new blob storage

azure/terraform/README.md
1

I'd be delighted to have a 'what' part, i.e. an explanation of what this does, with a description of the deployed architecture, before entering the 'how' part.

azure/terraform/vault.tf
86

Count is not needed, so this will be simplified (other instances as well).

109

Thanks for olasd's heads up, it's not one museum, so i'll change to 'vangogh' instead.

Simplify resources declaration

  • azure/terraform/README.md: Describe what the vault.tf is about
This revision was not accepted when it landed; it landed in state Needs Review.May 21 2019, 5:00 PM
This revision was automatically updated to reflect the committed changes.