Page MenuHomeSoftware Heritage

azure/terraform: Add azure vault vm using terraform
ClosedPublic

Authored by ardumont on Mon, May 20, 6:31 PM.

Details

Summary

Related T1716

Test Plan
($ terraform init)
$ terraform plan

Output:

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.azurerm_subnet.default: Refreshing state...
data.azurerm_network_security_group.worker-nsg: Refreshing state...

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + azurerm_network_interface.vault-servers_interfaces
      id:                                                                 <computed>
      applied_dns_servers.#:                                              <computed>
      dns_servers.#:                                                      <computed>
      enable_accelerated_networking:                                      "false"
      enable_ip_forwarding:                                               "false"
      internal_dns_name_label:                                            <computed>
      internal_fqdn:                                                      <computed>
      ip_configuration.#:                                                 "1"
      ip_configuration.0.application_gateway_backend_address_pools_ids.#: <computed>
      ip_configuration.0.application_security_group_ids.#:                <computed>
      ip_configuration.0.load_balancer_backend_address_pools_ids.#:       <computed>
      ip_configuration.0.load_balancer_inbound_nat_rules_ids.#:           <computed>
      ip_configuration.0.name:                                            "vaultNicConfiguration"
      ip_configuration.0.primary:                                         <computed>
      ip_configuration.0.private_ip_address_allocation:                   "dynamic"
      ip_configuration.0.private_ip_address_version:                      "IPv4"
      ip_configuration.0.subnet_id:                                       "/subscriptions/<redacted>/resourceGroups/swh-resource/providers/Microsoft.Network/virtualNetworks/swh-vnet/subnets/default"
      location:                                                           "westeurope"
      mac_address:                                                        <computed>
      name:                                                               "vault-server-0-interface"
      network_security_group_id:                                          "/subscriptions/<redacted>/resourceGroups/swh-resource/providers/Microsoft.Network/networkSecurityGroups/worker-nsg"
      private_ip_address:                                                 <computed>
      private_ip_addresses.#:                                             <computed>
      resource_group_name:                                                "euwest-vault"
      tags.%:                                                             <computed>
      virtual_machine_id:                                                 <computed>

  + azurerm_resource_group.euwest-vault
      id:                                                                 <computed>
      location:                                                           "westeurope"
      name:                                                               "euwest-vault"
      tags.%:                                                             "1"
      tags.environment:                                                   "SWH Vault"

  + azurerm_storage_account.vault-storage
      id:                                                                 <computed>
      access_tier:                                                        "Cool"
      account_encryption_source:                                          "Microsoft.Storage"
      account_kind:                                                       "BlobStorage"
      account_replication_type:                                           "LRS"
      account_tier:                                                       "Standard"
      enable_blob_encryption:                                             "false"
      enable_file_encryption:                                             "false"
      identity.#:                                                         <computed>
      is_hns_enabled:                                                     "false"
      location:                                                           "westeurope"
      name:                                                               "vaultstorage"
      primary_access_key:                                                 <computed>
      primary_blob_connection_string:                                     <computed>
      primary_blob_endpoint:                                              <computed>
      primary_blob_host:                                                  <computed>
      primary_connection_string:                                          <computed>
      primary_dfs_endpoint:                                               <computed>
      primary_dfs_host:                                                   <computed>
      primary_file_endpoint:                                              <computed>
      primary_file_host:                                                  <computed>
      primary_location:                                                   <computed>
      primary_queue_endpoint:                                             <computed>
      primary_queue_host:                                                 <computed>
      primary_table_endpoint:                                             <computed>
      primary_table_host:                                                 <computed>
      primary_web_endpoint:                                               <computed>
      primary_web_host:                                                   <computed>
      resource_group_name:                                                "euwest-vault"
      secondary_access_key:                                               <computed>
      secondary_blob_connection_string:                                   <computed>
      secondary_blob_endpoint:                                            <computed>
      secondary_blob_host:                                                <computed>
      secondary_connection_string:                                        <computed>
      secondary_dfs_endpoint:                                             <computed>
      secondary_dfs_host:                                                 <computed>
      secondary_file_endpoint:                                            <computed>
      secondary_file_host:                                                <computed>
      secondary_location:                                                 <computed>
      secondary_queue_endpoint:                                           <computed>
      secondary_queue_host:                                               <computed>
      secondary_table_endpoint:                                           <computed>
      secondary_table_host:                                               <computed>
      secondary_web_endpoint:                                             <computed>
      secondary_web_host:                                                 <computed>
      tags.%:                                                             "1"
      tags.environment:                                                   "SWH Vault"

  + azurerm_virtual_machine.vault-servers
      id:                                                                 <computed>
      availability_set_id:                                                <computed>
      delete_data_disks_on_termination:                                   "false"
      delete_os_disk_on_termination:                                      "false"
      identity.#:                                                         <computed>
      location:                                                           "westeurope"
      name:                                                               "vatican"
      network_interface_ids.#:                                            <computed>
      os_profile.#:                                                       "1"
      os_profile.3272332376.admin_password:                               <sensitive>
      os_profile.3272332376.admin_username:                               "ardumont"
      os_profile.3272332376.computer_name:                                "vatican"
      os_profile.3272332376.custom_data:                                  <computed>
      os_profile_linux_config.#:                                          "1"
      os_profile_linux_config.69840937.disable_password_authentication:   "true"
      os_profile_linux_config.69840937.ssh_keys.#:                        "1"
      os_profile_linux_config.69840937.ssh_keys.0.key_data:               "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZarzgHrzUYspvrgSI6fszrALo92BDys7QOkJgUfZa9t9m4g7dUANNtwBiqIbqijAQPmB1zKgG6QTZC5rJkRy6KqXCW/+Qeedw/FWIbuI7jOD5WxnglbEQgvPkkB8kf1xIF7icRfWcQmK2je/3sFd9yS4/+jftNMPPXkBCxYm74onMenyllA1akA8FLyujLu6MNA1D8iLLXvz6pBDTT4GZ5/bm3vSE6Go8Xbuyu4SCtYZSHaHC2lXZ6Hhi6dbli4d3OwkUWz+YhFGaEra5Fx45Iig4UCL6kXPkvL/oSc9KGerpT//Xj9qz1K7p/IrBS8+eA4X69bHYYV0UZKDADZSn ardumont@bespin"
      os_profile_linux_config.69840937.ssh_keys.0.path:                   "/home/ardumont/.ssh/authorized_keys"
      resource_group_name:                                                "euwest-vault"
      storage_data_disk.#:                                                <computed>
      storage_image_reference.#:                                          "1"
      storage_image_reference.1202893792.id:                              ""
      storage_image_reference.1202893792.offer:                           "Debian"
      storage_image_reference.1202893792.publisher:                       "credativ"
      storage_image_reference.1202893792.sku:                             "9"
      storage_image_reference.1202893792.version:                         "latest"
      storage_os_disk.#:                                                  "1"
      storage_os_disk.0.caching:                                          "ReadWrite"
      storage_os_disk.0.create_option:                                    "FromImage"
      storage_os_disk.0.disk_size_gb:                                     <computed>
      storage_os_disk.0.managed_disk_id:                                  <computed>
      storage_os_disk.0.managed_disk_type:                                "Premium_LRS"
      storage_os_disk.0.name:                                             "vault-server-0_osdisk"
      storage_os_disk.0.write_accelerator_enabled:                        "false"
      tags.%:                                                             "1"
      tags.environment:                                                   "SWH Vault"
      vm_size:                                                            "Standard_DS2_v2"


Plan: 4 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

Diff Detail

Repository
rSPRE sysadm-provisioning
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Mon, May 20, 6:31 PM
ardumont added inline comments.Mon, May 20, 6:35 PM
.gitignore
2

As far as i understood it, this is filled with needed plugins installed by terraform init.

azure/terraform/README.md
30

Maybe we should have a generic admin user with a key in the swh password-store and set that as default here.

azure/terraform/variables.tf
9

Here would be the use case for the generic admin user and its associated public key.
(Maybe we already have that somewhere, i don't remember).

lgtm

azure/terraform/vault.tf
70

s/servers/workers/ ?

78

What about a B2ms or B4ms? They are roughly the same price, but you get more "burst" CPU power, which is good for a worker

ardumont added inline comments.Tue, May 21, 10:34 AM
azure/terraform/vault.tf
70

It's for the vault's api and the objstorage's api, that's why i called it servers.

The cooking workers are already set up (and not part of this, but could be? <- outside of the diff's scope ;)

78

I did not get into that much details yet.
Thanks for reminding me. I kept the initial reference of the worker machine.

The use case here:

  • 2 apis (objstorage, cooking) running on the same machine.
  • discussion with a remote db (cache on prado)
  • discussion with a remote storage (running on azure)

Comparison:

|-----------+----------+-----------------+-------+------+-----------+----------+--------------+----------------|
| vm size   | offering | family          | vcpus | rams | data disk | max iops | temp storage | cost/month (€) |
|-----------+----------+-----------------+-------+------+-----------+----------+--------------+----------------|
| B2ms      | Standard | General purpose |     2 |    8 |         4 |     2400 | 16 GB        |          52.20 |
| B4ms      | Standard | General purpose |     4 |   16 |         8 |     3600 | 32 GB        |         104.15 |
| DS2_v2    | Standard | General purpose |     2 |    7 |         8 |     6400 | 14 GB        |          91.60 |
| D2s_v3    | Standard | General purpose |     2 |    8 |         4 |     3200 | 16 GB        |          60.23 |
|-----------+----------+-----------------+-------+------+-----------+----------+--------------+----------------|
| orangerie | X        | X               |     2 |    2 |         X |        ? |              |                |
|-----------+----------+-----------------+-------+------+-----------+----------+--------------+----------------|

The actual machine running the vault is orangerie.

ardumont updated this revision to Diff 4899.Tue, May 21, 10:50 AM

Use B2ms as default vm size machine

ardumont edited the test plan for this revision. (Show Details)Tue, May 21, 10:51 AM
ardumont updated this revision to Diff 4901.Tue, May 21, 11:44 AM

Rename vaultstorage blobstorage to swhvaultstorage

As otherwise, terraform complains it already exists.

ardumont updated this revision to Diff 4903.Tue, May 21, 12:16 PM

Add missing container for the new blob storage

douardda added inline comments.Tue, May 21, 12:29 PM
azure/terraform/README.md
2

I'd be delighted to have a 'what' part, i.e. an explanation of what this does, with a description of the deployed architecture, before entering the 'how' part.

ardumont added inline comments.Tue, May 21, 1:09 PM
azure/terraform/vault.tf
87

Count is not needed, so this will be simplified (other instances as well).

110

Thanks for olasd's heads up, it's not one museum, so i'll change to 'vangogh' instead.

ardumont updated this revision to Diff 4904.Tue, May 21, 1:11 PM

Simplify resources declaration

ardumont updated this revision to Diff 4919.Tue, May 21, 1:45 PM
  • azure/terraform/README.md: Describe what the vault.tf is about
This revision was not accepted when it landed; it landed in state Needs Review.Tue, May 21, 5:00 PM
This revision was automatically updated to reflect the committed changes.