Except for cooking tasks (running on azure), the remaining services runs on the unibo infra (node: orangerie):
As the db is not that big, it can be moved to prado's secondary cluster.
Other services can move to a dedicated azure vm.
For the objstorage, the following should be enough:
ardumont@orangerie:~% sudo df -h /srv/softwareheritage/vault_cache/ Filesystem Size Used Avail Use% Mounted on 10.100.0.1:/Q/shares/swh/orangerie 59T 5.5T 54T 10% /srv/softwareheritage
ncdu 1.12 ~ Use the arrow keys to navigate, press ? for help
--- /var/lib/postgresql -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1.1 GiB [##########] /11
...Considering the size of that database, and the fact that we don't have any provisions to automatically spin up a new database server, I think it would make more sense to repatriate it on our main postgres setup, rather than movig it to a new machine on azure.
To be clear, that would mean moving the database on the secondary postgres cluster on prado.
To be clear, that would mean moving the database on the secondary postgres cluster on prado.
Ok.
Created the db on prado on secondary cluster (i normalized the user name from swhvault to swh-vault, same for the db name):
$ createuser -p 5434 -P swh-vault $ createdb -p 5434 --lc-ctype=C.UTF-8 -T template1 swh-vault -O swh-vault $ cat swh-vault.dump.sql | psql -p 5434 swh-vault
note:
Updating the reference to orangerie's api to target the new db instance on prado
Related af9cd6965d7f01f2d20d735b5e812398bc130c38
Note: orangeriedev got its db updated to swh-vault as well.
So, as in D1495:
$ terraform apply $ cd sysadm-provisioning $ scp azure/provision-vm.sh ardumont@<ip>:/tmp/provision-vm.sh $ ssh ardumont@<ip> > chmod +x /tmp/provision-vm.sh > sudo /tmp/provision-vm.sh private $ ssh louvre > sudo clush -b -w ns0.euwest.azure -w pergamon puppet agent --test
Now still heading towards swh-site to make the vault configuration converge according to the new resources created.
Heads up, checking the new vault's objstorage speaking to azure blob storage is fine (credential and all):
So i'm now ready to actually migrate the webapp's setup regarding vault from orangerie to vangogh.
Webapp/cookers migrated to use the azure vault instance.
As per oral/irc discussion with @moranegg @douardda, the existing cooked data can be migrated next week [1]
[1] irc excerpt:
14:29 <douardda> moranegg[m]: do we know if we can start the (new) vault without cooked archives? 14:36 <moranegg[m]> douardda: no response. but as we said face to face- go go go!
Great!
As per oral/irc discussion with @moranegg @douardda, the existing cooked data can be migrated next week [1]
I'd say don't bother migrating old archives. Not migrating them will give us an opportunity to reassess the actual vault usage starting form a clean slate, and also reduce the storage cost.