as per title: we want to have a first, complete, off-site mirror of our object storage, hosted on Azure
|Resolved||zack||T239 preserve at least 2 copies of each content object|
|Resolved||olasd||T691 complete object storage mirror on Azure (meta task)|
|Unknown Object (Maniphest Task)|
We now have a "full" content mirror on azure (the data for each 16th bucket is up to date as of the time the snapshot was taken).
We still need to process the logs for the full injection.
- The main archive currently synchronously writes all contents to Azure as well as the local storage (the gap is strictly closing)
- all partitions from uffizi have been copied to azure and mass-injected (except for partition 8 which only got partially mass injected)
- after this process, it looks like azure is missing 10% of all objects (excluding partition 8), which are all on banco
- I've started a procedure to copy the missing objects from banco directly. Estimated time to completion ~ 1 month
- The same procedure has been started to copy the missing objects from partition 8 on uffizi. Estimated time to completion ~ 15 days
After processing the logs of the backfilling process to make sure to redo all the ranges that were interrupted in various database migrations, I'm now confident that this task is complete: we have a full mirror of all contents on Azure, which is kept up to date by the main archive storage backend writing synchronously to it.