Page MenuHomeSoftware Heritage

Move the raw imported data off uffizi/banco /srv/storage/space, which is getting full
Started, Work in Progress, HighPublic


(some) loaders store their raw imported data in a subdirectory of /srv/storage/space on uffizi.

Some initial imports have also been archived on that space, as well as on banco.

This allows us to replay loading of an origin with the exact original data if the conversion is somehow buggy (which happened a lot in the early days of the git loader).

This storage is not critical, and does not need to be available on a live partition of our main storage server. We should move it away to make space for more relevant live data.

Event Timeline

olasd triaged this task as High priority.Nov 20 2018, 12:04 PM
olasd created this task.
olasd renamed this task from Move the raw imported data off uffizi /srv/storage/space, which is getting full to Move the raw imported data off uffizi/banco /srv/storage/space, which is getting full.Jan 4 2019, 2:22 PM
olasd updated the task description. (Show Details)
olasd changed the task status from Open to Work in Progress.Jan 4 2019, 2:25 PM

The one-shot task of moving our raw data off uffizi and banco has been started:

  • Created a new azure storage account 'archiveeuwestswh', with "cold" storage semantics/billing
  • started moving the whole contents of uffizi:/srv/storage/space/mirrors/
  • started moving the tarred repositories present in banco:/srv/storage/space/mirrors/

All files from have been moved to the archive.

I'll start copying gitorious data.

not sure this is the right ticket for it, but I've made a copy of the annex on a new volume (on zfs) to free a bot more of /srv/storage/space