Page MenuHomeSoftware Heritage
Create Task
Maniphest T1371

Move the raw imported data off uffizi/banco /srv/storage/space, which is getting full
Closed, MigratedEdits Locked
Actions

Description

(some) loaders store their raw imported data in a subdirectory of /srv/storage/space on uffizi.

Some initial imports have also been archived on that space, as well as on banco.

This allows us to replay loading of an origin with the exact original data if the conversion is somehow buggy (which happened a lot in the early days of the git loader).

This storage is not critical, and does not need to be available on a live partition of our main storage server. We should move it away to make space for more relevant live data.

Related Objects
Search...

Event Timeline

olasd triaged this task as High priority.Nov 20 2018, 12:04 PM
olasd created this task.
olasd closed subtask T1381: Survey of cold storage options as Resolved.Nov 23 2018, 3:47 PM
olasd changed the status of subtask T1382: Survey of the data stored in uffizi:/srv/storage/space and banco:/srv/storage/space. from Open to Work in Progress.Nov 23 2018, 3:50 PM
olasd renamed this task from Move the raw imported data off uffizi /srv/storage/space, which is getting full to Move the raw imported data off uffizi/banco /srv/storage/space, which is getting full.Jan 4 2019, 2:22 PM
olasd updated the task description. (Show Details)
olasd changed the task status from Open to Work in Progress.Jan 4 2019, 2:25 PM

The one-shot task of moving our raw data off uffizi and banco has been started:

  • Created a new azure storage account 'archiveeuwestswh', with "cold" storage semantics/billing
  • started moving the whole contents of uffizi:/srv/storage/space/mirrors/code.google.com
  • started moving the tarred repositories present in banco:/srv/storage/space/mirrors/github.com
olasd added a comment.Jan 10 2019, 1:37 PM

All files from code.google.com have been moved to the archive.

I'll start copying gitorious data.

douardda added a subscriber: douardda.Jul 22 2019, 10:49 AM

not sure this is the right ticket for it, but I've made a copy of the annex on a new volume (on zfs) to free a bot more of /srv/storage/space

ardumont added a subscriber: ardumont.Jul 22 2019, 11:13 AM

Related T1925, i think

gitlab-migration closed this task as Migrated.Oct 19 2022, 5:54 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

gitlab-migration changed the status of subtask T1381: Survey of cold storage options from Resolved to Migrated.Oct 19 2022, 5:54 PM
gitlab-migration closed subtask T1382: Survey of the data stored in uffizi:/srv/storage/space and banco:/srv/storage/space. as Migrated.
gitlab-migration closed subtask T1387: Keep track of exported original artifacts as Migrated.
About · Developer information · Legal