Page MenuHomeSoftware Heritage
Create Task
Maniphest T3819

Deploy swh.model 4.1.0 / swh.storage 0.41.0 to production
Closed, MigratedEdits Locked
Actions

Description

Deploy the latest version of swh.model and swh.storage to production.

  • make sure all new packages are available (swh.model, swh.storage, swh.deposit)
  • run the first half of the database migration, adding the new fields to both databases (somerset then belvedere)
  • check that postgresql replication still runs properly
  • stop all workers
  • upgrade and restart gunicorn-swh-storage instances
    • saam
    • moma
    • webapp1
    • storage01.euwest.azure
  • upgrade and restart worker08
  • check that new revisions and releases are getting the new offset fields populated
  • restart all other workers
  • run the data migration for the new fields on old objects
  • add database constraints for all objects once the migration has run

Related Objects
Search...

StatusAssignedTask
 Migratedgitlab-migrationT3096 Efficient and reliable download via the Vault
 Migratedgitlab-migrationT887 Vault: "snapshot" cooker
 Migratedgitlab-migrationT3504 Make the git-bare cooker publicly available
 Migratedgitlab-migrationT3505 Make the git-bare cooker available to the staff and beta-testers in the production webapp
 Migratedgitlab-migrationT3506 Get rid of the concept of vault "object_type"
 Migratedgitlab-migrationT3507 prod: vault: Deploy v1.0.0
 Migratedgitlab-migrationT3503 staging: vault: Deploy v1.0.0
 Migratedgitlab-migrationT843 Vault: Add a "git bare" tarball cooker
 Migratedgitlab-migrationT3412 Deploy vault "git bare" tarball cooker (swh-vault v0.6)
 Migratedgitlab-migrationT3518 Enable vault cookers to access swh-graph
 Migratedgitlab-migrationT3543 Debian package python3-swh.graph.client
 Migratedgitlab-migrationT3565 Add an option in swh-web to allow git-bare cooking via the UI for beta-testers
 Migratedgitlab-migrationT3551 Fix git-fsck errors in the git-bare cooker
 Migratedgitlab-migrationT3552 Fix corrupted releases, revisions, and directories in the storage
 Migratedgitlab-migrationT3566 git-fsck errors in the git-bare cooker are not properly reported
 Migratedgitlab-migrationT3731 Try parallelizing directory_get_entries with an async swh-storage client
 Migratedgitlab-migrationT3924 Write mailmaps after cooking git-bare archives with display names?
 Migratedgitlab-migrationT3135 Improve integrity of ingested content
 Migratedgitlab-migrationT3594 Faithfully store weird git objects
 Migratedgitlab-migrationT3752 Store/represent time offsets as strings
 Migratedgitlab-migrationT3819 Deploy swh.model 4.1.0 / swh.storage 0.41.0 to production

Event Timeline

olasd changed the task status from Open to Work in Progress.Dec 23 2021, 11:45 AM
olasd triaged this task as High priority.
olasd created this task.
olasd updated the task description. (Show Details)Dec 23 2021, 12:48 PM

I've started stopping most workers, except for a high priority loader on worker01.

olasd updated the task description. (Show Details)Dec 23 2021, 12:52 PM
olasd updated the task description. (Show Details)Dec 23 2021, 1:06 PM
olasd updated the task description. (Show Details)Dec 23 2021, 1:25 PM
olasd added a comment.Dec 23 2021, 1:37 PM

All systems upgraded to the new schema.

I've started the data migration process with queries similar to what was written in T3818#76247.

olasd added a comment.Dec 28 2021, 10:53 AM

The data migration for the revision table was taking forever (eta was a couple of months); I restarted it with no filtering on ids, monitoring the growth in logical replication spill files...

olasd added a comment.Dec 30 2021, 5:13 PM

Well, that wasn't going to work either, so I've now re-launched the migration using id ranges, with 16 parallel queries...

• Jokerboy added a subscriber: 01shobitha.Jan 2 2022, 5:14 AM
olasd removed a subscriber: 01shobitha.Jan 7 2022, 1:27 PM

Around half the revision rows have now been migrated to bytes offsets. The migration is still ongoing.

olasd updated the task description. (Show Details)Jan 18 2022, 12:26 PM
olasd added a parent task: T3752: Store/represent time offsets as strings.
olasd added a comment.Jan 18 2022, 12:28 PM

All revisions (supposedly) have been migrated to bytes offsets. I'll wait for the ongoing base backup and vacuum to complete before adding the constraints on the production database.

olasd added a comment.Jan 19 2022, 7:11 PM
softwareheritage=#   alter table revision
    add constraint revision_date_offset_not_null
    check (date is null or date_offset_bytes is not null) not valid,
    add constraint revision_committer_date_offset_not_null
    check (committer_date is null or committer_date_offset_bytes is not null) not valid;
ALTER TABLE

softwareheritage=# alter table revision validate constraint revision_date_offset_not_null, validate constraint revision_committer_date_offset_not_null;
ALTER TABLE

All done.

olasd closed this task as Resolved.Jan 19 2022, 7:12 PM
olasd updated the task description. (Show Details)
gitlab-migration changed the task status from Resolved to Migrated.Oct 19 2022, 6:05 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal