Page MenuHomeSoftware Heritage

Storage database migration tooling
Closed, MigratedEdits Locked

Description

Rationale

We currently use untested handwritten and maybe-run database migration scripts for the main storage database.

Being already a cumbersome solution for the current production setup, it will pupate a real nightmare once we (at last) have proper mirrors here and there.

We need to use better tooling for this, and define a proper migration strategy.

Note that this is also linked to The Journal Migration Problem, aka T1279

Objectives

How to:

  • test the migrations
  • run the migrations
  • avoid mismatches between python and sql
  • properly handle our different flavors of database (write db vs. read replica)

Tooling

Investigate what tooling can be helpful for this.

Revisions and Commits