Page MenuHomeSoftware Heritage

Test rollback of a repo migration
Closed, ResolvedPublic

Description

Check if it's possible to rollback the migration of a given repo (e.g. on a fallback) and re-import it

Event Timeline

bchauvet created this task.

Actions:

  • connect through the gitlab ui
  • Select the repository to drop (e.g. swh-ansible)
  • Settings > General > Section "Advanced" > Expand > Bottom of the page click on "Delete"
  • input the name of the repository to destroy and validate on "Yes, delete project"
  • Cleanup eventually remains from the previous run on the host running the migration:
rm -rvf /var/tmp/migrate-gitlab/forgerie/{gitlab,phabricator}/swh-ansible
  • Trigger back the migration:
docker container rm forgerie; docker run \
    -v $PWD/../forgerie:/opt/forgerie/ -v $PWD/docker/ssh:/srv/forgerie/.ssh \
    -v /srv/phabricator:/srv/phabricator \
    -v ~/.kube:/srv/forgerie/.kube \
    -v /var/tmp/migrate-gitlab/forgerie:/tmp/forgerie \
    --ulimit nofile=1024 --name forgerie --net=host -it forgerie
WARNING: Support for the legacy ~/.dockercfg configuration file and file-format is deprecated and will be removed in an upcoming release
forgerie
WARNING: Support for the legacy ~/.dockercfg configuration file and file-format is deprecated and will be removed in an upcoming release
## Running migration logs in /tmp/forgerie/run-20220622-1420.log ##
To load "forgerie":
  Load 1 ASDF system:
    forgerie
; Loading "forgerie"
..........
Failed to handle revision 3880, due to error Got error running git command cat-file with args ("-t"
                                                                                               "14b15b2f4f63efbb4deb7b06cea6cd598b22f8ef") in dir /srv/phabricator/repos/SPSITE/, skipping.
Shutting down....

real    5m18.740s
user    2m8.426s
sys     0m3.874s
  • Check against the ui the repository is back...

or not, i no longer see swh-ansible...

ardumont changed the task status from Open to Work in Progress.Wed, Jun 22, 4:53 PM
ardumont moved this task from Weekly backlog to in-progress on the System administration board.

I'm gonna check again tomorrow (well friday since thursday we are at rocq for hardware installation).

New test. tl; dr There is a way to update a repository migrated (drop it and import it
back, see 2nd run.)

Modus operandi

  • Main migration routine is done (nothing more to do [1]).
  • Drop the targetted repository [2] (same modus operandi as [3]).

First run: KO

    1. steps
  • same general m.o
  • Trigger migration [1]

Status

The repository is not detected as missing and is not imported.

Second run: OK

steps

  • Same general m.o
  • Backing up the mapping (kind of cache) file used by the migration.
  • Edit the mapping [6] file to drop the repository properties mentioned [4].
  • Trigger migration (same output [1]).

status

The repository is migrated! So providing the right amount of tampering with the
migration state, we can import back a repository.

Conclusion

Experiment 2nd run is the proper way to "update" a repository. Importing it from
scratch requires to drop it from gitlab (and the mapping file) then trigger back the
migration.

[1]

docker container rm forgerie; docker run \
    -v $PWD/../forgerie:/opt/forgerie/ -v $PWD/docker/ssh:/srv/forgerie/.ssh \
    -v /srv/phabricator:/srv/phabricator \
    -v ~/.kube:/srv/forgerie/.kube \
    -v /var/tmp/migrate-gitlab/forgerie:/tmp/forgerie \
    --ulimit nofile=1024 --name forgerie --net=host -it forgerie
WARNING: Support for the legacy ~/.dockercfg configuration file and file-format is deprecated and will be removed in an upcoming release
forgerie
WARNING: Support for the legacy ~/.dockercfg configuration file and file-format is deprecated and will be removed in an upcoming release
## Running migration logs in /tmp/forgerie/run-20220624-1237.log ##
To load "forgerie":
  Load 1 ASDF system:
    forgerie
; Loading "forgerie"
..........
Shutting down....

real    0m35.650s
user    0m5.750s
sys     0m0.343s

[2] https://gitlab-staging.swh.network/migrated/getsentry-onpremise

[3] T4334#87139

[4]

:PROJECT-ID NIL)#S(FORGERIE-GITLAB::MAPPED-ITEM                                      │
:TYPE :PROJECT                                                                       │
:ORIGINAL-ID "getsentry-onpremise"                                                   │
:ID 69                                                                               │
:IID NIL                                                                             │

[5]

$ git clone $repo
$ cd $repo
# writing existing repository's master branch to an anterior release
$ git reset --hard 21.12.0
# activating force push authorization in the gitlab ui
$ git push origin master --force

[6] /tmp/forgerie/gitlab/mapping

ardumont claimed this task.
ardumont moved this task from deployed/landed/monitoring to done on the System administration board.