New test. tl; dr There is a way to update a repository migrated (drop it and import it
back, see 2nd run.)

I'm reseting the gitlab state again and doing a migration from scratch.

My tryout in T4334 made me reset an important file which now messes up everything.
I'm reseting the gitlab state again and doing a migration from scratch.

I'm gonna check again tomorrow (well friday since thursday we are at rocq for hardware installation).

Actions:

• connect through the gitlab ui
• Select the repository to drop (e.g. swh-ansible)
• Settings > General > Section "Advanced" > Expand > Bottom of the page click on "Delete"
• input the name of the repository to destroy and validate on "Yes, delete project"
• Cleanup eventually remains from the previous run on the host running the migration:

rm -rvf /var/tmp/migrate-gitlab/forgerie/{gitlab,phabricator}/swh-ansible

• Trigger back the migration:

A new one...

Got error running git command push with args ("gitlab" "--delete"
                                              "generated-differential-D1804-source") in dir /tmp/forgerie/gitlab/swh-jenkins-jobs

Thanks.

Looks like something is wrong in the operator state management.
For what I found on internet, it could be related to the cert-manager version but it should be already fixed. For example: https://gitlab.com/gitlab-org/cloud-native/gitlab-operator/-/issues/315
(The current cert-manager version in the cluster is 1.8.0)

After unstucking the migration again, he avatar upload routine is now compliant with the
last http client optimization change and works! I've rebased the commit again and pushed
in the swh branch [1].

/me grunts
another one bites the dust:

Unstuck after some mob debugging ;)
There is a fallback which tries and use the git am command and fails as some git config is missing.
We unstuck it by adding some .gitconfig in the dockerfile declaration.

The migration did a lot more work this time. It finally crashed on the git am command though [1] [2] [3].

\o/ well done

• Private repository after migration are private.

And it seems it finished without any issues this time [1] [2]

Changing the command args from [1] to [2], the run cli no longer complains! [3]

• Report migration status

Status update:

• Clean up gitlab state
• Checks
  • make sure host ssh key did not change after state reset (ssh-keyscan)
  • state folder permissions ok
  • access token key generated (and properly referenced in config.lisp)
• Trigger migration
• Report migration status

So i dug as to why my laptop was ok but not my desktop.

Better cli to execute...

forgerie@bespin:~$ logpath="/tmp/forgerie/run-$(date +%Y%m%d-%H%M).log"; echo "## Running migration logs in $logpath ##"; time /opt/forgerie/bin/run | tee $logpath
## Running migration logs in /tmp/forgerie/run-20220613-1410.log ##
...

Ah the migration scripts are still relying on the rails console to actually create merge requests... so we need that setup to work.
So i dug as to why my laptop was ok but not my desktop.

And now it fails on snippet creation which needs access to the rails console (in the current state of the migration script) [1]
I've got no proper setup which allows this to work now [2].
The only way i see is to comment that part for now... [3]

Migration failed again with a problem when creating a snippet [1]

The source of that issue was the ssh connection failing to be established [1]
Fixing with ssh-keygen [2]

Failed [1] (full logs [2])

After killing the minio data store, we've decided (as we had planned anyway) to start again from a clean slate.

• Upgrade gitlab operator

Backup done:

well, no, computer says no ¯\_(ツ)_/¯

fwiw, we're received notifications that the upstream repository have been delivered with some fixes.
So i've pulled the upstream branch and rebase the swh branch on it.

"Full" (with the overall sysadm repositories) run [1].

Another "private" (for the test) repository to try and migrate [1], it should be private but it's not [2]:

Added 2 more repositories [1] to migrate and triggered it [2] [3]:

It seems to have finished badly [1].
But that may only be because it's configured to run in debug mode.
Latest run logs for later [2]

Migration is now ongoing without failing thus far.
Configured to only import puppet-environment for now [1] (to fix papercuts).
But that's still migrating the rest (users, issues, etc...).

Summary sent to Frank Duncan and Karl Fogel.

Clean up gitlab instance

Gist of the actions are currently:

• Mirror the forgerie repository to add some docker commands to allow sandboxed execution (see diffs ^)
• Adaptations in the forgerie code source to allow migration runs with our current gitlab/phabricator instances (see diffs ^)
• Current runs are working a bit but do not succeed entirely