Page MenuHomeSoftware Heritage
Create Task
Maniphest T4064

Test GitLab migration scripts
Started, Work in Progress, HighPublic
Actions

Description

Retrieve and test the migration scripts.

Plan:

  • Test migration
  • Ask for help by email if needed
  • Report status in [1]
  • Follow up to Frank Duncan and Karl Fogel by email
  • Waiting for feedback
  • T4064#86439: Trigger new migration run with fixes (crashes)
  • Status meeting
  • w/ olasd, ardumont: T4320: Upgrade gitlab
  • w/ olasd, ardumont: Clean up gitlab state
  • w/ olasd, ardumont: Trigger new migration run with fixes (running on bespin) for one repository
  • Analyse and troubleshoot and fix problems (latest one: T4064#86790)
  • Trigger back migration from scratch one repository
  • T4064#86791: Trigger back migration with another private repository
  • T4064#86792: Trigger with remaining sysadm repositories
  • Optimization (use http request pipelining)
  • T4064#87194: Successful run without human attendance
  • Analysis

[1] https://hedgedoc.softwareheritage.org/ezUtxkTDSJ-JBLARYrWIjA?view

Extra:

  • What to do with local patches to make the migration work? Propose upstream?

Revisions and Commits

R263 forgerie
ClosedD7856 docs: Fix typos and clean up whitespace
D7885R263:45dc0f03d962 docker: Install required ssh/config so migration can push to gitlab
D7866R263:cdecc3dc007c gitlab: Allow gitlab rails command to run without ssh
D7865R263:64e042052da0 Allow remote mysql connection
D7858R263:d269fbc44a13 Allow main script executions within container
D7857R263:ef75d7099967 Install Dockerfile to allow containerized script execution

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
bchauvet renamed this task from Gitlab migration tests to Test GitLab migration scripts.Mar 21 2022, 10:45 AM
ardumont changed the task status from Open to Work in Progress.May 19 2022, 4:55 PM
ardumont added a project: System administration.
ardumont moved this task from Backlog to in-progress on the System administration board.
ardumont added a revision: D7857: Install Dockerfile to allow containerized script execution.May 20 2022, 8:59 AM
ardumont added a revision: D7856: docs: Fix typos and clean up whitespace.
ardumont added a revision: D7858: Allow main script executions within container.
ardumont added a revision: D7865: Allow remote mysql connection.May 20 2022, 9:05 AM
ardumont added a revision: D7866: gitlab: Allow gitlab rail command to run without ssh.
ardumont updated the task description. (Show Details)May 20 2022, 12:15 PM
ardumont added a subscriber: ardumont.

Gist of the actions are currently:

  • Mirror the forgerie repository to add some docker commands to allow sandboxed execution (see diffs ^)
  • Adaptations in the forgerie code source to allow migration runs with our current gitlab/phabricator instances (see diffs ^)
  • Current runs are working a bit but do not succeed entirely

More information to come during the day.

ardumont added a comment.May 20 2022, 12:30 PM

Clean up gitlab instance

$ kubectx euwest-gitlab-staging
$ kubens gitlab-system
# delete gitlab instance
$ kubectl delete -f gitlab-staging.yaml
gitlab.apps.gitlab.com "gitlab" deleted
# Delete operator
$ VERSION=0.6.3
$ kubectl delete -f "https://gitlab.com/api/v4/projects/18899486/packages/generic/gitlab-operator/${VERSION}/gitlab-operator-kubernetes-${VERSION}.yaml"
# once the pods are no longer running
# delete pvc once the gitlab cluster is
$ kubectl get pvc | grep 'gitlab' | awk '{print $1}' | xargs -t -r kubectl delete pvc
kubectl delete pvc data-gitlab-postgresql-0 export-gitlab-minio-0 redis-data-gitlab-redis-master-0 repo-data-gitlab-gitaly-0
persistentvolumeclaim "data-gitlab-postgresql-0" deleted
persistentvolumeclaim "export-gitlab-minio-0" deleted
persistentvolumeclaim "redis-data-gitlab-redis-master-0" deleted
persistentvolumeclaim "repo-data-gitlab-gitaly-0" deleted

Start fresh gitlab instance

$ VERSION=0.6.3
$ kubectl apply -f "https://gitlab.com/api/v4/projects/18899486/packages/generic/gitlab-operator/${VERSION}/gitlab-operator-kubernetes-${VERSION}.yaml"
$ kubectl get pods | grep controller | grep -i running
# once controller is running
$ kubectl apply -f gitlab-staging.yaml
gitlab.apps.gitlab.com/gitlab created
# watch gitlab being installed
$ watch kubectl get pods
$ while true; do kubectl -n gitlab-system logs deployment/gitlab-controller-manager -c manager -f ; sleep 2; done

Configure access token

  • Retrieve the root credentials
$ kubectl get secret -n gitlab-system gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo
  • Connect to the root user through web view
  • User > Preferences > Access Tokens > Create access token (check all permissions)

Run migration

Prepare configuration

(setf forgerie-core:*working-directory* "/srv/forgerie/")

; insert your public key (and make sure the key has no passphrase and the container has
; access to it)
(setf forgerie-gitlab:*ssh-public-key* "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG7F8I7695YzFsZlBVk2145dHui04ZVxfbZb6G1yKepX tony@yavin4")

(setf forgerie-gitlab:*private-token* "<private-token>")
(setf forgerie-gitlab:*server-address* "https://gitlab-staging.swh.network/")

(setf forgerie-gitlab:*rails-command* "/usr/bin/kubectl")
(setf forgerie-gitlab:*rails-command-args* '("exec -ti -n gitlab-system deployment/gitlab-toolbox -- /srv/gitlab/bin/rails console"))

(setf forgerie-phabricator:*database-host* "127.0.0.1")
(setf forgerie-phabricator:*database-username* "root")
(setf forgerie-phabricator:*database-password* "<redacted>")
(setf forgerie-phabricator:*storage-location* "/srv/phabricator/storage/")
...
(setf forgerie-phabricator:*included-repositories*
 '("puppet-environment")); only 1 for now
  • Tunnel ssh over tate to expose phabricator's mysql port
ssh -L 3306:localhost:3306 tate

Run

$ cd forgerie
$ docker run \
    -v $SWH_ENVIRONMENT_HOME/../forgerie:/opt/forgerie/ \
    -v /srv/phabricator:/srv/phabricator \
    -v ~/.kube:/srv/forgerie/.kube \
    -v /var/tmp/migrate-gitlab/forgerie:/tmp/forgerie \
    -v ~/.ssh/:/srv/forgerie/.ssh \
    --name forgerie --net=host -it forgerie
    /opt/forgerie/bin/run | tee "/tmp/forgerie/run-$(date +%Y%m%d-%H%M).log"

Note: --net=host to expose host network interfaces to the container

postmortem

It breaks when trying to push on the 'gitlab' remote origin of the puppet-environment
repository [1].

The repository is cloned but the remote origin gitlab is not configured somehow.

forgerie@yavin4:/$ cd /tmp/forgerie/phabricator/puppet-environment/
forgerie@yavin4:/tmp/forgerie/phabricator/puppet-environment$ git remote -v
origin  /srv/phabricator/repos/SENV/ (fetch)
origin  /srv/phabricator/repos/SENV/ (push)

[1] https://forge.softwareheritage.org/source/forgerie/browse/main/src/main/gitlab/export.lisp$370

ardumont merged a task: T4146: Test gitlab migrations in the gitlab staging instance.May 23 2022, 2:26 PM
ardumont added a subscriber: vsellier.
ardumont added a comment.May 23 2022, 4:05 PM

Summary sent to Frank Duncan and Karl Fogel.

postmortem

It breaks when trying to push on the 'gitlab' remote origin of the puppet-environment
repository [1].

The repository is cloned but the remote origin gitlab is not configured somehow.

forgerie@yavin4:/$ cd /tmp/forgerie/phabricator/puppet-environment/
forgerie@yavin4:/tmp/forgerie/phabricator/puppet-environment$ git remote -v
origin  /srv/phabricator/repos/SENV/ (fetch)
origin  /srv/phabricator/repos/SENV/ (push)

[1] https://forge.softwareheritage.org/source/forgerie/browse/main/src/main/gitlab/export.lisp$370

Turns out i was reading the wrong repository there ^. So my analysis is wrong. The
repository that interests us is in /tmp/forgerie/gitlab/puppet-environment. And this one
is properly configured.

What's actually missing is the ~/.ssh/config that goes with the ssh key configured in
the config.lisp read by the migration process.

I'm gonna attend to reply to the discussion and open a diff for that (for the
dockerfile).

To make the migration process restart, drop the repository ^ and drop the repository in
gitlab as well. Then trigger back the migration (currently running).

ardumont added a revision: D7885: docker: Install required ssh/config so migration can push to gitlab.May 23 2022, 4:15 PM
ardumont added a comment.May 23 2022, 4:39 PM

Migration is now ongoing without failing thus far.
Configured to only import puppet-environment for now [1] (to fix papercuts).
But that's still migrating the rest (users, issues, etc...).

[1] https://gitlab-staging.swh.network/software-heritage/puppet-environment

ardumont added a comment.May 23 2022, 5:12 PM

It seems to have finished badly [1].
But that may only be because it's configured to run in debug mode.
Latest run logs for later [2]

[2]

run-20220523-1356.tar.gz1 MBDownload

[1]

...
12: ((LAMBDA NIL :IN FORGERIE-CORE:RUN))
13: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
14: ((FLET "WITHOUT-INTERRUPTS-BODY-11" :IN SB-THREAD::RUN))
15: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
16: ((FLET "WITHOUT-INTERRUPTS-BODY-4" :IN SB-THREAD::RUN))
17: (SB-THREAD::RUN)
18: ("foreign function: call_into_lisp")
19: ("foreign function: funcall1")

unhandled condition in --disable-debugger mode, quitting
Shutting down....
WARNING:
   Problem running exit hook #<FUNCTION (LAMBDA ()
                                          :IN
                                          FORGERIE-CORE:RUN) {1001A30F9B}>:
  In thread #<THREAD RUNNING {1001878103}>, attempt to join the current thread.
Shutting down....
ardumont added a comment.May 24 2022, 2:58 PM

Added 2 more repositories [1] to migrate and triggered it [2] [3]:

[1] config.lisp:

(setf forgerie-core:*debug* nil)
...
(setf forgerie-phabricator:*included-repositories*
 '("puppet-swh-site"
   "swh-sysadmin-provisioning"
   "puppet-environment"
))

[2]

$ time /opt/forgerie/bin/run | tee "/tmp/forgerie/run-$(date +%Y%m%d-%H%M).log"
...

real    98m27.570s
user    36m34.774s
sys     49m26.570s
$ time /opt/forgerie/bin/postmortem | tee "/tmp/forgerie/postmortem-run-$(date +%Y%m%d-%H%M).log"
...

real    0m2.016s
user    0m0.792s
sys     0m0.153s

[3]

run-20220524-0832.tar.gz2 KBDownload

ardumont added a comment.May 24 2022, 3:05 PM

Another "private" (for the test) repository to try and migrate [1], it should be private but it's not [2]:

(setf forgerie-phabricator:*included-repositories*
 '(;; "swh-ansible"
   "training-mob-programming"  ;; private repository (well staff tagged)
   "puppet-swh-site"
   "swh-sysadmin-provisioning"
   "puppet-environment"
))

[1]

run-20220524-1259.tar.gz2 KBDownload

[2] https://gitlab-staging.swh.network/software-heritage/training-mob-programming

ardumont added a comment.EditedMay 25 2022, 8:36 AM

"Full" (with the overall sysadm repositories) run [1].

Results can be seen in the staging gitlab instance [2]

This migration run [2] imported the remaining repositories, it took:

real    414m4.028s
user    170m59.024s
sys     235m9.664s

[1]

run-20220524-1502.tar.gz3 KBDownload

[2] https://gitlab-staging.swh.network/

[3] The first import (see above) imported the initial issues. It's possible this run
also imported incrementally new issues.

ardumont updated the task description. (Show Details)May 25 2022, 2:09 PM
ardumont moved this task from in-progress to deployed/landed/monitoring on the System administration board.
ardumont moved this task from deployed/landed/monitoring to code-review/await-feedback/pause on the System administration board.
ardumont added a commit: R263:ef75d7099967: Install Dockerfile to allow containerized script execution.May 31 2022, 2:53 PM
ardumont added a commit: R263:d269fbc44a13: Allow main script executions within container.
ardumont added a commit: R263:64e042052da0: Allow remote mysql connection.
ardumont added a commit: R263:cdecc3dc007c: gitlab: Allow gitlab rails command to run without ssh.
ardumont added a commit: R263:45dc0f03d962: docker: Install required ssh/config so migration can push to gitlab.
ardumont added a comment.Jun 7 2022, 4:32 PM

fwiw, we're received notifications that the upstream repository have been delivered with some fixes.
So i've pulled the upstream branch and rebase the swh branch on it.

Then triggered a run. We'll see what happens.

ardumont updated the task description. (Show Details)Jun 7 2022, 4:33 PM
ardumont added a comment.Jun 7 2022, 4:54 PM

well, no, computer says no ¯\_(ツ)_/¯

Unhandled FORGERIE-GITLAB::HTTP-ERROR in thread #<SB-THREAD:THREAD RUNNING
                                                   {1005560003}>:
  Http error code: 400, resp: (MESSAGE
                               (NAME (has already been taken) PATH
                                (has already been taken)))
Backtrace for: #<SB-THREAD:THREAD RUNNING {1005560003}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<FORGERIE-GITLAB::HTTP-ERROR {100430CEF3}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<FORGERIE-GITLAB::HTTP-ERROR {100430CEF3}>)
2: (INVOKE-DEBUGGER #<FORGERIE-GITLAB::HTTP-ERROR {100430CEF3}>)
3: (ERROR #<FORGERIE-GITLAB::HTTP-ERROR {100430CEF3}>)
4: (FORGERIE-GITLAB::MAKE-REQUEST "projects" :POST (("namespace_id" . "4") ("name" . "puppet-puppet-letsencrypt") ("path" . "puppet-puppet-letsencrypt") ("tag_list" . "language-puppet,system_administrators,sysadm,sync_to_github") ("issues_access_level" . #1="enabled") ("visibility" . "public") ("merge_requests_access_level" . #1#)) :SUDO NIL)
5: (FORGERIE-GITLAB::CREATE-PROJECT #S(FORGERIE-CORE::VC-REPOSITORY :NAME "puppet-puppet-letsencrypt" :SLUG "puppet-puppet-letsencrypt" :PRIMARY-PROJECTS NIL :GIT-LOCATION "/srv/phabricator/repos/192/" :PROJECTS (#S(FORGERIE-CORE::PROJECT :NAME "Language-Puppet" :TAGS ("language-puppet")) #S(FORGERIE-CORE::PROJECT :NAME "System administrators" :TAGS ("system_administrators" "sysadm")) #S(FORGERIE-CORE::PROJECT :NAME "Sync to GitHub" :TAGS ("sync_to_github"))) :PRIVATE NIL :COMMITS (#S(FORGERIE-CORE:COMMIT :SHA "61ffd8dee7c0f10f7ea7e84f23c4d732b7cd669b" :PARSED-COMMENT ("Merge pull request #265 from nod0n/unsupported_os
...
ardumont updated the task description. (Show Details)Jun 9 2022, 2:37 PM
ardumont updated the task description. (Show Details)
ardumont updated the task description. (Show Details)Jun 9 2022, 5:19 PM
ardumont changed the status of subtask T4320: Upgrade gitlab instance to v15 from Open to Work in Progress.Jun 10 2022, 10:06 AM
ardumont mentioned this in R263:7d55c183835d: Install Dockerfile to allow containerized script execution.Jun 10 2022, 5:01 PM
ardumont mentioned this in R263:89ea6bfd9f2f: Allow main script executions within container.
ardumont mentioned this in R263:59ff8d8126a7: Allow remote mysql connection.
ardumont mentioned this in R263:cde188463674: gitlab: Allow gitlab rails command to run without ssh.
ardumont mentioned this in R263:fd27626b4c22: docker: Install required ssh/config so migration can push to gitlab.
ardumont updated the task description. (Show Details)Jun 10 2022, 7:02 PM
ardumont added a comment.Jun 13 2022, 10:02 AM

Failed [1] (full logs [2])

[1]

Backtrace for: #<SB-THREAD:THREAD RUNNING {1001890103}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<SIMPLE-ERROR "Got error running git command ~A with args ~S in dir ~A" {10034946C3}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<SIMPLE-ERROR "Got error running git command ~A with args ~S in dir ~A" {10034946C3}>)
2: (INVOKE-DEBUGGER #<SIMPLE-ERROR "Got error running git command ~A with args ~S in dir ~A" {10034946C3}>)
3: (ERROR "Got error running git command ~A with args ~S in dir ~A" "push" ("gitlab" "--all") "/tmp/forgerie/gitlab/swh-sysadmin-provisioning")
4: (FORGERIE-CORE:GIT-CMD "/tmp/forgerie/gitlab/swh-sysadmin-provisioning" "push" ("gitlab" "--all") :ERROR T :DEBUG NIL :INPUT NIL)
5: (FORGERIE-GITLAB::CREATE-PROJECT #S(FORGERIE-CORE::VC-REPOSITORY :NAME "sysadm-provisioning" :SLUG "swh-sysadmin-provisioning" :PRIMARY-PROJECTS NIL :GIT-LOCATION "/srv/phabricator/repos/SPRE/" :PROJECTS (#S(FORGERIE-CORE::PROJECT :NAME
 "System administration" :TAGS ("system_administration")) #S(FORGERIE-CORE::PROJECT :NAME "System administrators" :TAGS ("system_administrators" "sysadm"))) :PRIVATE NIL :COMMITS (#S(FORGERIE-CORE:COMMIT :SHA "7b90b55041ccb14ebb15d8ed59c42
d2bb64ebfd3" :PARSED-COMMENT ("Align worker17 hardware spec to worker18
...
6: ((:METHOD FORGERIE-CORE:EXPORT-FORGE ((EQL :GITLAB) T)) #<unused argument> (:USERS (#S(FORGERIE-CORE::USER :USERNAME "administrator" :NAME "Software Heritage Administrator" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "nicolas+swhforgeadmi
n_dandrimont.eu@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR NIL) #S(FORGERIE-CORE::USER :USERNAME "olasd" :NAME "Nicolas Dandrimont" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "nicolas.dandrimont_gmail.com@example.org" :IS-PRIMARY NIL) #S
(FORGERIE-CORE::EMAIL :ADDRESS "nicolas_dandrimont.eu@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 2966263 :NAME #1="profile" :SIZE 12136 :LOCATION "/tmp/forgerie/phabricator/files/463/PHID-FILE-sepmsswq26g5dmbn
c463" :MIMETYPE #2="image/jpeg")) #S(FORGERIE-CORE::USER :USERNAME "ardumont" :NAME "Antoine R. Dumont" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "ardumont_softwareheritage.org@example.org" :IS-PRIMARY NIL) #S(FORGERIE-CORE::EMAIL :ADDRESS
 "antoine.romain.dumont_gmail.com@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 3 :NAME #3="profile" :SIZE 11009 :LOCATION "/tmp/forgerie/phabricator/files/wbp/PHID-FILE-nma6v42ey5ihn5i2uwbp" :MIMETYPE #4="image/
png")) #S(FORGERIE-CORE::USER :USERNAME "zack" :NAME "Stefano Zacchiroli" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "zack_upsilon.cc@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME #5="profile" :SIZE 1
6531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE #6="image/jpeg")) #S(FORGERIE-CORE::USER :USERNAME "rdicosmo" :NAME "Roberto Di Cosmo" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "roberto_dicosmo
.org@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 2875599 :NAME "profile" :SIZE 196883 :LOCATION "/tmp/forgerie/phabricator/files/uff/PHID-FILE-jck7nk5ue2ftdwq3cuff" :MIMETYPE "image/png")) #S(FORGERIE-CORE::USE
R :USERNAME "grouss" :NAME "Guillaume ROUSSEAU" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "guillaume_rousseaubottin.fr@example.org" :IS-PRIMARY NIL) #S(FORGERIE-CORE::EMAIL :ADDRESS "guillaume.rousseau_univ-paris-diderot.fr@example.org" :I
S-PRIMARY T)) :ADMIN NIL :AVATAR NIL) #S(FORGERIE-CORE::USER :USERNAME "swhphab" :NAME "SWH Phabricator Bot" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "nicolas+swhbot_dandrimont.eu@example.org" :IS-PRIMARY T)) :ADMIN NIL :AVATAR NIL) #S(FO
RGERIE-CORE::USER :USERNAME "swh-sysadm-deploy" :NAME "Sysadmin Deploy User" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "nicolas+swh-sysadm-deploy_dandrimont.eu@example.org" :IS-PRIMARY T)) :ADMIN NIL :AVATAR NIL) #S(FORGERIE-CORE::USER :US
ERNAME "swh-public-deploy" :NAME "Public Deploy User" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "zack+swhbot_upsilon.cc@example.org" :IS-PRIMARY T)) :ADMIN NIL :AVATAR NIL) #S(FORGERIE-CORE::USER :USERNAME "test-public" :NAME "Test User, w
ith no special permissions" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "zack+swhtest_upsilon.cc@example.org" :IS-PRIMARY T)) :ADMIN NIL :AVATAR NIL) #S(FORGERIE-CORE::USER :USERNAME "steff" :NAME "Laurent Steff" :EMAILS (#S(FORGERIE-CORE::E
MAIL :ADDRESS "laurent.steff_inria.fr@example.org" :IS-PRIMARY T)) :ADMIN NIL :AVATAR NIL) #S(FORGERIE-CORE::USER :USERNAME "dimitri" :NAME "Dimitri Fontaine" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "dimitri_2ndQuadrant.fr@example.org" :
IS-PRIMARY T)) :ADMIN NIL :AVATAR NIL) ...) :PROJECTS (#S(FORGERIE-CORE::PROJECT :NAME "Staff" :TAGS ("members" "staff")) #S(FORGERIE-CORE::PROJECT :NAME "Developers" :TAGS ("developers" "devel")) #S(FORGERIE-CORE::PROJECT :NAME "Language-
Python" :TAGS ("python" "language-python")) #S(FORGERIE-CORE::PROJECT :NAME "Language-Puppet" :TAGS ("language-puppet")) #S(FORGERIE-CORE::PROJECT :NAME "Origin-GitHub" :TAGS ("github" "gh" "origin-github")) #S(FORGERIE-CORE::PROJECT :NAME
 "Origin-CGit" :TAGS ("cgit" "origin-cgit")) #S(FORGERIE-CORE::PROJECT :NAME "System administrators" :TAGS ("system_administrators" "sysadm")) #S(FORGERIE-CORE::PROJECT :NAME "Format-Git" :TAGS ("git" "format-git")) #S(FORGERIE-CORE::PROJE
CT :NAME "Origin-Debian" :TAGS ("debian" "origin-debian")) #S(FORGERIE-CORE::PROJECT :NAME "Managers" :TAGS ("managers" "mgmt")) #S(FORGERIE-CORE::PROJECT :NAME "Paper authors" :TAGS ("paper_authors" "paper")) #S(FORGERIE-CORE::PROJECT :NA
ME "Format-Deb" :TAGS ("format-deb" "deb")) ...) :VC-REPOSITORIES (#S(FORGERIE-CORE::VC-REPOSITORY :NAME "sysadm-provisioning" :SLUG "swh-sysadmin-provisioning" :PRIMARY-PROJECTS NIL :GIT-LOCATION "/srv/phabricator/repos/SPRE/" :PROJECTS (
#S(FORGERIE-CORE::PROJECT :NAME #7="System administration" :TAGS #) #S(FORGERIE-CORE::PROJECT :NAME "System administrators" :TAGS #)) :PRIVATE NIL :COMMITS (#S(FORGERIE-CORE:COMMIT :SHA "7b90b55041ccb14ebb15d8ed59c42d2bb64ebfd3" :PARSED-CO
MMENT #) #S(FORGERIE-CORE:COMMIT :SHA "d7d5ff6b4b64cf76ed8738fd171edeec54abc4e8" :PARSED-COMMENT #) #S(FORGERIE-CORE:COMMIT :SHA "6e7b13218cbcdbd7b8fd02f3e327f6924511ef67" :PARSED-COMMENT #) #S(FORGERIE-CORE:COMMIT :SHA "179087d6fee28ad2b7
a47ff50ef02720bc707250" :PARSED-COMMENT #) #S(FORGERIE-CORE:COMMIT :SHA "9a4d7636d7958b1c2793f06ebbbc0bf982c53b1d" :PARSED-COMMENT #) #S(FORGERIE-CORE:COMMIT :SHA "51f802c63f79b64e2f6d294f1b53f9bdd08de6c6" :PARSED-COMMENT #) #S(FORGERIE-CO
RE:COMMIT :SHA "d62700b98e98fa4c8f4caee6fdc2ea4f3ceb98d2" :PARSED-COMMENT #) #S(FORGERIE-CORE:COMMIT :SHA "0ff550d0f8892144630fa4e06d03224d36a6aa75" :PARSED-COMMENT #) #S(FORGERIE-CORE:COMMIT :SHA "529a2d63d2363a9a63454ab122b31dc85d16e8ef"
 :PARSED-COMMENT #) #S(FORGERIE-CORE:COMMIT :SHA "d43e6bb05d85e218dcbf8c3fbf158ee633b815da" :PARSED-COMMENT #) #S(FORGERIE-CORE:COMMIT :SHA "062ec99d21010120892f9c6cb53c1ef2748154bf" :PARSED-COMMENT #) #S(FORGERIE-CORE:COMMIT :SHA "1697be7
f5e9ed16c7290678ef4611b56e5f60ea3" :PARSED-COMMENT #) ...))) :SNIPPETS (#S(FORGERIE-CORE:SNIPPET :ID 1 :TITLE "dockscript.sh" :FILES (#S(FORGERIE-CORE::FILE :ID 5 :NAME "dockscript.sh" :SIZE 621 :LOCATION "/tmp/forgerie/phabricator/files/z
zv/PHID-FILE-ijvbukainvcve35ujzzv" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME #8="olasd" :NAME #9="Nicolas Dandrimont" :EMAILS (# #) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 2966263 :NAME #1# :SIZE
12136 :LOCATION "/tmp/forgerie/phabricator/files/463/PHID-FILE-sepmsswq26g5dmbnc463" :MIMETYPE #2#)) :NOTES (#S(FORGERIE-CORE::NOTE :ID "P1" :TEXT # :AUTHOR # :DATE 3650776468)) :DATE 3650776401) #S(FORGERIE-CORE:SNIPPET :ID 2 :TITLE NIL :
FILES (#S(FORGERIE-CORE::FILE :ID 6 :NAME NIL :SIZE 1242 :LOCATION "/tmp/forgerie/phabricator/files/cn6/PHID-FILE-x7e5i3yxuj3qkc6dncn6" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME #10="zack" :NAME #11="
Stefano Zacchiroli" :EMAILS (#) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME #5# :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE #6#)) :NOTES NIL :DATE 3650861920) #S(FORGERI
E-CORE:SNIPPET :ID 3 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 7 :NAME NIL :SIZE 965 :LOCATION "/tmp/forgerie/phabricator/files/7gk/PHID-FILE-2wnlhh67hd72mbamq7gk" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER
:USERNAME #10# :NAME #11# :EMAILS (#) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME #5# :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE #6#)) :NOTES NIL :DATE 3650865542) #S(F
ORGERIE-CORE:SNIPPET :ID 4 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 8 :NAME NIL :SIZE 2219 :LOCATION "/tmp/forgerie/phabricator/files/3o2/PHID-FILE-qil2rcesnq2gku3kl3o2" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE
::USER :USERNAME #8# :NAME #9# :EMAILS (# #) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 2966263 :NAME #1# :SIZE 12136 :LOCATION "/tmp/forgerie/phabricator/files/463/PHID-FILE-sepmsswq26g5dmbnc463" :MIMETYPE #2#)) :NOTES NIL :DATE 36508668
15) #S(FORGERIE-CORE:SNIPPET :ID 5 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 10 :NAME NIL :SIZE 445 :LOCATION "/tmp/forgerie/phabricator/files/kvb/PHID-FILE-7xrvc3nxazqsflfwqkvb" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGE
RIE-CORE::USER :USERNAME #10# :NAME #11# :EMAILS (#) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME #5# :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE #6#)) :NOTES NIL :DATE 3
650872498) #S(FORGERIE-CORE:SNIPPET :ID 6 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 12 :NAME NIL :SIZE 655 :LOCATION "/tmp/forgerie/phabricator/files/cf5/PHID-FILE-dir54l6pgfrwouj37cf5" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #
S(FORGERIE-CORE::USER :USERNAME #10# :NAME #11# :EMAILS (#) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME #5# :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE #6#)) :NOTES NIL
:DATE 3650907685) #S(FORGERIE-CORE:SNIPPET :ID 7 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 13 :NAME NIL :SIZE 868 :LOCATION "/tmp/forgerie/phabricator/files/5c6/PHID-FILE-i37jrihwpb2fwxp755c6" :MIMETYPE "text/plain; charset=utf-8")) :A
UTHOR #S(FORGERIE-CORE::USER :USERNAME #10# :NAME #11# :EMAILS (#) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME #5# :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE #6#)) :NOT
ES NIL :DATE 3651219510) #S(FORGERIE-CORE:SNIPPET :ID 8 :TITLE "gzip-sha1-files.py" :FILES (#S(FORGERIE-CORE::FILE :ID 15 :NAME NIL :SIZE 1627 :LOCATION "/tmp/forgerie/phabricator/files/hs2/PHID-FILE-nwhmaiq6cltt26mhkhs2" :MIMETYPE "text/p
lain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME #10# :NAME #11# :EMAILS (#) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME #5# :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7
tya" :MIMETYPE #6#)) :NOTES NIL :DATE 3651298031) #S(FORGERIE-CORE:SNIPPET :ID 9 :TITLE "directory entry / list duplication on linux.git" :FILES (#S(FORGERIE-CORE::FILE :ID 17 :NAME "directory entry / list duplication on linux.git" :SIZE 5
08 :LOCATION "/tmp/forgerie/phabricator/files/qhq/PHID-FILE-dvz5gxtxjky57possqhq" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME #8# :NAME #9# :EMAILS (# #) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 2966
263 :NAME #1# :SIZE 12136 :LOCATION "/tmp/forgerie/phabricator/files/463/PHID-FILE-sepmsswq26g5dmbnc463" :MIMETYPE #2#)) :NOTES NIL :DATE 3651308475) #S(FORGERIE-CORE:SNIPPET :ID 10 :TITLE "First import of linux.git" :FILES (#S(FORGERIE-CO
RE::FILE :ID 18 :NAME "First import of linux.git" :SIZE 3180 :LOCATION "/tmp/forgerie/phabricator/files/fwm/PHID-FILE-tz5f5kxsgplyubxbvfwm" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME #8# :NAME #9# :EMA
ILS (# #) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 2966263 :NAME #1# :SIZE 12136 :LOCATION "/tmp/forgerie/phabricator/files/463/PHID-FILE-sepmsswq26g5dmbnc463" :MIMETYPE #2#)) :NOTES NIL :DATE 3651309746) #S(FORGERIE-CORE:SNIPPET :ID 11
 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 19 :NAME NIL :SIZE 373 :LOCATION "/tmp/forgerie/phabricator/files/4a3/PHID-FILE-fnx3t6vhnglhkhbao4a3" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME #10# :NAM
E #11# :EMAILS (#) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME #5# :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE #6#)) :NOTES NIL :DATE 3651386530) #S(FORGERIE-CORE:SNIPPE
T :ID 12 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 21 :NAME NIL :SIZE 692 :LOCATION "/tmp/forgerie/phabricator/files/lwo/PHID-FILE-tduzyqgor7d26ki3ylwo" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME #
10# :NAME #11# :EMAILS (#) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME #5# :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE #6#)) :NOTES NIL :DATE 3651583116) ...) :MERGE-REQ
UESTS (#S(FORGERIE-CORE:MERGE-REQUEST :ID 1495 :VC-REPOSITORY #S(FORGERIE-CORE::VC-REPOSITORY :NAME #12="sysadm-provisioning" :SLUG #13="swh-sysadmin-provisioning" :PRIMARY-PROJECTS NIL :GIT-LOCATION #14="/srv/phabricator/repos/SPRE/" :PRO
JECTS (# #) :PRIVATE NIL :COMMITS NIL) :TITLE "azure/terraform: Add azure vault vm using terraform" :DESCRIPTION ("Related " (:TICKET "1716" "T1716") "
" (:H2 "Test Plan" "
== Test Plan ==
") #<(SIMPLE-ARRAY CHARACTER (9381))

...
7: ((LAMBDA NIL :IN FORGERIE-CORE:RUN))
8: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
9: ((FLET "WITHOUT-INTERRUPTS-BODY-11" :IN SB-THREAD::RUN))
10: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
11: ((FLET "WITHOUT-INTERRUPTS-BODY-4" :IN SB-THREAD::RUN))
12: (SB-THREAD::RUN)
13: ("foreign function: call_into_lisp")
14: ("foreign function: funcall1")

unhandled condition in --disable-debugger mode, quitting
Shutting down....
WARNING:
   Problem running exit hook #<FUNCTION (LAMBDA ()
                                          :IN
                                          FORGERIE-CORE:RUN) {10087F5D6B}>:
  In thread #<THREAD RUNNING {1001890103}>, attempt to join the current thread.
Shutting down....

real    14m54.306s
user    3m54.792s
sys     4m11.098s

[2]

run-20220610-1654.tar.gz1 KBDownload

ardumont added a comment.EditedJun 13 2022, 10:35 AM

The source of that issue was the ssh connection failing to be established [1]
Fixing with ssh-keygen [2]

Triggering back the migration fails [3] as the repository now exists.
So cleaning it up (in gitlab ui and in forgerie container [4]) then triggering back the migration.

[1]

forgerie@bespin:~$ cat ~/.ssh/config
Host gitlab-staging.swh.network
  User git
  IdentitiesOnly yes
  IdentityFile ~/.ssh/id-migration-gitlab
forgerie@bespin:~$ ssh git@gitlab-staging.swh.network
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:ysZeeO6dabyhp6pTNq07TWWzI8K/+urH//zNcSG5wDY.
Please contact your system administrator.
Add correct host key in /srv/forgerie/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /srv/forgerie/.ssh/known_hosts:1
  remove with:
  ssh-keygen -f "/srv/forgerie/.ssh/known_hosts" -R "gitlab-staging.swh.network"
ECDSA host key for gitlab-staging.swh.network has changed and you have requested strict checking.
Host key verification failed.

[2]

forgerie@bespin:~$ ssh-keygen -f "/srv/forgerie/.ssh/known_hosts" -R "gitlab-staging.swh.network"
# Host gitlab-staging.swh.network found: line 1
/srv/forgerie/.ssh/known_hosts updated.
Original contents retained as /srv/forgerie/.ssh/known_hosts.old
forgerie@bespin:~$ ssh git@gitlab-staging.swh.network
The authenticity of host 'gitlab-staging.swh.network (20.103.140.206)' can't be established.
ECDSA key fingerprint is SHA256:ysZeeO6dabyhp6pTNq07TWWzI8K/+urH//zNcSG5wDY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes                                                                                                                                                                       Warning: Permanently added 'gitlab-staging.swh.network' (ECDSA) to the list of known hosts.                                                                                                                                                    Warning: the ECDSA host key for 'gitlab-staging.swh.network' differs from the key for the IP address '20.103.140.206'                                                                                                                          Offending key for IP in /srv/forgerie/.ssh/known_hosts:1                                                                                                                                                                                       Are you sure you want to continue connecting (yes/no)? ^C                                                                                                                                                                                      forgerie@bespin:~$ ssh-keygen -f "/srv/forgerie/.ssh/known_hosts" -R "20.103.140.206"                                                                                                                                                          # Host 20.103.140.206 found: line 1                                                                                                                                                                                                            /srv/forgerie/.ssh/known_hosts updated.                                                                                                                                                                                                        Original contents retained as /srv/forgerie/.ssh/known_hosts.old                                                                                                                                                                               forgerie@bespin:~$ ssh git@gitlab-staging.swh.network                                                                                                                                                                                          Warning: Permanently added the ECDSA host key for IP address '20.103.140.206' to the list of known hosts.                                                                                                                                      PTY allocation request failed on channel 0                                                                                                                                                                                                     Welcome to GitLab, @root!                                                                                                                                                                                                                      Connection to gitlab-staging.swh.network closed.

[3]

forgerie@bespin:~$ time /opt/forgerie/bin/run | tee "/tmp/forgerie/run-$(date +%Y%m%d-%H%M).log"                                                                                                                                               To load "forgerie":                                                                                                                                                                                                                              Load 1 ASDF system:                                                                                                                                                                                                                              forgerie                                                                                                                                                                                                                                   ; Loading "forgerie"                                                                                                                                                                                                                           .........                                                                                                                                                                                                                                      Unhandled FORGERIE-GITLAB::HTTP-ERROR in thread #<SB-THREAD:THREAD RUNNING                                                                                                                                                                                                                        {10066FC653}>:                                                                                                                                                                                Http error code: 400, resp: (MESSAGE                                                                                                                                                                                                                                        (NAME (has already been taken) PATH                                                                                                                                                                                                             (has already been taken)))                                                                                                                                                                                                                                                                                                                                                                                                                                    Backtrace for: #<SB-THREAD:THREAD RUNNING {10066FC653}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<FORGERIE-GITLAB::HTTP-ERROR {1008E76D13}> #<unused argument> :QUIT T)                                                                                                                                   1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<FORGERIE-GITLAB::HTTP-ERROR {1008E76D13}>)                                                                                                                                                     2: (INVOKE-DEBUGGER #<FORGERIE-GITLAB::HTTP-ERROR {1008E76D13}>)
3: (ERROR #<FORGERIE-GITLAB::HTTP-ERROR {1008E76D13}>)
4: (FORGERIE-GITLAB::MAKE-REQUEST "projects" :POST (("namespace_id" . "4") ("name" . "sysadm-provisioning") ("path" . "swh-sysadmin-provisioning") ("tag_list" . "system_administration,system_administrators,sysadm") ("issues_access_level" .
 #1="enabled") ("visibility" . "public") ("merge_requests_access_level" . #1#)) :SUDO NIL)
5: (FORGERIE-GITLAB::CREATE-PROJECT #S(FORGERIE-CORE::VC-REPOSITORY :NAME "sysadm-provisioning" :SLUG "swh-sysadmin-provisioning" :PRIMARY-PROJECTS NIL :GIT-LOCATION "/srv/phabricator/repos/SPRE/" :PROJECTS (#S(FORGERIE-CORE::PROJECT :NAME
 "System administration" :TAGS ("system_administration")) #S(FORGERIE-CORE::PROJECT :NAME "System administrators" :TAGS ("system_administrators" "sysadm"))) :PRIVATE NIL :COMMITS (#S(FORGERIE-CORE:COMMIT :SHA "7b90b55041ccb14ebb15d8ed59c42
d2bb64ebfd3" :PARSED-COMMENT ("Align worker17 hardware spec to worker18

[4]

forgerie@bespin:~$ rm -rf /tmp/forgerie/phabricator/swh-sysadmin-provisioning/ /tmp/forgerie/gitlab/swh-sysadmin-provisioning/
ardumont added a comment.EditedJun 13 2022, 1:10 PM

Migration failed again with a problem when creating a snippet [1]

[2]

run-20220610-1654.tar.gz1 KBDownload

[1]

forgerie@bespin:~$ time /opt/forgerie/bin/run | tee "/tmp/forgerie/run-$(date +%Y%m%d-%H%M).log"                                                                                                                                     [371/2681]
To load "forgerie":
  Load 1 ASDF system:
    forgerie
; Loading "forgerie"
.........
^O1Unhandled FORGERIE-GITLAB::HTTP-ERROR in thread #<SB-THREAD:THREAD RUNNING
                                                   {1001888103}>:
  Http error code: 400, resp: (MESSAGE
                               (ERROR
                                Repository Error creating the snippet - Invalid file name))

Backtrace for: #<SB-THREAD:THREAD RUNNING {1001888103}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<FORGERIE-GITLAB::HTTP-ERROR {1013CE6D93}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<FORGERIE-GITLAB::HTTP-ERROR {1013CE6D93}>)
2: (INVOKE-DEBUGGER #<FORGERIE-GITLAB::HTTP-ERROR {1013CE6D93}>)
3: (ERROR #<FORGERIE-GITLAB::HTTP-ERROR {1013CE6D93}>)
4: (FORGERIE-GITLAB::MAKE-REQUEST "/projects/2/snippets" :POST (("title" . "Des projets après Inria ? // What’s about after Inria ?") ("content" . #<(SIMPLE-ARRAY CHARACTER (2118))


-------- Forwarded Message --------
Subject: Des projets après Inria ? // What’s about after Inria ?
Date: Mon, 24 Jan 2022 17:35:30 +0100 (CET)
From: Gregoire Maurice <gregoire.maurice@inria.fr... {1013C7C24F}>) ("visibility" . "public") ("file_name" . "Des projets après Inria ? // What’s about after Inria ?")) :SUDO NIL)
5: (FORGERIE-GITLAB::CREATE-SNIPPET #S(FORGERIE-CORE:SNIPPET :ID 1262 :TITLE "Des projets après Inria ? // What’s about after Inria ?" :FILES (#S(FORGERIE-CORE::FILE :ID 5332516 :NAME "Des projets après Inria ? // What’s about after Inria
?" :SIZE 2118 :LOCATION "/tmp/forgerie/phabricator/files/zhl/PHID-FILE-4thcaeg7bkeizqb4szhl" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME "vlorentz" :NAME "Valentin Lorentz" :EMAILS (#S(FORGERIE-CORE::EM
AIL :ADDRESS "vlorentz_softwareheritage.org@example.org" :IS-PRIMARY T)) :ADMIN NIL :AVATAR #S(FORGERIE-CORE::FILE :ID 3315393 :NAME "profile" :SIZE 24576 :LOCATION "/tmp/forgerie/phabricator/files/efx/PHID-FILE-6yg2nqica7dzwao3yefx" :MIME
TYPE "image/jpeg")) :NOTES NIL :DATE 3852033412))
6: ((FLET FORGERIE-GITLAB::MAP-WITH-NOTE-MAPPING-CATCH :IN FORGERIE-CORE:EXPORT-FORGE) #<FUNCTION FORGERIE-GITLAB::CREATE-SNIPPET> (#S(FORGERIE-CORE:SNIPPET :ID 1 :TITLE "dockscript.sh" :FILES (#S(FORGERIE-CORE::FILE :ID 5 :NAME "dockscrip
t.sh" :SIZE 621 :LOCATION "/tmp/forgerie/phabricator/files/zzv/PHID-FILE-ijvbukainvcve35ujzzv" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME "olasd" :NAME "Nicolas Dandrimont" :EMAILS (#S(FORGERIE-CORE::E
MAIL :ADDRESS "nicolas.dandrimont_gmail.com@example.org" :IS-PRIMARY NIL) #S(FORGERIE-CORE::EMAIL :ADDRESS "nicolas_dandrimont.eu@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 2966263 :NAME "profile" :SIZE 12136
:LOCATION "/tmp/forgerie/phabricator/files/463/PHID-FILE-sepmsswq26g5dmbnc463" :MIMETYPE "image/jpeg")) :NOTES (#S(FORGERIE-CORE::NOTE :ID "P1" :TEXT (#<(SIMPLE-ARRAY CHARACTER (262)) To be used with a udev config snippet such as:
ACTION=="add", ENV{DEVTYPE}=="usb_device", ENV{ID_MODEL}=="USB5534B", RUN+="/usr/local/bin/dock add"
ACTION=="remove", ENV{DEVTYPE}=="usb_device", EN... {100344DF8F}>) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME "olasd" :NAME "Nicolas Dandrimont" :EMAILS # :ADMIN T :AVATAR #) :DATE 3650776468)) :DATE 3650776401) #S(FORGERIE-CORE:SNIPPET :ID
2 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 6 :NAME NIL :SIZE 1242 :LOCATION "/tmp/forgerie/phabricator/files/cn6/PHID-FILE-x7e5i3yxuj3qkc6dncn6" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME "zack" :
NAME "Stefano Zacchiroli" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "zack_upsilon.cc@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME "profile" :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya
/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE "image/jpeg")) :NOTES NIL :DATE 3650861920) #S(FORGERIE-CORE:SNIPPET :ID 3 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 7 :NAME NIL :SIZE 965 :LOCATION "/tmp/forgerie/phabricator/files/7gk/PHID-F
ILE-2wnlhh67hd72mbamq7gk" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME "zack" :NAME "Stefano Zacchiroli" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "zack_upsilon.cc@example.org" :IS-PRIMARY T)) :ADMIN T :
AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME "profile" :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7tya" :MIMETYPE "image/jpeg")) :NOTES NIL :DATE 3650865542) #S(FORGERIE-CORE:SNIPPET :ID 4 :TITLE
 NIL :FILES (#S(FORGERIE-CORE::FILE :ID 8 :NAME NIL :SIZE 2219 :LOCATION "/tmp/forgerie/phabricator/files/3o2/PHID-FILE-qil2rcesnq2gku3kl3o2" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME "olasd" :NAME "N
icolas Dandrimont" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "nicolas.dandrimont_gmail.com@example.org" :IS-PRIMARY NIL) #S(FORGERIE-CORE::EMAIL :ADDRESS "nicolas_dandrimont.eu@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR #S(FORGERIE-CORE
::FILE :ID 2966263 :NAME "profile" :SIZE 12136 :LOCATION "/tmp/forgerie/phabricator/files/463/PHID-FILE-sepmsswq26g5dmbnc463" :MIMETYPE "image/jpeg")) :NOTES NIL :DATE 3650866815) #S(FORGERIE-CORE:SNIPPET :ID 5 :TITLE NIL :FILES (#S(FORGER
IE-CORE::FILE :ID 10 :NAME NIL :SIZE 445 :LOCATION "/tmp/forgerie/phabricator/files/kvb/PHID-FILE-7xrvc3nxazqsflfwqkvb" :MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME "zack" :NAME "Stefano Zacchiroli" :EMA
ILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "zack_upsilon.cc@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 203038 :NAME "profile" :SIZE 16531 :LOCATION "/tmp/forgerie/phabricator/files/tya/PHID-FILE-vc2fhnioi2wwib4o7ty
a" :MIMETYPE "image/jpeg")) :NOTES NIL :DATE 3650872498) #S(FORGERIE-CORE:SNIPPET :ID 6 :TITLE NIL :FILES (#S(FORGERIE-CORE::FILE :ID 12 :NAME NIL :SIZE 655 :LOCATION "/tmp/forgerie/phabricator/files/cf5/PHID-FILE-dir54l6pgfrwouj37cf5" :MI
ardumont moved this task from code-review/await-feedback/pause to in-progress on the System administration board.Jun 13 2022, 1:11 PM
ardumont added a comment.EditedJun 13 2022, 3:47 PM

And now it fails on snippet creation which needs access to the rails console (in the current state of the migration script) [1]
I've got no proper setup which allows this to work now [2].
The only way i see is to comment that part for now... [3]

[2] P1381

[1]

forgerie@bespin:~$ time /opt/forgerie/bin/run | tee "/tmp/forgerie/run-$(date +%Y%m%d-%H%M).log"
To load "forgerie":
  Load 1 ASDF system:
    forgerie
; Loading "forgerie"
.........
Unhandled END-OF-FILE in thread #<SB-THREAD:THREAD RUNNING {1004B30003}>:
  end of file on #<SB-SYS:FD-STREAM for "descriptor 8" {100C75ECE3}>

Backtrace for: #<SB-THREAD:THREAD RUNNING {1004B30003}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<END-OF-FILE {100C75F363}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<END-OF-FILE {100C75F363}>)
2: (INVOKE-DEBUGGER #<END-OF-FILE {100C75F363}>)
3: (ERROR END-OF-FILE :STREAM #<SB-SYS:FD-STREAM for "descriptor 8" {100C75ECE3}>)
4: (READ-LINE #<SB-SYS:FD-STREAM for "descriptor 8" {100C75ECE3}> T NIL #<unused argument>)
5: (FORGERIE-GITLAB::RAILS-COMMAND "s = Snippet.find(1)")
6: (FORGERIE-GITLAB::CREATE-SNIPPET #S(FORGERIE-CORE:SNIPPET :ID 1 :TITLE "dockscript.sh" :FILES (#S(FORGERIE-CORE::FILE :ID 5 :NAME "dockscript.sh" :SIZE 621 :LOCATION "/tmp/forgerie/phabricator/files/zzv/PHID-FILE-ijvbukainvcve35ujzzv" :
MIMETYPE "text/plain; charset=utf-8")) :AUTHOR #S(FORGERIE-CORE::USER :USERNAME "olasd" :NAME "Nicolas Dandrimont" :EMAILS (#S(FORGERIE-CORE::EMAIL :ADDRESS "nicolas.dandrimont_gmail.com@example.org" :IS-PRIMARY NIL) #S(FORGERIE-CORE::EMAI
L :ADDRESS "nicolas_dandrimont.eu@example.org" :IS-PRIMARY T)) :ADMIN T :AVATAR #S(FORGERIE-CORE::FILE :ID 2966263 :NAME "profile" :SIZE 12136 :LOCATION "/tmp/forgerie/phabricator/files/463/PHID-FILE-sepmsswq26g5dmbnc463" :MIMETYPE "image/
jpeg")) :NOTES (#S(FORGERIE-CORE::NOTE :ID "P1" :TEXT (#<(SIMPLE-ARRAY CHARACTER (262)) To be used with a udev config snippet such as:

[3]

forgerie@bespin:~$ grep -r "create-snippet" /opt/forgerie/
/opt/forgerie/src/main/gitlab/export.lisp:     ;; (map-with-note-mapping-catch #'create-snippet (getf data :snippets))
/opt/forgerie/src/main/gitlab/export.lisp:(defun create-snippet (snippet)
ardumont added a comment.Jun 13 2022, 4:07 PM

Ah the migration scripts are still relying on the rails console to actually create merge requests... so we need that setup to work.
So i dug as to why my laptop was ok but not my desktop.

And i turns out i had an extra dns entry in yavin4...

forgerie@bespin:~$ grep euwest /etc/hosts
192.168.200.13 euwest-gitlab-staging-89229db0.dd8c7b66-1c78-4e28-9fe3-6f454883861d.privatelink.westeurope.azmk8s.io
forgerie@bespin:~$ kubectl exec -ti -n gitlab-system deployment/gitlab-toolbox -- /srv/gitlab/bin/rails console
--------------------------------------------------------------------------------
 Ruby:         ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
 GitLab:       15.0.0 (8a186dedfc1) FOSS
 GitLab Shell: 14.3.0
 PostgreSQL:   12.7
------------------------------------------------------------[ booted in 19.68s ]
Loading production environment (Rails 6.1.4.7)
irb(main):001:0> Snippet.find(1)
=> #<ProjectSnippet id:1 migrated/Migration$1>

So maybe that will unstuck it all.

ardumont added a comment.Jun 13 2022, 4:14 PM

Better cli to execute...

forgerie@bespin:~$ logpath="/tmp/forgerie/run-$(date +%Y%m%d-%H%M).log"; echo "## Running migration logs in $logpath ##"; time /opt/forgerie/bin/run | tee $logpath
## Running migration logs in /tmp/forgerie/run-20220613-1410.log ##
...
ardumont added a comment.Jun 13 2022, 5:10 PM

So i dug as to why my laptop was ok but not my desktop.

@vsellier reminded me of the following that i had executed a while back in the laptop (yavin4) indeed [1].
Instructions include the /etc/hosts modification (so that explain why it unstuck stuff a bit).
(As for the ~/.kube/config, it's already pushed on bespin).

[1] https://hedgedoc.softwareheritage.org/ynL9z-6JRnGYhW8vrLDl-w#Azure-Kubernetes-cluster-provisionning

ardumont added a comment.EditedJun 15 2022, 10:42 AM

Status update:

  • Clean up gitlab state
  • Checks
    • make sure host ssh key did not change after state reset (ssh-keyscan)
    • state folder permissions ok
    • access token key generated (and properly referenced in config.lisp)
  • Trigger migration
  • Report migration status
ardumont added a comment.EditedJun 15 2022, 11:23 AM
  • Report migration status

Fail [1] on the rails command execution again.
Actually troubleshooting...
and, after activating the debug option [2], finally something actionable [3]

It's not happy about our rails-command-args setup... [4]

[2]

(setf forgerie-core:*debug* t)

[3]

...
Booting: Error: unknown command "exec -ti -n gitlab-system deployment/gitlab-toolbox -- /srv/gitlab/bin/rails console" for "kubectl"
Booting: Run 'kubectl --help' for usage.
Unhandled END-OF-FILE in thread #<SB-THREAD:THREAD RUNNING {1004B30003}>:
  end of file on #<SB-SYS:FD-STREAM for "descriptor 8" {10034B89B3}>
...

[4]

(setf forgerie-gitlab:*rails-command* "/usr/bin/kubectl")
(setf forgerie-gitlab:*rails-command-args* '("exec -ti -n gitlab-system deployment/gitlab-toolbox -- /srv/gitlab/bin/rails console"))

[1]

run-20220615-0855.tar.gz1 KBDownload

ardumont added a comment.EditedJun 15 2022, 1:56 PM

Changing the command args from [1] to [2], the run cli no longer complains! [3]

[3]

Booting: Unable to use a TTY - input is not a terminal or the right kind of file
Booting: --------------------------------------------------------------------------------
Booting:  Ruby:         ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
Booting:  GitLab:       15.0.0 (8a186dedfc1) FOSS
Booting:  GitLab Shell: 14.3.0
Booting:  PostgreSQL:   12.7
Booting: ------------------------------------------------------------[ booted in 16.11s ]

[1]

(setf forgerie-gitlab:*rails-command-args* '("exec -ti -n gitlab-system deployment/gitlab-toolbox -- /srv/gitlab/bin/rails console"))

[2]

(setf forgerie-gitlab:*rails-command-args* '("exec" "-ti" "-n" "gitlab-system" "deployment/gitlab-toolbox" "--" "/srv/gitlab/bin/rails" "console"))
ardumont added a comment.Jun 15 2022, 2:25 PM

And it seems it finished without any issues this time [1] [2]

[1]

Shutting down....

real    14m3.534s
user    0m35.738s
sys     0m25.314s

[2] full logs

run-20220615-1154.tar.gz625 KBDownload

ardumont added a comment.Jun 15 2022, 2:38 PM

Note:

  • Private repository after migration are private.

I've triggered the remaining sysadm repositories to migrate.

ardumont updated the task description. (Show Details)Jun 15 2022, 2:39 PM
ardumont updated the task description. (Show Details)
ardumont updated the task description. (Show Details)
ardumont updated the task description. (Show Details)
ardumont moved this task from in-progress to deployed/landed/monitoring on the System administration board.Jun 15 2022, 4:07 PM
ardumont mentioned this in R263:3bf442577900: docker: Install required ssh/config so migration can push to gitlab.Jun 15 2022, 4:12 PM
vsellier added a comment.Jun 15 2022, 6:41 PM

\o/ well done

ardumont added a comment.EditedJun 16 2022, 10:15 AM

The migration did a lot more work this time. It finally crashed on the git am command though [1] [2] [3].

[1] man git am says: 'git-am - Apply a series of patches from a mailbox'

$ grep "//working.patch" archives/run-20220615-1235-all-sysadm-repositories.log | head -1
  Got error running git command am with args ("/tmp/forgerie/gitlab//working.patch") in dir /tmp/forgerie/gitlab/puppet-swh-site

[2] with a tiny bit more context

Unhandled SIMPLE-ERROR in thread #<SB-THREAD:THREAD RUNNING {10068A0793}>:
  Got error running git command am with args ("/tmp/forgerie/gitlab//working.patch") in dir /tmp/forgerie/gitlab/puppet-swh-site

Backtrace for: #<SB-THREAD:THREAD RUNNING {10068A0793}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<SIMPLE-ERROR "Got error running git command ~A with args ~S in dir ~A" {1017080693}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<SIMPLE-ERROR "Got error running git command ~A with args ~S in dir ~A" {1017080693}>)
2: (INVOKE-DEBUGGER #<SIMPLE-ERROR "Got error running git command ~A with args ~S in dir ~A" {1017080693}>)
3: (ERROR "Got error running git command ~A with args ~S in dir ~A" "am" ("/tmp/forgerie/gitlab//working.patch") "/tmp/forgerie/gitlab/puppet-swh-site")
4: (FORGERIE-CORE:GIT-CMD "/tmp/forgerie/gitlab/puppet-swh-site" "am" ("/tmp/forgerie/gitlab//working.patch") :ERROR T :DEBUG NIL :INPUT NIL)
5: (FORGERIE-GITLAB::CREATE-MERGE-REQUEST #S(FORGERIE-CORE:MERGE-REQUEST :ID 6686 :VC-REPOSITORY #S(FORGERIE-CORE::VC-REPOSITORY :NAME "puppet-swh-site" :SLUG "puppet-swh-site" :PRIMARY-PROJECTS (#S(FORGERIE-CORE::PROJECT :NAME "Puppet rec
ipes" :TAGS ("puppet_recipes" "puppet"))) :GIT-LOCATION "/srv/phabricator/repos/SPSITE/" :PROJECTS (#S(FORGERIE-CORE::PROJECT :NAME "Language-Puppet" :TAGS ("language-puppet")) #S(FORGERIE-CORE::PROJECT :NAME "System administration" :TAGS
("system_administration")) #S(FORGERIE-CORE::PROJECT :NAME "Sync to GitHub" :TAGS ("sync_to_github"))) :PRIVATE NIL :COMMITS NIL) :TITLE "hedgedoc: Drop unnecessary and breaking dependencies" :DESCRIPTION ("With the following package decla
ration, puppet fails to apply. They are unecessary as
they are not installed (or pulled as dependencies) on the node without impacting the
runtime.
...
real    484m15.340s
user    194m42.485s
sys     255m37.446s

[3] full logs

run-20220615-1235-all-sysadm-repositories.tar.gz292 KBDownload

ardumont mentioned this in R263:6f7374634393: docker: Install required ssh/config so migration can push to gitlab.Jun 17 2022, 3:59 PM
ardumont mentioned this in R263:4f7b3469f506: Configure .gitconfig so git command do not break.Jun 17 2022, 4:12 PM
ardumont added a comment.EditedJun 17 2022, 4:13 PM

Unstuck after some mob debugging ;)
There is a fallback which tries and use the git am command and fails as some git config is missing.
We unstuck it by adding some .gitconfig in the dockerfile declaration.

But note that the applied commit is messing up the original author in the commit so it's not perfect. It has the advantage to no longer failing the migration so fine.

ardumont mentioned this in R263:4965146eff61: Dockerfile: Add missing file dependency.Jun 17 2022, 6:30 PM
ardumont mentioned this in R263:e8522c134d9b: docker/README: Explicit the ulimit usage.
ardumont updated the task description. (Show Details)Jun 20 2022, 3:35 PM
ardumont mentioned this in R263:64bf79bdb31b: docker: Install required ssh/config so migration can push to gitlab.Jun 20 2022, 5:11 PM
ardumont mentioned this in R263:c74b705f0788: Configure .gitconfig so git command do not break.
ardumont mentioned this in R263:6100b9d4effd: Dockerfile: Add missing file dependency.
ardumont mentioned this in R263:6f9e565e16cc: docker/README: Explicit the ulimit usage.
ardumont added a comment.Jun 20 2022, 10:49 PM

/me grunts
another one bites the dust:

                                                   {1001888103}>:
  Http error code: 400, resp: (MESSAGE
                               (AVATAR
                                (file format is not supported. Please try one of the following supported formats: png, jpg, jpeg, gif, bmp, tiff, ico, webp)))

Backtrace for: #<SB-THREAD:THREAD RUNNING {1001888103}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<FORGERIE-GITLAB::HTTP-ERROR {10064C1263}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<FORGERIE-GITLAB::HTTP-ERROR {10064C1263}>)
2: (INVOKE-DEBUGGER #<FORGERIE-GITLAB::HTTP-ERROR {10064C1263}>)
3: (ERROR #<FORGERIE-GITLAB::HTTP-ERROR {10064C1263}>)
4: (FORGERIE-GITLAB::MAKE-REQUEST "users" :POST (("name" . "Nicolas Dandrimont") ("email" . "nicolas_dandrimont.eu@example.org") ("admin" . #1="true") ("reset_password" . #1#) ("username" . "olasd") ("avatar" . #P"/tmp/forgerie/phabricator
/files/463/PHID-FILE-sepmsswq26g5dmbnc463")) :SUDO NIL)
ardumont mentioned this in R263:b046be9f3808: docker: Install required ssh/config so migration can push to gitlab.Jun 22 2022, 9:34 AM
ardumont mentioned this in R263:d8f53d796303: Configure .gitconfig so git command do not break.
ardumont mentioned this in R263:2ca08c19a348: Dockerfile: Add missing file dependency.
ardumont mentioned this in R263:b49110b1e9a2: docker/README: Explicit the ulimit usage.
ardumont added a comment.EditedJun 22 2022, 9:54 AM

After unstucking the migration again, he avatar upload routine is now compliant with the
last http client optimization change and works! I've rebased the commit again and pushed
in the swh branch [1].

The migration was done and it was time to trigger it back from scratch to have a
non-failure and timed scenario ok (to have a rough estimation of the time it can be left
unattended and doing the job).

I'm now stuck in reseting the migration state which is now blocked for unknown reason
(for now). I've summarized that blocking point in T4347.

[1] At some point, i'll have to attend to proposing upstream all those commits (as merge
requests) to ease reproductibility migration runs.

ardumont changed the status of subtask T4347: gitlab migration reset state routine is flaky from Open to Work in Progress.Jun 22 2022, 12:52 PM
ardumont closed subtask T4320: Upgrade gitlab instance to v15 as Resolved.
ardumont closed subtask T4347: gitlab migration reset state routine is flaky as Resolved.Jun 22 2022, 1:59 PM
ardumont added a comment.Jun 22 2022, 3:07 PM

A new one...

Got error running git command push with args ("gitlab" "--delete"
                                              "generated-differential-D1804-source") in dir /tmp/forgerie/gitlab/swh-jenkins-jobs

Troubleshooting it:

forgerie@bespin:/tmp/forgerie/gitlab/swh-jenkins-jobs$ git push gitlab --delete generated-differential-D1804-source
Warning: Permanently added 'gitlab-staging.swh.network,20.103.140.206' (ECDSA) to the list of known hosts.
error: unable to delete 'generated-differential-D1804-source': remote ref does not exist
error: failed to push some refs to 'gitlab-staging.swh.network:migrated/swh-jenkins-jobs.git'
ardumont changed the status of subtask T4334: Test rollback of a repo migration from Open to Work in Progress.Jun 22 2022, 4:53 PM
ardumont moved this task from deployed/landed/monitoring to in-progress on the System administration board.
ardumont added a comment.Jun 22 2022, 5:13 PM

My tryout in T4334 made me reset an important file which now messes up everything.
I'm reseting the gitlab state again and doing a migration from scratch.

The last one roughly took 200min (in multiple steps though) so it's fast enough [1]

[1]

real    39m24.705s
real    122m27.491s
real    40m42.343s

(+ 40 39 122) ; 201 min
ardumont added a comment.Jun 24 2022, 10:26 AM

I'm reseting the gitlab state again and doing a migration from scratch.

That migration actually run without issues and without being human managed [1].

[1]

run-20220622-1511.log32 KBDownload

ardumont updated the task description. (Show Details)Jun 24 2022, 11:05 AM
ardumont closed subtask T4334: Test rollback of a repo migration as Resolved.Jun 24 2022, 3:11 PM
ardumont moved this task from in-progress to deployed/landed/monitoring on the System administration board.Jun 24 2022, 4:15 PM
ardumont reopened subtask T4334: Test rollback of a repo migration as Open.Jul 4 2022, 4:00 PM
ardumont mentioned this in T4383: Test full phabricator migration to gitlab.Jul 6 2022, 2:42 PM
About · Developer information · Legal