For information, the last injection has been done. The remaining errors:

(but we should have a list of those repos, for posterity).

These should be rescheduled and driven to successful completion.

In T674#13455, @ardumont wrote:

• we send something that was not a git repository.
• integrity error (which is expected for now)

After much learning on how to read and extract logs from our kibana instance, here is the error repartition.

As of now, ingestion, after multiple (re)schedulings, has been done.

Update on this.

Visit dates have been fixed for the origins already injected.

Command to trigger the messages (from worker01):

cat /srv/storage/space/mirrors/gitorious.org/full_mapping.txt | SWH_WORKER_INSTANCE=swh_loader_git_disk ./load_gitorious.py --root-repositories /srv/storage/space/mirrors/gitorious.org/mnt/repositories

(The script defaults to use the right queue 'swh_loader_git_express' and the right origin-date 'Wed, 30 Mar 2016 09:40:04 +0200')

start-date: Fri Feb 10 16:40:00 UTC 2017

The full mapping of gitorious repositories URLs to on-disk location is at uffizi:/srv/storage/space/mirrors/gitorious.org/full_mapping.txt

I'm now running a git fsck on all the repositories. Output and results in worker01:/tmp/fsck.

I've collapsed the two mappings into a single file: /srv/softwareheritage/mirrors/gitorious.org/full_mapping.txt

Here are all the information I have about the on-disk gitorious layout (credit: astrid):

Deployed the uid+gid changes and added the filesystem to uffizi:/etc/exports

To prepare for this, I moved temp-drydock to uid=10000 on worker01. If CI is broken it's my fault.

for reference, see the content of the gitorious disk image uffizi:/srv/softwareheritage/mirrors/gitorious.org/gitorious.img

This is now done. I'm running an fsck on the retrieved file system image just in case.

The transfer is now in progress on uffizi:/srv/softwareheritage/mirros/gitorious.org/, within a screen session of my user with title "gitorious-transfer".

We are now all set to start (after having automated it properly…) the transfer of Gitorious stuff to SWH.

Here is the complete list of URL that can be used to "git clone" (via HTTPS) all the repositories available from the Gitorious valhalla: