Page MenuHomeSoftware Heritage
Create Task
Maniphest T2313

Archive git.fsfe.org (Gitea)
Closed, MigratedEdits Locked
Actions

Description

At the Free Software Foundation Europe (FSFE), we use a git.fsfe.org (based on Gitea) to host all our code and websites. This includes a full archive of the commits to fsfe.org which definitely has some historical value ;)

How could this be achieved? I learnt that there is no connector from Gitea to Software Heritage, but perhaps there is some automatic scanner for the public repos we host.

Unfortunately we do not have the resources (mostly regarding Go skills) to come up with a patch for Gitea's code, but I would be happy to assist, test, and give feedback!

PS: @zack asked me to file a feature request for my inquiry here. If I did not sort/label my request correctly, please feel free to edit.

Related Objects
Search...

Event Timeline

mxmehl triaged this task as Wishlist priority.Mar 13 2020, 12:55 PM
mxmehl created this task.
mxmehl created this object in space S1 Public.
olasd added projects: Lister, Archive coverage.Mar 13 2020, 1:21 PM
olasd added a subscriber: olasd.Mar 13 2020, 1:25 PM

Hi!

Thanks for submitting this request. There's a good chance that this can be the same lister as gogs: T1721.

There's a public Repository API: https://git.fsfe.org/api/v1/repos/search, so I guess the only thing missing is a lister on our side.

This looks like a cross between GitHub and GitLab's APIs, shouldn't be too hard for someone to quickly implement it in our codebase.

olasd added a comment.Mar 13 2020, 1:26 PM

Unfortunately, try.gogs.io's API is hidden behind auth so I can't confirm that the responses actually have the same shape between gogs and gitea.

zack updated the task description. (Show Details)Mar 24 2020, 1:40 PM
zack added a subscriber: zack.
legau mentioned this in D3107: Added GiteaLister.Apr 30 2020, 4:01 PM
ardumont added a subscriber: ardumont.May 27 2020, 6:06 PM

I've add multiple looks to the proposed gitea lister.
This looks fine to me, i've accepted it but not completely.
If some other team member could do a second pass, that'd be neat.

Cheers,

legau mentioned this in rDLS1408517c089f: Added GiteaLister.Jun 10 2020, 5:04 PM
douardda added a subscriber: douardda.EditedAug 26 2020, 10:23 AM

now we have the gitea lister, we should (upgrade swh.lister on prod and) add a few listing tasks, like this fsfe instance, as well as other instances like https://codeberg.org.

discussions on having lists of public gitea instances:

and

douardda added a comment.Aug 26 2020, 10:48 AM

Ok I was expecting something a bit smart in explore.sapk.fr, but not really:

https://gitea.com/sapk/explore/src/branch/master/main.go#L28

douardda added a comment.Aug 26 2020, 11:08 AM

Also beware that the default pagination value in the gitea lister is 3 (https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gitea/lister.py$23) so it is very slow.

So when a gitea listing task is about to be added, make sure to give this limit argument a decent value (eg. 100)

douardda raised the priority of this task from Wishlist to High.Aug 27 2020, 4:28 PM
vsellier mentioned this in T2577: Test gitea lister on staging environment.Sep 8 2020, 4:34 PM
vsellier added a subtask: T2577: Test gitea lister on staging environment.
vsellier changed the status of subtask T2577: Test gitea lister on staging environment from Open to Work in Progress.
vsellier closed subtask T2577: Test gitea lister on staging environment as Resolved.Sep 10 2020, 1:07 PM
vsellier reopened subtask T2577: Test gitea lister on staging environment as Work in Progress.Sep 10 2020, 7:03 PM
vsellier added a subtask: T2608: Deploy launchpad and gitea listers on production.Sep 17 2020, 10:40 AM
vsellier closed subtask T2577: Test gitea lister on staging environment as Resolved.Sep 17 2020, 11:41 AM
ardumont closed subtask T2608: Deploy launchpad and gitea listers on production as Resolved.Sep 21 2020, 1:45 PM
douardda added a comment.Sep 28 2020, 9:47 AM

Can this be closed now? What's missing? Adding a listing task?

ardumont added a comment.Sep 28 2020, 10:54 AM

The lister is deployed, this forge is not listed though (codeberg.org is).

douardda added a comment.Sep 29 2020, 11:39 AM

I've sent an email to the fsfe.

douardda closed this task as Resolved.Sep 30 2020, 9:37 AM
douardda claimed this task.

Listed (oneshot full + recurring incremental) and loaded (as far as I can tell).

gitlab-migration changed the status of subtask T2608: Deploy launchpad and gitea listers on production from Resolved to Migrated.Oct 19 2022, 5:58 PM
gitlab-migration changed the task status from Resolved to Migrated.Jan 8 2023, 4:29 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

gitlab-migration changed the status of subtask T2577: Test gitea lister on staging environment from Resolved to Migrated.Jan 8 2023, 4:31 PM
About · Developer information · Legal