Adapt according to review:

• Add more cgit instance samples
• Rework docstring
• Rework url computation a bit

Build is green

Adapt according to review

The current implementation will not work for most cases where the base cgit URL and the base clone URL differs (see inline comment).

Landed and deployed in staging.

Current status of all listings:

npm listing done, so status ok as well:

npm run scheduled, run in progress:

pypi run triggered itself and went well all alone (cool):

Status lister: ok (with local patch):

I thought having fixed that bug (also encountered when developing the lister).

lister-cran status: run ko [1]

swhworker@worker0:~$ SWH_CONFIG_FILENAME=/etc/softwareheritage/lister.yml swh lister run --lister cran
Traceback (most recent call last):
  File "/usr/bin/swh", line 11, in <module>
    load_entry_point('swh.core==0.11.0', 'console_scripts', 'swh')()
...
  File "/usr/lib/python3/dist-packages/swh/core/api/__init__.py", line 344, in raise_for_status
    raise exception from None
swh.core.api.RemoteException: <RemoteException 500 TypeError: ["can not serialize 'Attribute' object"]>

[1] https://sentry.softwareheritage.org/share/issue/2cd53c7575834b1aaf65760b80bcbcef/

It's ok now.

Yes, I completely forgot about your diff that fixed it.

Node patch D4965, this gets better, the launchpad listed origins:

Note that this lister seems to need some writing improvments though.
It seemed to have flushed the writing only at the end of the listing.
If that's the real behavior (i'll need to check), that won't bode well for relatively high dimensioned instance like the cgit eclispe instance for example.

cgit lister should flush origins after each page, which instance has been listed here ?

Some listers like debian might flush a large amount of origins per page, will be curious to see how it goes.

lister launchpad run ko, see details [1]

My guess is that swh-scheduler 0.9.2 is not installed on staging, issue has been fixed in rDSCH2906b4e8a08517f5e3d86272232ad8ba926a43d7.

launchpad listing in progress and it seems to display the same behavior as the cgit lister (T2998#57500) [1]

lister-cran status: run ko [1]

gitea lister instance (https://try.gitea.io/api/v1), status ok:

cgit lister should flush origins after each page, which instance has been listed here ?

Note that this lister seems to need some writing improvments though.
It seemed to have flushed the writing only at the end of the listing.
If that's the real behavior (i'll need to check), that won't bode well for relatively high dimensioned instance like the cgit eclispe instance for example.

one cgit lister scheduled, status, it finished ok but [1]

Lister phabricator deployed with one instance (swh), status ok:

cli run for the bitbucket lister, status ok:

cli run for the github instance, status ok:

gitlab instance deployed, status ok:

Looks good to me.

Build is green

Add implementation instructions to the template

On a related note, I found this list of Community-Hosted GitLab Instances. Most of them have public access and could be added to the set of Gitlab instances listed by Software Heritage.