Ok, so for those, the idea of having patterns to fill in seems simple.

We could have the following pattern to initialize on an instance basic
(init):

<url>/<main-pattern><sub-pattern>

With this, we could simplify the other instances.
Without failing to deal with the first one which are already ok.

• swh.lister.cgit: Remove repo page visit step

• swh.lister.cgit: Remove repo page visit step

Well, as a first approximation, i'd say let's go towards removing the visit step anyway (and compute basic git clone url, self.get_url sounds good enough IIRC).

self.get_url finds the url of the repo page
This method could work for https://git.kernel.org/ and http://git.savannah.gnu.org/cgit/ (maybe some more) but

Make recommended changes

mozilla
https://phabricator.services.mozilla.com/

• Add instance to the model

• swh.lister.cgit: Add pagination support

In D1441#36876, @nahimilega wrote:

I am not able to push this

I think because I have rebased it on latest master after merging D1612. This could be the reason

Rebase on latest master

I am not able to push this

• Add point 3 in 'run a new lister' and made recommended changes

I tested it in docker. It is running fine. I tested this on https://git.zx2c4.com/ instance
Here is one of the task created by the lister

Made docstrings according to napoleon style

Improved code quality as recommended by vlorentz

Fixed indendation of docstring and dict as recommended

Remove description from model

Made changes as advised

• Made recommended changes

In T1659#33593, @anarcat wrote:

but i'd say convert to python. depending on xmllint is very brittle... i already had to tweak the thing once to make it work at all, and the pipeline is kind of nasty. i think you will have to import some HTML parser at some point anyways, so you might as well bite that bullet now.

I would be pretty interested in integrating it with other listers and moving it to the common repo. I guess we can proceed in two ways.

1. Use the script that we already have. Just run this script via python to get the list of repos.

The point of this documentation snippet here was to demonstrate a docker refusing to start.
And to see what's happening through a docker-compose logs command as a first debugging step.

So if there needs an update (which i doubt), it'd be to rename swh-lister-debian_1 to swh-lister_1...

I tried

doco logs swh-lister_1

But it didn't work ` ERROR: No such service: swh-lister_1 `

Also, please refer to https://wiki.softwareheritage.org/wiki/Git_style_guide

We need three backquotes after the line

I did a bit investigation on data dumps, and it seems, they can serve the purpose well
To get the package release, we mainly need the name and version of packages. The link can, therefore, can be generated as -

@twitu please use imperative form while writing commit messages. Also, do keep in mind the why-not-how when writing the body of the commit message.

• Fixed the docunment according to suggestion

• Add new page "run_a new_lister"

I tried to run this for first 5 packages in packagist, and it worked fine

• Made change in the way api response is requested

• Made test to support different orders
• Put triple backquote around docstring

You can also use this phabricator instances.
Blender
https://developer.blender.org/

Make recommended changes

Made suggested changes

Add testing and how to run a new lister in docker section in tutorial, also fixed few grammatical errors