• maven-lister: add last modif time to extra args for jar files.

• maven-lister: add extra loader arguments for jar loader output.
• maven-lister: fix tests for extra loader asserts.

Updating D6106: T3339 Fix various formatting typos in getting started with SWH API.

Hiho good people, here is a quick fix for some minor typos, mostly linked to the output rendering of pre's.

• T3339 Getting started with the SWH API.
• Merge branch 'master' of https://forge.softwareheritage.org/source/swh-docs

@vlorentz Thanks for the review! Updated diff with your last comment (api in index.rst). I squashed commits and am ready to push to master, as soon as you tell me to.

• Merge branch 'master' into T3339_getting_started_api
• Fix wrong index after api rename.

• Fix sections.

• Merge branch 'master' into T3339_getting_started_api
• Fix url to https in sphinx.rst.
• Fix api getting started + rename file.
• More fixes to api getting started.

Thanks @ardumont for the heads-up.
As a side note I still have on my to-do to add authentication and svn support to the tuleap lister (but that will happen *after* the maven lister work).

Updates:

• A ticket has been submitted in the Sonatype JIRA to let them know we will fetch maven poms and src jars soon.
• An email has been sent on the maven-dev mailing list with a few kind answers, mainly stating to let Sonatype know through a JIRA issue.
• Hervé Bouthemy provided some precious insights about the best way to use the poms; it seems we can get a near-complete list of maven repositories worldwide by parsing some pom arguments and following dependencies up. It should probably not be used directly by the lister (which should provide only the list of src jars and scm attributes to the loaders), but we can output it somewhere to feed the lister manually.

Update for the Maven Indexer prototype: it works! (finally)

In D5754#147866, @ardumont wrote:

By the way @borisbaldassari, you can push, that should land and close the diff ;)

Some more information about the maven indexer. Beware people it's a bit dirty, and you're not going to like it infra-wise.

So, to sum up the options we have.. Basically we "just" need all artifacts coordinates. From there for each artifact we can:

Few more cents in the bucket..

• scrapping is explicitly forbidden, see https://repo1.maven.org/terms.html -- however making contact first will help us go through most of the abuse-limiting rules I guess.
• regarding fasten, there are indeed some bits that could be useful. However most of our difficulties are in getting a list of projects, whereas this information is already provided by the user in the case of fasten. So, interesting and useful, but not a game changer regarding the difficult part of our job.

Hey @ardumont , ok, was not sure of what to do next. Thanks, just pushed it.
BTW I got some news from the nice Enalean people, so I'll add the authentication and svn repos soon, in another task/diff/push.

I think I'm done for now. It works well on docker-dev, tests pass, all review comments have been treated.
For the records the following improvements could be implemented:

• support for svn repos -- I can't find any tuleap forge with svn repos, so I'll drop the feature for now.
• authentication -- not a big deal, but tuleap.net has 41 repositories so rate-limiting and auth aren't absolutely required. I'm waiting for inputs from enalean (makers of tuleap) to decide on what would actually be needed.

D5754: tuleap: initialise lister.

• tuleap-lister: code review: Add empty repo test, minor typo fixes.

• tuleap-lister: code review: fix chmod on json files.
• tuleap-lister: code review: fix var names + add tests.
• tuleap-lister: code review: fix useless indirection.

@ardumont @vlorentz thanks a lot for all the help and inputs.

• tuleap-lister: code review: Update copyright to 2021 exactly.
• tuleap-lister: code review: Update py files perms -X.
• tuleap-lister: code review: minimise json files.

Hi vlorentz, thanks for the inputs (and good shot for the copyrights + chmod!!)

• tuleap-lister: code review: improve results_simplified for svn repos.
• tuleap-lister: code review: add name to CONTRIBUTORS file.
• tuleap-lister: code review: Update tutorial for misc files to edit.

• tuleap-lister: code review: Remove authentication useless lines + fix typos.