HomeSoftware Heritage

Add a non-incremental sourceforge lister

Description

Add a non-incremental sourceforge lister

Following zack's work on T735, this change introduces an actual SWH lister for
SourceForge.

SourceForge provides a main sitemap that lists sharded sitemaps, which
themselves list pages. Each page belongs to a project (or sub-project,
though those are rare), information about which can be found by querying
a REST API, which gives us the list of any and all VCS used for said
project. Both sitemaps and pages have a "last modified" timestamp that
will be used in a future patch to implement incremental listing.

More precise information can be found as inline comments or docstrings.

Details

Provenance
AlphareAuthored on Mar 17 2021, 5:39 PM
AlpharePushed on Mar 23 2021, 6:41 PM
Differential Revision
D5293: Add a non-incremental sourceforge lister
Parents
rDLS879170a57d17: GitHub: handle edge cases with empty responses
Branches
Unknown
Tags
Unknown
Build Status
Buildable 20160
Build 31295: test-and-buildJenkins console · Jenkins