HomeSoftware Heritage

pattern: Ensure accurate origin counts returned by run method

Description

pattern: Ensure accurate origin counts returned by run method

Previously, the run method was returning the total count of ListedOrigin
objects sent to scheduler database.

However, some listers can send multiple ListedOrigin objects for a given
origin URL during the listing process, for instance when an origin is
contained in multiple pages (e.g. gogs listing) or when the listing
is gathering multiple versions of an origin spread across multiple
pages (e.g. maven listing).

This changes ensures an accurate count of listed origins by maintaining
a set of origin URLs associated to the sent ListedOrigin objects.

Details

Provenance
anlambertAuthored on Sep 29 2022, 11:14 AM
anlambertPushed on Sep 29 2022, 11:26 AM
Differential Revision
D8565: pattern: Ensure accurate origin counts returned by run method
Parents
rDLS3928fc9ee9ff: Nuget: Lister for NuGet the package manager for .NET
Branches
Unknown
Tags
Unknown
Build Status
Buildable 31913
Build 49951: test-and-buildJenkins console · Jenkins