HomeSoftware Heritage

pypi.loader: Make the loader incremental

Description

pypi.loader: Make the loader incremental

Leverage snapshot of the last visit (if any) to filter out the already
seen artifacts. A new snapshot is then creating from the previous one
adding the new release artifact(s) detected (if any).

If no new artifact is detected, the snapshot remains the same.

Related T421

Details

Provenance
ardumontAuthored on Sep 5 2018, 3:58 PM
ardumontPushed on Sep 5 2018, 4:07 PM
Parents
rDLDPYad7a684cffbb: pypi.model: Clarify warning log
Branches
Unknown
Tags
Unknown