HomeSoftware Heritage

Crates.io: Add last_update for each version of a crate

Description

Crates.io: Add last_update for each version of a crate

In order to reduce http api call amount made by the loader, download a
crates.io database dump, and parse its csv files to get a last_update
value for each versions of a Crate.
Those values are sent to the loader through extra_loader_arguments
'crates_metadata'.

'artifacts' and 'crates_metadata' now uses "version" as key.

Related T4104, D8171

Details

Provenance
franckbretAuthored on Sep 12 2022, 9:33 PM
franckbretPushed on Oct 5 2022, 5:13 PM
franckbretPushed on Oct 5 2022, 5:11 PM
Differential Revision
D8454: Crates.io: Add last_update for each version of a crate
Parents
rDLS2e6e282d4464: nixguix: Deal with manifest entries without an integrity field
Branches
Unknown
Tags
Unknown
Tasks
T4104: Ingest crates.io (Rust)
Build Status
Buildable 32130
Build 50308: test-and-buildJenkins console · Jenkins