Page MenuHomeSoftware Heritage

Implement a Guix/Nix loader
Closed, MigratedEdits Locked

Description

The objective of this task is to create a loader for Nix/Guix distributions. These distributions expose a sources.json containing all sources used at a specific moment.

This work could be split into the following diffs:

  • D2792: Create a loader able to load archives
  • D2807: Add a pointer in the snapshot to the commit used to generate the sources.json file
  • T2411: Deploy current nixguix loader version (fix issues along the way)

Related Objects

Event Timeline

ardumont triaged this task as Normal priority.Sep 9 2019, 10:08 AM
ardumont created this task.
ardumont renamed this task from Implement a Guix/GuixSD/Nix/NixOS lister to Implement a Guix/GuixSD/Nix/NixOS loader.Mar 5 2020, 7:28 PM
ardumont added a project: Core Loader.

Implementation of a lister started at D2025.

ardumont renamed this task from Implement a Guix/GuixSD/Nix/NixOS loader to Implement a Guix/Nix loader.Mar 5 2020, 7:30 PM
ardumont assigned this task to lewo.

For information, this has been running on staging daily for a while now without much issues. The sources.json are nix specifics for now.

@lewo decreased the number of potential errors [1] in the current loader by
removing potential issues in the sources.json [2]

Latest run on staging:

May 17 10:55:39 worker0 python3[1066]: [2020-05-17 10:55:39,333: INFO/ForkPoolWorker-3] Task swh.loader.package.nixguix.tasks.LoadNixguix[7982729e-96af-4700-8ae0-afa499a10830] succeeded in 2393.330544631928s: {'status': 'eventful', 'snapshot_id': 'a990925694c53453311030c652570901848b8a1d'}

[1] which will be improved upon later

[2] This is the files declaring the artifacts for the nixguix_loader to ingest.

ardumont changed the status of subtask T2411: Deploy nixguix loader from Open to Work in Progress.May 20 2020, 10:18 AM

The main loader is now deployed.
Another task got opened to keep the references on the possible improvments [1]

[1] T2879

gitlab-migration changed the status of subtask T2371: nixguix: fails to use previous visit snapshot from Resolved to Migrated.
gitlab-migration changed the status of subtask T2411: Deploy nixguix loader from Resolved to Migrated.
gitlab-migration changed the status of subtask T2510: nixguix: Filter out unsupported artifacts from ingestion from Resolved to Migrated.