Page MenuHomeSoftware Heritage
Feed Advanced Search

Jan 8 2023

gitlab-migration closed T4538: Consider archiving NAR hashes as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:24 PM · SVN Loader, Tarball loader, Nixguix loader
gitlab-migration changed the status of T539: Update loaders to register origin_visit's state from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:56 PM · Directory loader, Tarball loader, Git loader, SVN Loader
gitlab-migration changed the status of T683: swh-loader-tar generates dangling releases from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:20 PM · Tarball loader, Restricted Project
gitlab-migration changed the status of T345: Update db swh.revision.metadata data with tarball length, a subtask of T339: add missing "length" property to revision.metadata JSON for all imported tar/zip, from Resolved to Migrated.
Jan 8 2023, 4:18 PM · Tarball loader
gitlab-migration changed the status of T344: Add missing length tarball property in revision.metadata json in loader, a subtask of T339: add missing "length" property to revision.metadata JSON for all imported tar/zip, from Resolved to Migrated.
Jan 8 2023, 4:18 PM · Tarball loader
gitlab-migration changed the status of T345: Update db swh.revision.metadata data with tarball length from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Tarball loader
gitlab-migration changed the status of T344: Add missing length tarball property in revision.metadata json in loader from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Tarball loader
gitlab-migration changed the status of T339: add missing "length" property to revision.metadata JSON for all imported tar/zip from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Tarball loader
gitlab-migration changed the status of T324: Check loader-tar's storage compatibility from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Tarball loader
gitlab-migration changed the status of T302: swh-loader-tar origin validities are the current time instead of the mirroring time from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Tarball loader
gitlab-migration changed the status of T192: analyze 4 loading failures for GNU tarballs and reimport them from Wontfix to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Tarball loader
gitlab-migration changed the status of T184: Create a tarball from swh storage from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Tarball loader
gitlab-migration changed the status of T183: gnu/old-gnu injection to production, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T183: gnu/old-gnu injection to production from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T182: investigate gnu.org tarball ingestion failures from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Origin-GNU, Directory loader, Tarball loader
gitlab-migration changed the status of T178: Replay loading mirror gnu and old-gnu from scratch, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T178: Replay loading mirror gnu and old-gnu from scratch from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T176: Fix `Permission denied: write to /usr/local/lib/gcl-3...` , a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T176: Fix `Permission denied: write to /usr/local/lib/gcl-3...` from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T177: Investigate `Not implemented 'compression type 6 implode'` from Duplicate to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T175: Redeploy and trigger sublist of tarballs impacted by errors, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T175: Redeploy and trigger sublist of tarballs impacted by errors from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T174: Analyse and fix last errors from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T174: Analyse and fix last errors, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T170: Random order in tarball messages computation, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T171: Fix content loading only when needed, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T170: Random order in tarball messages computation from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T158: Package latest swh-loader-tar, a subtask of T155: Deploy and load tarballs from worker01, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T158: Package latest swh-loader-tar from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T157: Package latest swh-loader-dir, a subtask of T155: Deploy and load tarballs from worker01, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T157: Package latest swh-loader-dir from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T156: Package latest swh-storage from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T156: Package latest swh-storage, a subtask of T155: Deploy and load tarballs from worker01, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T155: Deploy and load tarballs from worker01, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T155: Deploy and load tarballs from worker01 from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T154: Send tarball checksums along the revision, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T154: Send tarball checksums along the revision from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T149: Populate fetch_history table, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T148: Populate fetch_history, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T148: Populate fetch_history from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Tarball loader
gitlab-migration changed the status of T147: Migrate code to use latest schema improvments from T3, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Invalid to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T147: Migrate code to use latest schema improvments from T3 from Invalid to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T142: reload missing tarballs, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Invalid to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T143: understand why errors does not appear in log db from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T142: reload missing tarballs from Invalid to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T143: understand why errors does not appear in log db, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T140: analysis errors on first loading batch from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T141: compute missing tarballs list for reloading the ones missing, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T140: analysis errors on first loading batch, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T141: compute missing tarballs list for reloading the ones missing from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T136: Load tarballs in a production-like swh-storage instance on worker01, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T136: Load tarballs in a production-like swh-storage instance on worker01 from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Directory loader, Tarball loader
gitlab-migration changed the status of T127: release swh-loader-tar, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T127: release swh-loader-tar from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T125: Permits loading of tarball subsets from a file, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T124: Tryouts run on gnu to complete the tarball listing exhaustiveness, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T126: release swh-core, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T123: Tryouts run on old-gnu to complete the tarball listing exhaustiveness, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T107: Load an actual tarball in swh, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T105: make worker01 stop consuming swh_loader_git messages, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T106: Test on worker01: producing and consuming one swh_loader_tar message, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T105: make worker01 stop consuming swh_loader_git messages from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T106: Test on worker01: producing and consuming one swh_loader_tar message from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T104: release swh-loader-dir, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:17 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T90: swh-loader-tar - Load gnu tarballs in swh-storage from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:16 PM · Tarball loader, Directory loader
gitlab-migration changed the status of T22: DB schema: add metadata field to revision table, a subtask of T90: swh-loader-tar - Load gnu tarballs in swh-storage, from Resolved to Migrated.
Jan 8 2023, 4:16 PM · Tarball loader, Directory loader

Oct 31 2022

ardumont added a comment to T4538: Consider archiving NAR hashes.

Oh awesome! Thanks for pushing forward. :-)

Oct 31 2022, 12:14 PM · SVN Loader, Tarball loader, Nixguix loader

Oct 22 2022

zimoun added a comment to T4538: Consider archiving NAR hashes.

Oh awesome! Thanks for pushing forward. :-)

Oct 22 2022, 12:46 PM · SVN Loader, Tarball loader, Nixguix loader

Oct 21 2022

ardumont updated subscribers of T4538: Consider archiving NAR hashes.

fwiw, I've iterated a bit over @zimoun's code and pushed it into the snippets repository
(see commits above and their commit description message). It's also able to deal with
git, hg and svn trees (ignoring their respective top metadata folder .git, .svn, ...
without impacting the performance).

Oct 21 2022, 3:46 PM · SVN Loader, Tarball loader, Nixguix loader

Oct 14 2022

zimoun added a comment to T4538: Consider archiving NAR hashes.
import os, stat, hashlib, io
from pathlib import Path
Oct 14 2022, 10:23 AM · SVN Loader, Tarball loader, Nixguix loader
zimoun added a comment to T4538: Consider archiving NAR hashes.

This tree

$ tree foo
foo
├── bar
│   └── exe
└── baz
Oct 14 2022, 10:23 AM · SVN Loader, Tarball loader, Nixguix loader

Oct 11 2022

zimoun added a comment to T4538: Consider archiving NAR hashes.

To be complete, the integrity checksum that Guix provides (here one example about the package guix-modules),

Oct 11 2022, 8:19 PM · SVN Loader, Tarball loader, Nixguix loader
zimoun added a comment to T4538: Consider archiving NAR hashes.

Related to P1473.

Oct 11 2022, 7:59 PM · SVN Loader, Tarball loader, Nixguix loader

Oct 5 2022

ardumont added a revision to T4538: Consider archiving NAR hashes: D8625: docker: Install nix binaries in swh/stack image.
Oct 5 2022, 11:30 AM · SVN Loader, Tarball loader, Nixguix loader

Sep 23 2022

zimoun added a comment to T4538: Consider archiving NAR hashes.

Thanks for raising the questions.

Sep 23 2022, 8:05 PM · SVN Loader, Tarball loader, Nixguix loader

Sep 19 2022

ardumont added a comment to T4538: Consider archiving NAR hashes.

yes, nix too. As far as i remember, it's NAR for Nix ARchive ;)

Sep 19 2022, 9:52 AM · SVN Loader, Tarball loader, Nixguix loader

Sep 18 2022

vlorentz triaged T4538: Consider archiving NAR hashes as Normal priority.
Sep 18 2022, 6:54 PM · SVN Loader, Tarball loader, Nixguix loader

Jun 8 2021

anlambert merged task T177: Investigate `Not implemented 'compression type 6 implode'` into T3369: swh.core.tarballl.uncompress: Some zip files fail to be uncompressed.
Jun 8 2021, 5:02 PM · Directory loader, Tarball loader

Dec 10 2020

zack closed T192: analyze 4 loading failures for GNU tarballs and reimport them as Wontfix.

Closing as long-stale. Further investigation should be done when we restart periodic GNU release ingestion.

Dec 10 2020, 11:19 AM · Tarball loader

May 25 2019

zack renamed T192: analyze 4 loading failures for GNU tarballs and reimport them from Analyze 4 unexpected gnu tarball failures (+ reimport the archives) to analyze 4 loading failures for GNU tarballs and reimport them.
May 25 2019, 4:52 PM · Tarball loader
zack changed the visibility for T192: analyze 4 loading failures for GNU tarballs and reimport them.
May 25 2019, 4:52 PM · Tarball loader
zack closed T182: investigate gnu.org tarball ingestion failures as Resolved.

oh, the info is in T192 already
closing

May 25 2019, 4:52 PM · Origin-GNU, Directory loader, Tarball loader
zack added a comment to T182: investigate gnu.org tarball ingestion failures.

is this still going? can it be closed as obsolete, maybe just noting down here the 4 failed tarballs? (we're going to do a full swipe soon anyway with the new listers/loaders)

May 25 2019, 4:51 PM · Origin-GNU, Directory loader, Tarball loader

Mar 12 2019

ardumont added a comment to P369 draft: gnu mail to ask for listing.

Related T1351

Mar 12 2019, 6:50 PM · Tarball loader
ardumont edited P369 draft: gnu mail to ask for listing.
Mar 12 2019, 6:50 PM · Tarball loader
ardumont edited P369 draft: gnu mail to ask for listing.
Mar 12 2019, 5:53 PM · Tarball loader
ardumont created P369 draft: gnu mail to ask for listing.
Mar 12 2019, 5:52 PM · Tarball loader

Jul 18 2018

ardumont removed a member for Tarball loader: ardumont.
Jul 18 2018, 12:30 PM

Feb 24 2017

ardumont added a comment to T683: swh-loader-tar generates dangling releases.

Actions undertook to clean dangling releases:

Feb 24 2017, 8:56 AM · Tarball loader, Restricted Project

Feb 22 2017

ardumont closed T683: swh-loader-tar generates dangling releases as Resolved.
Feb 22 2017, 4:35 PM · Tarball loader, Restricted Project
ardumont updated the task description for T683: swh-loader-tar generates dangling releases.
Feb 22 2017, 4:31 PM · Tarball loader, Restricted Project
ardumont updated the task description for T683: swh-loader-tar generates dangling releases.
Feb 22 2017, 2:14 PM · Tarball loader, Restricted Project
ardumont updated the task description for T683: swh-loader-tar generates dangling releases.
Feb 22 2017, 2:14 PM · Tarball loader, Restricted Project
ardumont updated the task description for T683: swh-loader-tar generates dangling releases.
Feb 22 2017, 2:14 PM · Tarball loader, Restricted Project

Feb 21 2017

ardumont claimed T683: swh-loader-tar generates dangling releases.
Feb 21 2017, 12:25 PM · Tarball loader, Restricted Project

Feb 20 2017

ardumont added a comment to T683: swh-loader-tar generates dangling releases.

Of course we will also need to remove all dangling releases that have been generated thus far.

Feb 20 2017, 3:19 PM · Tarball loader, Restricted Project
zack added a comment to T683: swh-loader-tar generates dangling releases.

Yep. Of course we will also need to remove all dangling releases that have been generated thus far.

Feb 20 2017, 3:15 PM · Tarball loader, Restricted Project
ardumont added a comment to T683: swh-loader-tar generates dangling releases.

After discussion, we will stop generating dangling releases and keep the occurrences pointing at revisions.

Feb 20 2017, 2:19 PM · Tarball loader, Restricted Project

Feb 16 2017

zack moved T683: swh-loader-tar generates dangling releases from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Feb 16 2017, 9:15 AM · Tarball loader, Restricted Project

Feb 15 2017

olasd created T683: swh-loader-tar generates dangling releases.
Feb 15 2017, 6:06 PM · Tarball loader, Restricted Project