Page MenuHomeSoftware Heritage

Archive contentFolder
ActivePublic

Members

  • This project does not have any members.
  • View All

Watchers

  • This project does not have any watchers.
  • View All

Details

Description

stuff related to content (of all kinds, not only "blobs") that is already stored in the Software Heritage archive

Recent Activity

Apr 11 2022

olasd added a comment to T3656: Survey revisions/releases with partially loaded history.

Looks like the number of affected revisions is fluctuating a bit:

Apr 11 2022, 4:10 PM · Archive content
bchauvet added a comment to T3660: Nodes with missing ancestors in SWH DAG / SWH-graph.
  • needs review against the list generated for T3656
Apr 11 2022, 11:54 AM · Archive content
bchauvet added a comment to T3656: Survey revisions/releases with partially loaded history.
  • quantify the “hole” problem which prevents us to do T3655 altogether
Apr 11 2022, 11:54 AM · Archive content

Feb 1 2022

vlorentz added a comment to T75: Check integrity of directories, revisions, and releases.

With some more heuristics, retrying failed origins, and looking at dumps on banco (great idea from @olasd), I was able to bring down the total number of unrecoverable objects to 95k:

Feb 1 2022, 9:43 AM · Archive content, Restricted Project

Jan 31 2022

vlorentz added a comment to T3860: Document types/sources of object corruption.

got a list of weirdness types here: https://github.com/dulwich/dulwich/pull/927 and https://github.com/dulwich/dulwich/issues/931

Jan 31 2022, 10:49 AM · Archive content, Documentation

Jan 18 2022

vlorentz added a parent task for T75: Check integrity of directories, revisions, and releases: T3860: Document types/sources of object corruption.
Jan 18 2022, 12:44 PM · Archive content, Restricted Project
vlorentz added a subtask for T3860: Document types/sources of object corruption: T75: Check integrity of directories, revisions, and releases.
Jan 18 2022, 12:44 PM · Archive content, Documentation
vlorentz triaged T3860: Document types/sources of object corruption as High priority.
Jan 18 2022, 12:44 PM · Archive content, Documentation

Dec 13 2021

ardumont moved T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 from Backlog to Deployed on the SWORD deposit board.
Dec 13 2021, 11:27 AM · System administration, SWORD deposit, Package Loader, Data Model, Archive content

Nov 26 2021

vlorentz added a comment to T3638: Make package loaders create releases objects instead of revisions.

Copy of an email I sent on 2021-11-17:

Nov 26 2021, 4:02 PM · Package Loader, Data Model, Archive content

Nov 22 2021

vlorentz closed T3636: Make the opam loader write extrinsic metadata, a subtask of T3638: Make package loaders create releases objects instead of revisions, as Resolved.
Nov 22 2021, 2:44 PM · Package Loader, Data Model, Archive content
vlorentz closed T3638: Make package loaders create releases objects instead of revisions, a subtask of T1258: Synthesize release objects for all upstream things that match the concept of a release, as Resolved.
Nov 22 2021, 2:44 PM · Archive content
vlorentz closed T3638: Make package loaders create releases objects instead of revisions as Resolved.
Nov 22 2021, 2:43 PM · Package Loader, Data Model, Archive content
ardumont closed T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3, a subtask of T3638: Make package loaders create releases objects instead of revisions, as Resolved.
Nov 22 2021, 2:29 PM · Package Loader, Data Model, Archive content
ardumont closed T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3 as Resolved.
Nov 22 2021, 2:29 PM · System administration, Package Loader, Data Model, Archive content
ardumont moved T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3 from deployed/landed/monitoring to Component upgrades on the System administration board.
Nov 22 2021, 2:29 PM · System administration, Package Loader, Data Model, Archive content
ardumont moved T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3 from in-progress to deployed/landed/monitoring on the System administration board.
Nov 22 2021, 2:05 PM · System administration, Package Loader, Data Model, Archive content
ardumont changed the status of T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3, a subtask of T3638: Make package loaders create releases objects instead of revisions, from Open to Work in Progress.
Nov 22 2021, 2:05 PM · Package Loader, Data Model, Archive content
ardumont changed the status of T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3 from Open to Work in Progress.
Nov 22 2021, 2:05 PM · System administration, Package Loader, Data Model, Archive content
ardumont added a project to T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3: System administration.
Nov 22 2021, 2:04 PM · System administration, Package Loader, Data Model, Archive content
ardumont updated the task description for T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3.
Nov 22 2021, 2:02 PM · System administration, Package Loader, Data Model, Archive content
ardumont updated the task description for T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3.
Nov 22 2021, 1:59 PM · System administration, Package Loader, Data Model, Archive content
ardumont updated the task description for T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3.
Nov 22 2021, 1:56 PM · System administration, Package Loader, Data Model, Archive content
ardumont updated the task description for T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3.
Nov 22 2021, 1:52 PM · System administration, Package Loader, Data Model, Archive content
ardumont updated the task description for T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3.
Nov 22 2021, 1:15 PM · System administration, Package Loader, Data Model, Archive content
ardumont renamed T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3 from production: Deploy package loader v1.0, deposit server v0.16, lister v2.3 to production: Deploy package loader v1.1, deposit server v0.16, lister v2.3.
Nov 22 2021, 1:06 PM · System administration, Package Loader, Data Model, Archive content
vlorentz placed T1260: Extend the release object model to allow synthetic objects up for grabs.
Nov 22 2021, 12:05 PM · Archive content
ardumont triaged T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3 as Normal priority.
Nov 22 2021, 11:30 AM · System administration, Package Loader, Data Model, Archive content

Nov 10 2021

ardumont closed T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3, a subtask of T3638: Make package loaders create releases objects instead of revisions, as Resolved.
Nov 10 2021, 4:43 PM · Package Loader, Data Model, Archive content
ardumont closed T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 as Resolved.
Nov 10 2021, 4:43 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
ardumont placed T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 up for grabs.
Nov 10 2021, 4:43 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
ardumont moved T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 from in-progress to deployed/landed/monitoring on the System administration board.
Nov 10 2021, 4:29 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
ardumont added a comment to T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3.

At least loader deposit and npm [1] are fine.

Nov 10 2021, 4:24 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
ardumont updated the task description for T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3.
Nov 10 2021, 4:24 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
ardumont moved T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 from Backlog to in-progress on the System administration board.
Nov 10 2021, 3:35 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
ardumont renamed T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 from staging: Deploy package loader v1.0 and deposit server v0.16 to staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3.
Nov 10 2021, 3:33 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
ardumont changed the status of T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3, a subtask of T3638: Make package loaders create releases objects instead of revisions, from Open to Work in Progress.
Nov 10 2021, 3:33 PM · Package Loader, Data Model, Archive content
ardumont changed the status of T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 from Open to Work in Progress.
Nov 10 2021, 3:33 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
ardumont updated the task description for T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3.
Nov 10 2021, 3:32 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
vlorentz updated the task description for T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3.
Nov 10 2021, 3:21 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
vlorentz added a parent task for T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3: T3636: Make the opam loader write extrinsic metadata.
Nov 10 2021, 3:20 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content
ardumont triaged T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 as Normal priority.
Nov 10 2021, 3:17 PM · System administration, SWORD deposit, Package Loader, Data Model, Archive content

Nov 9 2021

vlorentz added a revision to T3638: Make package loaders create releases objects instead of revisions: D6618: Document how each package loader populates fields..
Nov 9 2021, 12:39 PM · Package Loader, Data Model, Archive content

Nov 8 2021

vlorentz added revisions to T3638: Make package loaders create releases objects instead of revisions: D6616: Make package loaders write releases instead of revisions, D6617: Use release instead of revision as anchor in SWHID context instead..
Nov 8 2021, 11:58 AM · Package Loader, Data Model, Archive content
vlorentz added a comment to T3638: Make package loaders create releases objects instead of revisions.

Here is an overview of the fields (+ internal version name + branch name) used by each package loader, after D6616:

Nov 8 2021, 11:50 AM · Package Loader, Data Model, Archive content

Oct 22 2021

vlorentz added revisions to T1258: Synthesize release objects for all upstream things that match the concept of a release: D6529: deposit: Remove 'parent' deposit, D6530: Remove unused 'known_artifacts' code.
Oct 22 2021, 3:45 PM · Archive content
vlorentz added a comment to T75: Check integrity of directories, revisions, and releases.

Great news: of the 469k corrupt SVN revisions, all but 14 (yes, 14) can be fixed simply by adding 1 microsecond to their timestamp.

Oct 22 2021, 2:33 PM · Archive content, Restricted Project

Oct 20 2021

vlorentz added a comment to T75: Check integrity of directories, revisions, and releases.

After further investigation, I can't find any directory that is in a completely bad order; they are either ordered like git does (by adding a / at the end of dir entries) or by assuming a null byte at the end of dir entries.

Oct 20 2021, 12:18 PM · Archive content, Restricted Project

Oct 15 2021

vlorentz added a comment to T75: Check integrity of directories, revisions, and releases.

analysis on directories (some are also part of the fixable_trivial above, but I don't have the exact number, I lost it in my analysis):

Oct 15 2021, 11:21 AM · Archive content, Restricted Project
zack updated subscribers of T3656: Survey revisions/releases with partially loaded history.
Oct 15 2021, 9:34 AM · Archive content