Page MenuHomeSoftware Heritage

Data ModelFolder
ActivePublic

Members

  • This project does not have any members.
  • View All

Watchers

  • This project does not have any watchers.
  • View All

Recent Activity

Fri, Sep 23

ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

Hum, for the 7 false, I have to check. For the 88 packages with no-origin, it is more
annoying. Well, some are metapackages as gcc-toolchain, so they can be skipped. Is it
ok for you to let this 'no-origin' type? For some others, I have to check if they are
covered elsewhere.

Fri, Sep 23, 3:48 PM · Data Model, Nixguix loader
ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

For ^, something like this would do [1]

Fri, Sep 23, 3:38 PM · Data Model, Nixguix loader
ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

Thanks for all that ^! And great pointers!

Fri, Sep 23, 3:09 PM · Data Model, Nixguix loader
zimoun added a comment to T3781: Replace the Nixguix loader with a lister.
  • artifacts url which are mostly tarballs [1] and sometimes files [2]
  • dvcs repositories delegated to dedicated loader to ingestion: svn [3], hg [4], git [5] (out of guix manifest)
  • Other stuff can be ignored as we don't have anything relevant to ingest [6]
Fri, Sep 23, 12:46 PM · Data Model, Nixguix loader
ardumont added a subtask for T3781: Replace the Nixguix loader with a lister: T3294: nixguix: Add support for pseudo-URLs with a missing schema.
Fri, Sep 23, 11:23 AM · Data Model, Nixguix loader

Wed, Sep 7

ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Wed, Sep 7, 12:02 PM · Data Model, Nixguix loader

Tue, Sep 6

ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Tue, Sep 6, 7:07 PM · Data Model, Nixguix loader
ardumont added a revision to T3781: Replace the Nixguix loader with a lister: D8406: [WIP] archive.loader: Allow archive loader to deal with nixguix archives.
Tue, Sep 6, 6:21 PM · Data Model, Nixguix loader
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Tue, Sep 6, 10:39 AM · Data Model, Nixguix loader
ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Tue, Sep 6, 10:32 AM · Data Model, Nixguix loader
ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

Some more information regarding extensions supported in nixpkgs then guix:

In [33]: sources = "https://nix-community.github.io/nixpkgs-swh/sources-unstable.json"
Tue, Sep 6, 9:42 AM · Data Model, Nixguix loader

Tue, Aug 30

ardumont added a revision to T3781: Replace the Nixguix loader with a lister: D8341: [WIP] nixguix: Boostrap lister.
Tue, Aug 30, 11:19 AM · Data Model, Nixguix loader

Jul 1 2022

douardda triaged T4370: Refactor the origin visit data model (aka get rid of the OriginVisit model object) as High priority.
Jul 1 2022, 4:35 PM · Storage manager, Data Model

Jun 30 2022

ardumont updated the task description for T3781: Replace the Nixguix loader with a lister.
Jun 30 2022, 3:25 PM · Data Model, Nixguix loader

Jun 29 2022

ardumont renamed T3781: Replace the Nixguix loader with a lister from Replace the Nixguix loader with a lister? to Replace the Nixguix loader with a lister.
Jun 29 2022, 11:05 AM · Data Model, Nixguix loader
ardumont added a parent task for T3781: Replace the Nixguix loader with a lister: T2879: Finalize nixguix loader implementation.
Jun 29 2022, 10:28 AM · Data Model, Nixguix loader

Jun 28 2022

ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

So taking a bit more look into this possible new lister, we'd end up with the following
possible outputs:

  • artifacts url which are mostly tarballs [1] and sometimes files [2]
  • dvcs repositories delegated to dedicated loader to ingestion: svn [3], hg [4], git [5] (out of guix manifest)
  • Other stuff can be ignored as we don't have anything relevant to ingest [6]
Jun 28 2022, 6:03 PM · Data Model, Nixguix loader

May 25 2022

ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

Another argument: currently, there is always at least some failures when loading real
Nix and Guix repositories, so visits always have status partial; which prevents them
from being listed in
https://archive.softwareheritage.org/browse/search/?q=&with_visit=true&with_content=true&visit_type=nixguix
(but we get results when un-checking " only show origins visited at least once")

May 25 2022, 10:14 AM · Data Model, Nixguix loader
vlorentz added a comment to T3781: Replace the Nixguix loader with a lister.

Another argument: currently, there is always at least some failures when loading real Nix and Guix repositories, so visits always have status partial; which prevents them from being listed in https://archive.softwareheritage.org/browse/search/?q=&with_visit=true&with_content=true&visit_type=nixguix (but we get results when un-checking " only show origins visited at least once")

May 25 2022, 10:02 AM · Data Model, Nixguix loader

Mar 25 2022

bchauvet lowered the priority of T3134: SWHID v2 from Normal to Low.
Mar 25 2022, 5:26 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

Mar 23 2022

bchauvet added a project to T3134: SWHID v2: Roadmap 2022.
Mar 23 2022, 4:42 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

Mar 16 2022

vlorentz closed T3752: Store/represent time offsets as strings, a subtask of T3594: Faithfully store weird git objects, as Resolved.
Mar 16 2022, 10:36 AM · meta-task, Data Model, Storage manager
vlorentz closed T3752: Store/represent time offsets as strings as Resolved.

swh-model 5.0.0 released, which finalizes these changes

Mar 16 2022, 10:36 AM · Data Model, Storage manager
vlorentz added revisions to T3752: Store/represent time offsets as strings: D7011: Revert "Restore 'offset' and 'negative_utc' arguments and make them optional", D7012: Remove deprecated property 'TimestampWithTimezone.offset'.
Mar 16 2022, 10:36 AM · Data Model, Storage manager

Mar 15 2022

vlorentz updated the task description for T3752: Store/represent time offsets as strings.
Mar 15 2022, 10:33 AM · Data Model, Storage manager

Mar 7 2022

vlorentz updated the task description for T3924: Write mailmaps after cooking git-bare archives with display names?.
Mar 7 2022, 12:54 PM · Data Model, Vault

Mar 3 2022

vlorentz closed T3890: schema of timezones in the journal as Resolved.
Mar 3 2022, 2:46 PM · Journal, Data Model

Feb 14 2022

vlorentz added a parent task for T3924: Write mailmaps after cooking git-bare archives with display names?: T3504: Make the git-bare cooker publicly available.
Feb 14 2022, 2:34 PM · Data Model, Vault

Feb 10 2022

vlorentz updated the task description for T3924: Write mailmaps after cooking git-bare archives with display names?.
Feb 10 2022, 11:05 AM · Data Model, Vault
vlorentz triaged T3924: Write mailmaps after cooking git-bare archives with display names? as Normal priority.
Feb 10 2022, 10:57 AM · Data Model, Vault

Feb 8 2022

vlorentz closed T3594: Faithfully store weird git objects as Resolved.
Feb 8 2022, 11:53 AM · meta-task, Data Model, Storage manager
vlorentz closed T3753: Store original git manifests as Resolved.
Feb 8 2022, 11:53 AM · Data Model, Storage manager
vlorentz closed T3753: Store original git manifests, a subtask of T3594: Faithfully store weird git objects, as Resolved.
Feb 8 2022, 11:53 AM · meta-task, Data Model, Storage manager

Feb 7 2022

douardda added a comment to T3890: schema of timezones in the journal.

Looks like we are going to keep the status quo in the short term, ie. a numeric offset for old objects, and offset_bytes for new objects without renaming.

@douardda @olasd agreed?

Feb 7 2022, 9:46 AM · Journal, Data Model

Feb 3 2022

vlorentz added a comment to T3890: schema of timezones in the journal.

Looks like we are going to keep the status quo in the short term, ie. a numeric offset for old objects, and offset_bytes for new objects without renaming.

Feb 3 2022, 5:21 PM · Journal, Data Model

Feb 2 2022

vlorentz added a revision to T3753: Store original git manifests: D7067: git_bare: Use raw_manifest when available.
Feb 2 2022, 6:32 PM · Data Model, Storage manager

Jan 31 2022

vlorentz added a comment to T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.

I agree

Jan 31 2022, 9:24 AM · Roadmap 2020, Data Model, Web app, Roadmap 2021

Jan 29 2022

olasd added a comment to T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.

I've thought of mentioning here a couple of issues that I've seen come up again over the past few days:

Jan 29 2022, 10:16 PM · Roadmap 2020, Data Model, Web app, Roadmap 2021

Jan 28 2022

vlorentz added a comment to T3887: Storing multiple authors in Revisions and Releases.

I suggested Co-Authored-By because it is a de-facto standard in Git now thanks to GitHub, so we already have many revisions using this "format" (no releases as far as I know, though).

Jan 28 2022, 11:51 AM · SWORD deposit, Data Model, BZR loader

Jan 27 2022

zack added a comment to T3887: Storing multiple authors in Revisions and Releases.

Then let's just go for it (insert here ref. to upcoming separate task :-)).

Jan 27 2022, 6:00 PM · SWORD deposit, Data Model, BZR loader
olasd added a comment to T3887: Storing multiple authors in Revisions and Releases.
In T3887#77951, @zack wrote:

Do you foresee any issue in adding extra_headers to releases as well, other than "someone should do it"?

Jan 27 2022, 5:54 PM · SWORD deposit, Data Model, BZR loader
zack added a comment to T3887: Storing multiple authors in Revisions and Releases.
In T3887#77949, @olasd wrote:

Now that I've written it out loud, of course, Releases don't have extra_headers so the package loaders can't make use of this workaround/hack for now.

Jan 27 2022, 5:40 PM · SWORD deposit, Data Model, BZR loader
olasd added a comment to T3887: Storing multiple authors in Revisions and Releases.

Now that I've written it out loud, of course, Releases don't have extra_headers so the package loaders can't make use of this workaround/hack for now.

Jan 27 2022, 5:34 PM · SWORD deposit, Data Model, BZR loader
olasd renamed T3887: Storing multiple authors in Revisions and Releases from Storing multiple authors to Storing multiple authors in Revisions and Releases.
Jan 27 2022, 5:30 PM · SWORD deposit, Data Model, BZR loader
olasd added a comment to T3887: Storing multiple authors in Revisions and Releases.

From merged tasks, this would also be useful for some package loaders, e.g. npm, that support multiple authors in their packaging metadata.

Jan 27 2022, 5:29 PM · SWORD deposit, Data Model, BZR loader
olasd merged task T1645: Associating multiple authors to a revision into T3887: Storing multiple authors in Revisions and Releases.
Jan 27 2022, 5:28 PM · Data Model
olasd merged task T3284: Support for multiple revision authors? into T3887: Storing multiple authors in Revisions and Releases.
Jan 27 2022, 5:28 PM · Data Model
olasd merged tasks T1645: Associating multiple authors to a revision, T3284: Support for multiple revision authors? into T3887: Storing multiple authors in Revisions and Releases.
Jan 27 2022, 5:28 PM · SWORD deposit, Data Model, BZR loader
olasd lowered the priority of T3887: Storing multiple authors in Revisions and Releases from Normal to Wishlist.
Jan 27 2022, 5:27 PM · SWORD deposit, Data Model, BZR loader
olasd added a comment to T3887: Storing multiple authors in Revisions and Releases.

Practically, we could be storing the metadata on additional authors *now* in the extra_headers field (make them a bunch of (b'author', b'XXX <yyy@zzz.ttt>') entries). Of course, that doesn't solve the question of presenting the information.

Jan 27 2022, 5:26 PM · SWORD deposit, Data Model, BZR loader