Page MenuHomeSoftware Heritage
Feed Advanced Search

Feb 7 2022

douardda added a comment to T3890: schema of timezones in the journal.

Looks like we are going to keep the status quo in the short term, ie. a numeric offset for old objects, and offset_bytes for new objects without renaming.

@douardda @olasd agreed?

Feb 7 2022, 9:46 AM · Journal, Data Model

Feb 3 2022

vlorentz added a comment to T3890: schema of timezones in the journal.

Looks like we are going to keep the status quo in the short term, ie. a numeric offset for old objects, and offset_bytes for new objects without renaming.

Feb 3 2022, 5:21 PM · Journal, Data Model

Feb 2 2022

vlorentz added a revision to T3753: Store original git manifests: D7067: git_bare: Use raw_manifest when available.
Feb 2 2022, 6:32 PM · Data Model, Storage manager

Jan 31 2022

vlorentz added a comment to T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.

I agree

Jan 31 2022, 9:24 AM · Roadmap 2020, Data Model, Web app, Roadmap 2021

Jan 29 2022

olasd added a comment to T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.

I've thought of mentioning here a couple of issues that I've seen come up again over the past few days:

Jan 29 2022, 10:16 PM · Roadmap 2020, Data Model, Web app, Roadmap 2021

Jan 28 2022

vlorentz added a comment to T3887: Storing multiple authors in Revisions and Releases.

I suggested Co-Authored-By because it is a de-facto standard in Git now thanks to GitHub, so we already have many revisions using this "format" (no releases as far as I know, though).

Jan 28 2022, 11:51 AM · SWORD deposit, Data Model, BZR loader

Jan 27 2022

zack added a comment to T3887: Storing multiple authors in Revisions and Releases.

Then let's just go for it (insert here ref. to upcoming separate task :-)).

Jan 27 2022, 6:00 PM · SWORD deposit, Data Model, BZR loader
olasd added a comment to T3887: Storing multiple authors in Revisions and Releases.
In T3887#77951, @zack wrote:

Do you foresee any issue in adding extra_headers to releases as well, other than "someone should do it"?

Jan 27 2022, 5:54 PM · SWORD deposit, Data Model, BZR loader
zack added a comment to T3887: Storing multiple authors in Revisions and Releases.
In T3887#77949, @olasd wrote:

Now that I've written it out loud, of course, Releases don't have extra_headers so the package loaders can't make use of this workaround/hack for now.

Jan 27 2022, 5:40 PM · SWORD deposit, Data Model, BZR loader
olasd added a comment to T3887: Storing multiple authors in Revisions and Releases.

Now that I've written it out loud, of course, Releases don't have extra_headers so the package loaders can't make use of this workaround/hack for now.

Jan 27 2022, 5:34 PM · SWORD deposit, Data Model, BZR loader
olasd renamed T3887: Storing multiple authors in Revisions and Releases from Storing multiple authors to Storing multiple authors in Revisions and Releases.
Jan 27 2022, 5:30 PM · SWORD deposit, Data Model, BZR loader
olasd added a comment to T3887: Storing multiple authors in Revisions and Releases.

From merged tasks, this would also be useful for some package loaders, e.g. npm, that support multiple authors in their packaging metadata.

Jan 27 2022, 5:29 PM · SWORD deposit, Data Model, BZR loader
olasd merged task T1645: Associating multiple authors to a revision into T3887: Storing multiple authors in Revisions and Releases.
Jan 27 2022, 5:28 PM · Data Model
olasd merged task T3284: Support for multiple revision authors? into T3887: Storing multiple authors in Revisions and Releases.
Jan 27 2022, 5:28 PM · Data Model
olasd merged tasks T1645: Associating multiple authors to a revision, T3284: Support for multiple revision authors? into T3887: Storing multiple authors in Revisions and Releases.
Jan 27 2022, 5:28 PM · SWORD deposit, Data Model, BZR loader
olasd lowered the priority of T3887: Storing multiple authors in Revisions and Releases from Normal to Wishlist.
Jan 27 2022, 5:27 PM · SWORD deposit, Data Model, BZR loader
olasd added a comment to T3887: Storing multiple authors in Revisions and Releases.

Practically, we could be storing the metadata on additional authors *now* in the extra_headers field (make them a bunch of (b'author', b'XXX <yyy@zzz.ttt>') entries). Of course, that doesn't solve the question of presenting the information.

Jan 27 2022, 5:26 PM · SWORD deposit, Data Model, BZR loader
Alphare added a revision to T3887: Storing multiple authors in Revisions and Releases: D6344: Non-incremental, non-delta Bazaar/Breezy loader.
Jan 27 2022, 4:06 PM · SWORD deposit, Data Model, BZR loader
douardda renamed T3893: Version swh schema from Make the swh schema versioned to Version swh schema .
Jan 27 2022, 2:18 PM · Data Model
douardda updated the task description for T3893: Version swh schema .
Jan 27 2022, 2:16 PM · Data Model
vlorentz added a project to T3893: Version swh schema : Data Model.
Jan 27 2022, 2:13 PM · Data Model
vlorentz updated the task description for T3890: schema of timezones in the journal.
Jan 27 2022, 11:28 AM · Journal, Data Model
vlorentz claimed T3890: schema of timezones in the journal.
Jan 27 2022, 10:36 AM · Journal, Data Model

Jan 26 2022

vlorentz triaged T3890: schema of timezones in the journal as Normal priority.
Jan 26 2022, 11:57 AM · Journal, Data Model

Jan 25 2022

vlorentz added projects to T3887: Storing multiple authors in Revisions and Releases: Data Model, SWORD deposit.
Jan 25 2022, 3:08 PM · SWORD deposit, Data Model, BZR loader
vlorentz updated the task description for T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:35 PM · BZR loader, Data Model
vlorentz updated the task description for T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:35 PM · BZR loader, Data Model
vlorentz updated the task description for T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:31 PM · BZR loader, Data Model
vlorentz updated the task description for T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:29 PM · BZR loader, Data Model
vlorentz updated subscribers of T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:29 PM · BZR loader, Data Model
vlorentz triaged T3886: Representing timezones with non-integer number of minutes as Normal priority.
Jan 25 2022, 1:28 PM · BZR loader, Data Model

Jan 21 2022

vlorentz added revisions to T3752: Store/represent time offsets as strings: D7008: Stop using the deprecated 'TimestampWithTimezone.offset' attribute, D7007: Stop using the deprecated 'TimestampWithTimezone.offset' attribute, D7006: Stop using the deprecated 'TimestampWithTimezone.offset' attribute, D7005: Add method 'TimestampWithTimezone.offset_minutes', D7003: journal: Document the new format for gitdate..
Jan 21 2022, 1:29 PM · Data Model, Storage manager

Jan 19 2022

olasd closed T3819: Deploy swh.model 4.1.0 / swh.storage 0.41.0 to production, a subtask of T3752: Store/represent time offsets as strings, as Resolved.
Jan 19 2022, 7:12 PM · Data Model, Storage manager

Jan 18 2022

olasd merged task T2449: Consider switching timestamp offset storage to strings/byte arrays into T3752: Store/represent time offsets as strings.
Jan 18 2022, 12:27 PM · Storage manager, Data Model
olasd merged T2449: Consider switching timestamp offset storage to strings/byte arrays into T3752: Store/represent time offsets as strings.
Jan 18 2022, 12:27 PM · Data Model, Storage manager
olasd added a subtask for T3752: Store/represent time offsets as strings: T3819: Deploy swh.model 4.1.0 / swh.storage 0.41.0 to production.
Jan 18 2022, 12:26 PM · Data Model, Storage manager
anlambert updated the task description for T3858: Add diff features for class from_disk.Directory.
Jan 18 2022, 12:05 PM · Data Model
anlambert triaged T3858: Add diff features for class from_disk.Directory as Normal priority.
Jan 18 2022, 12:03 PM · Data Model

Jan 13 2022

vlorentz added revisions to T3752: Store/represent time offsets as strings: D6940: tests: Use 'offset_bytes' instead of 'offset'/'negative_utc', D6939: Stop passing 'offset' and 'negative_utc' to TimestampWithTimezone(), D6938: tests: Replace 'offset' and 'negative_utc' with 'offset_bytes', D6937: Remove 'offset' and 'negative_utc', D6935: deposit: Remove 'negative_utc' from test data.
Jan 13 2022, 12:26 PM · Data Model, Storage manager
vlorentz added a revision to T3752: Store/represent time offsets as strings: D6936: TimestampWithTimezone: Make 'offset' and 'negative_utc' optional.
Jan 13 2022, 12:16 PM · Data Model, Storage manager
vlorentz added a revision to T3752: Store/represent time offsets as strings: D6929: Remove 'negative_utc'..
Jan 13 2022, 11:28 AM · Data Model, Storage manager
ardumont closed T3837: clean up swh-scheduler model db as Resolved.
Jan 13 2022, 11:25 AM · Scheduling utilities, Data Model, System administration
ardumont moved T3837: clean up swh-scheduler model db from deployed/landed/monitoring to done on the System administration board.
Jan 13 2022, 11:24 AM · Scheduling utilities, Data Model, System administration

Jan 12 2022

vlorentz added revisions to T3752: Store/represent time offsets as strings: D6927: Remove special handling of negative_utc, D6923: converters: Write raw_manifest of Directory objects, D6894: converters: Write object_bytes and raw_manifest on revisions and releases.
Jan 12 2022, 3:24 PM · Data Model, Storage manager
ardumont moved T3837: clean up swh-scheduler model db from in-progress to deployed/landed/monitoring on the System administration board.
Jan 12 2022, 3:07 PM · Scheduling utilities, Data Model, System administration
ardumont added a comment to T3837: clean up swh-scheduler model db.

And same goes for production [1].

Jan 12 2022, 3:07 PM · Scheduling utilities, Data Model, System administration
ardumont added a comment to T3837: clean up swh-scheduler model db.

Triggered the migration (using a transaction as the dataset is not that huge) and all
went well.

Jan 12 2022, 12:13 PM · Scheduling utilities, Data Model, System administration
ardumont added a comment to T3837: clean up swh-scheduler model db.

So script is ok and landed.
I'll trigger it on staging first.

Jan 12 2022, 11:42 AM · Scheduling utilities, Data Model, System administration
ardumont added a revision to T3837: clean up swh-scheduler model db: D6921: sql: Clean up task/task_run data model.
Jan 12 2022, 11:04 AM · Scheduling utilities, Data Model, System administration

Jan 11 2022

vlorentz added a revision to T3752: Store/represent time offsets as strings: D6915: tests: Use TimestampWithTimezone.from_datetime() instead of the constructor.
Jan 11 2022, 3:25 PM · Data Model, Storage manager
vlorentz added revisions to T3752: Store/represent time offsets as strings: D6911: Remove strdate_to_timestamp, D6913: tests: Use TimestampWithTimezone.from_datetime() instead of the constructor, D6910: tests: Use TimestampWithTimezone.from_datetime() instead of the constructor, D6909: tests: Use TimestampWithTimezone.from_datetime() instead of the constructor, D6908: tests: Use TimestampWithTimezone.from_datetime() instead of the constructor.
Jan 11 2022, 2:15 PM · Data Model, Storage manager
ardumont changed the status of T3837: clean up swh-scheduler model db from Open to Work in Progress.
Jan 11 2022, 9:58 AM · Scheduling utilities, Data Model, System administration

Jan 10 2022

ardumont updated the task description for T3837: clean up swh-scheduler model db.
Jan 10 2022, 4:15 PM · Scheduling utilities, Data Model, System administration
ardumont added a comment to T3837: clean up swh-scheduler model db.

wip script [1]

Jan 10 2022, 3:53 PM · Scheduling utilities, Data Model, System administration
ardumont triaged T3837: clean up swh-scheduler model db as Normal priority.
Jan 10 2022, 10:48 AM · Scheduling utilities, Data Model, System administration

Jan 7 2022

ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

I'm growing fond of this idea.
That should take less time to refactor it now that we improved the lister scaffolding and
that we mostly know what the perimeter of the nixguix loader is.

Jan 7 2022, 2:57 PM · Data Model, Nixguix loader
vlorentz added revisions to T3752: Store/represent time offsets as strings: D6848: Add columns {,committer_}date_offset to rev/rel and raw_manifest to dir/rev/rel, D6890: git_objects: Use raw offset_bytes to format dates, and remove format_offset().
Jan 7 2022, 1:54 PM · Data Model, Storage manager
vlorentz added revisions to T3753: Store original git manifests: D6801: model: Add a raw_manifest attribute, D6811: model: Exclude 'raw_manifest' from dictionaries when it is null, D6847: hypothesis_strategies: Generate raw_manifest, D6848: Add columns {,committer_}date_offset to rev/rel and raw_manifest to dir/rev/rel.
Jan 7 2022, 1:54 PM · Data Model, Storage manager

Jan 3 2022

douardda added a comment to T3134: SWHID v2.

wishlist: it would be nice ot be able to check the whole hash of a revision/release even when the author name/email are replaced by a hash. (eg. by making SWHIDv2 a tree hash)

Jan 3 2022, 12:02 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

Dec 21 2021

vlorentz triaged T3816: Make DirectoryEntry validate `perms` is consistent with the entry type as Normal priority.
Dec 21 2021, 2:36 PM · Data Model

Dec 17 2021

vlorentz added a comment to T3134: SWHID v2.

wishlist: it would be nice ot be able to check the whole hash of a revision/release even when the author name/email are replaced by a hash. (eg. by making SWHIDv2 a tree hash)

Dec 17 2021, 10:53 AM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021
vlorentz removed a project from T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed: meta-task.
Dec 17 2021, 10:51 AM · Roadmap 2020, Data Model, Web app, Roadmap 2021

Dec 13 2021

ardumont moved T3722: staging: Deploy package loader v1.0, deposit server v0.16, lister v2.3 from Backlog to Deployed on the SWORD deposit board.
Dec 13 2021, 11:27 AM · System administration, SWORD deposit, Package Loader, Data Model, Archive content

Dec 10 2021

ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

Maybe another data point for the discussion is that the nixguix loader currently only
shows 1 origin for guix and 1 for nixos [well nixpkgs really} in the coverage part [1].
Which is somewhat true... but... feels weird at the same time.

Dec 10 2021, 9:32 AM · Data Model, Nixguix loader

Dec 9 2021

ardumont added a comment to T3781: Replace the Nixguix loader with a lister.

Not saying no.

Dec 9 2021, 4:17 PM · Data Model, Nixguix loader

Dec 8 2021

vlorentz triaged T3781: Replace the Nixguix loader with a lister as Normal priority.
Dec 8 2021, 6:45 PM · Data Model, Nixguix loader
rdicosmo added a comment to T3134: SWHID v2.

SWHID v2 should also prevent the hash collision issues we are currently facing (some related to the SHAttered attack on SHA1), see T3775 and SWH-LOADER-GIT-JT.

Dec 8 2021, 3:19 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

Dec 7 2021

anlambert added a comment to T3134: SWHID v2.

SWHID v2 should also prevent the hash collision issues we are currently facing (some related to the SHAttered attack on SHA1), see T3775 and SWH-LOADER-GIT-JT.

Dec 7 2021, 4:55 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021
vlorentz added a revision to T3752: Store/represent time offsets as strings: D6776: Add attribute TimestampWithTimezone.offset_bytes, to store raw Git offsets.
Dec 7 2021, 4:51 PM · Data Model, Storage manager

Dec 2 2021

vlorentz updated the task description for T3752: Store/represent time offsets as strings.
Dec 2 2021, 4:04 PM · Data Model, Storage manager
vlorentz closed T3586: Figure out what to do with 'misordered' directories in Cassandra as Resolved.

We don't care anymore, this will be handled by T3753.

Dec 2 2021, 3:14 PM · Data Model, Storage manager
vlorentz removed a parent task for T3752: Store/represent time offsets as strings: T3753: Store original git manifests.
Dec 2 2021, 3:01 PM · Data Model, Storage manager
vlorentz removed a subtask for T3753: Store original git manifests: T3752: Store/represent time offsets as strings.
Dec 2 2021, 3:01 PM · Data Model, Storage manager
vlorentz added a parent task for T3752: Store/represent time offsets as strings: T3753: Store original git manifests.
Dec 2 2021, 3:00 PM · Data Model, Storage manager
vlorentz added a subtask for T3753: Store original git manifests: T3752: Store/represent time offsets as strings.
Dec 2 2021, 3:00 PM · Data Model, Storage manager
vlorentz updated the task description for T3752: Store/represent time offsets as strings.
Dec 2 2021, 2:59 PM · Data Model, Storage manager
vlorentz updated the task description for T3752: Store/represent time offsets as strings.
Dec 2 2021, 2:55 PM · Data Model, Storage manager
vlorentz updated the task description for T3752: Store/represent time offsets as strings.
Dec 2 2021, 2:52 PM · Data Model, Storage manager
vlorentz updated the task description for T3753: Store original git manifests.
Dec 2 2021, 2:48 PM · Data Model, Storage manager
vlorentz updated the task description for T3752: Store/represent time offsets as strings.
Dec 2 2021, 2:22 PM · Data Model, Storage manager

Nov 26 2021

vlorentz removed a project from T3752: Store/represent time offsets as strings: meta-task.
Nov 26 2021, 5:19 PM · Data Model, Storage manager
vlorentz removed a project from T3753: Store original git manifests: meta-task.
Nov 26 2021, 5:19 PM · Data Model, Storage manager
vlorentz claimed T3594: Faithfully store weird git objects.
Nov 26 2021, 4:43 PM · meta-task, Data Model, Storage manager
vlorentz claimed T3753: Store original git manifests.
Nov 26 2021, 4:43 PM · Data Model, Storage manager
vlorentz triaged T3753: Store original git manifests as Normal priority.
Nov 26 2021, 4:43 PM · Data Model, Storage manager
vlorentz triaged T3752: Store/represent time offsets as strings as Normal priority.
Nov 26 2021, 4:42 PM · Data Model, Storage manager
vlorentz closed T3598: Support revisions with "extra headers" not at the end, a subtask of T3594: Faithfully store weird git objects, as Wontfix.
Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
vlorentz closed T3598: Support revisions with "extra headers" not at the end as Wontfix.

We decided to store manifests instead. T3594#74385

Nov 26 2021, 4:41 PM · Data Model, Storage manager
vlorentz closed T3596: Support "weird" permissions in directories as Wontfix.

We decided to store manifests instead. T3594#74385

Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
vlorentz closed T3596: Support "weird" permissions in directories, a subtask of T3594: Faithfully store weird git objects, as Wontfix.
Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
vlorentz closed T3595: Support disordered directory entries in git, a subtask of T3594: Faithfully store weird git objects, as Wontfix.
Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
vlorentz closed T3595: Support disordered directory entries in git as Wontfix.

We decided to store manifests instead. T3594#74385

Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
vlorentz added a comment to T3594: Faithfully store weird git objects.

Copy of an email I sent today:

Nov 26 2021, 4:40 PM · meta-task, Data Model, Storage manager
vlorentz added a comment to T3638: Make package loaders create releases objects instead of revisions.

Copy of an email I sent on 2021-11-17:

Nov 26 2021, 4:02 PM · Package Loader, Data Model, Archive content
vlorentz updated the task description for T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.
Nov 26 2021, 3:16 PM · Roadmap 2020, Data Model, Web app, Roadmap 2021

Nov 22 2021

vlorentz closed T3636: Make the opam loader write extrinsic metadata, a subtask of T3638: Make package loaders create releases objects instead of revisions, as Resolved.
Nov 22 2021, 2:44 PM · Package Loader, Data Model, Archive content
vlorentz closed T3638: Make package loaders create releases objects instead of revisions as Resolved.
Nov 22 2021, 2:43 PM · Package Loader, Data Model, Archive content
ardumont closed T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3, a subtask of T3638: Make package loaders create releases objects instead of revisions, as Resolved.
Nov 22 2021, 2:29 PM · Package Loader, Data Model, Archive content
ardumont closed T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3 as Resolved.
Nov 22 2021, 2:29 PM · System administration, Package Loader, Data Model, Archive content
ardumont moved T3745: production: Deploy package loader v1.1, deposit server v0.16, lister v2.3 from deployed/landed/monitoring to Component upgrades on the System administration board.
Nov 22 2021, 2:29 PM · System administration, Package Loader, Data Model, Archive content