In T3134#76032, @vlorentz wrote:

wishlist: it would be nice ot be able to check the whole hash of a revision/release even when the author name/email are replaced by a hash. (eg. by making SWHIDv2 a tree hash)

Jan 3 2022, 12:02 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

Dec 17 2021

vlorentz added a comment to T3134: SWHID v2.

wishlist: it would be nice ot be able to check the whole hash of a revision/release even when the author name/email are replaced by a hash. (eg. by making SWHIDv2 a tree hash)

Dec 17 2021, 10:53 AM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

vlorentz removed a project from T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed: meta-task.

Dec 17 2021, 10:51 AM · Roadmap 2020, Data Model, Web app, Roadmap 2021

Dec 8 2021

rdicosmo added a comment to T3134: SWHID v2.

In T3134#75077, @anlambert wrote:

SWHID v2 should also prevent the hash collision issues we are currently facing (some related to the SHAttered attack on SHA1), see T3775 and SWH-LOADER-GIT-JT.

Dec 8 2021, 3:19 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

Dec 7 2021

anlambert added a comment to T3134: SWHID v2.

SWHID v2 should also prevent the hash collision issues we are currently facing (some related to the SHAttered attack on SHA1), see T3775 and SWH-LOADER-GIT-JT.

Dec 7 2021, 4:55 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

Nov 26 2021

vlorentz updated the task description for T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.

Nov 26 2021, 3:16 PM · Roadmap 2020, Data Model, Web app, Roadmap 2021

Oct 14 2021

vlorentz updated the task description for T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.

Oct 14 2021, 12:15 PM · Roadmap 2020, Data Model, Web app, Roadmap 2021

vlorentz removed a subtask for T3134: SWHID v2: T1957: Handling missing DAG nodes.

Oct 14 2021, 12:14 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

vlorentz added a subtask for T3134: SWHID v2: T1957: Handling missing DAG nodes.

Oct 14 2021, 12:13 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

Oct 1 2021

moranegg added a comment to T2344: Build a connector for software deposit via Zenodo/InvenioRDM.

September 2021 update

Oct 1 2021, 12:32 PM · meta-task, Roadmap 2022, Roadmap 2020, SWORD deposit, Scientific Community Building

Sep 23 2021

vlorentz updated the task description for T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.

Sep 23 2021, 5:01 PM · Roadmap 2020, Data Model, Web app, Roadmap 2021

vlorentz triaged T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed as Normal priority.

Sep 23 2021, 5:00 PM · Roadmap 2020, Data Model, Web app, Roadmap 2021

Sep 20 2021

douardda closed T2196: Batch APIs as Wontfix.

not useful as a dedicated task, see T1805 for the main discussion one on this subject

Sep 20 2021, 11:54 AM · Roadmap 2020

Sep 10 2021

moranegg added a comment to T2344: Build a connector for software deposit via Zenodo/InvenioRDM.

Checklist available here:
https://hedgedoc.softwareheritage.org/InevnioRDM-SWH-checklist?both

Sep 10 2021, 5:48 PM · meta-task, Roadmap 2022, Roadmap 2020, SWORD deposit, Scientific Community Building

Sep 6 2021

vlorentz added a revision to T2513: Copy metadata on revisions to the extrinsic metadata storage: D6189: migrate_extrinsic_metadata: Add an option to limit the number of revisions.

Sep 6 2021, 2:46 PM · Metadata workflow, Roadmap 2020

Sep 3 2021

moranegg added a subtask for T2344: Build a connector for software deposit via Zenodo/InvenioRDM: T3549: Analyze deposit-client on InvenioRDM compatibility .

Sep 3 2021, 12:21 PM · meta-task, Roadmap 2022, Roadmap 2020, SWORD deposit, Scientific Community Building

Aug 19 2021

vlorentz removed a parent task for T2513: Copy metadata on revisions to the extrinsic metadata storage: T2202: Collect extrinsic metadata.

Aug 19 2021, 9:20 AM · Metadata workflow, Roadmap 2020

Jul 30 2021

vlorentz added a project to T2193: Add provenance feature: Provenance database.

Jul 30 2021, 10:15 AM · Provenance database, Roadmap 2020

Jul 15 2021

vlorentz closed T2938: Create API endpoint to access raw_extrinsic_metadata, a subtask of T2088: Specify and draw metadata view on web-app, as Resolved.

Jul 15 2021, 12:18 PM · Roadmap 2020, Web app, Metadata workflow

Jun 29 2021

moranegg changed the status of T2234: Write use case-specific documentation, a subtask of T2233: Update documentation to current project status, from Open to Work in Progress.

Jun 29 2021, 10:53 AM · Documentation, meta-task, Roadmap 2020

Jun 2 2021

moranegg added a comment to T2344: Build a connector for software deposit via Zenodo/InvenioRDM.

May 2021 update

Jun 2 2021, 3:25 PM · meta-task, Roadmap 2022, Roadmap 2020, SWORD deposit, Scientific Community Building

May 8 2021

zack closed T2210: Data Model as Invalid.

Closing this as it was a vague meta-task from 2020 roadmap (but we'll keep the actual sub-tasks, which were more clearly identified and are still relevant).

May 8 2021, 11:37 AM · Data Model, Roadmap 2020

Apr 27 2021

vlorentz moved T2540: support the loading of metadata-only deposits in the metadata storage from Backlog to Deployed on the SWORD deposit board.

Apr 27 2021, 2:07 PM · Roadmap 2020, SWORD deposit, Scientific Community Building

Apr 23 2021

vlorentz assigned T2225: Migrate to GitLab to olasd.

Apr 23 2021, 4:53 PM · meta-task, Roadmap 2022, GitLab migration, Roadmap 2020

vlorentz assigned T3134: SWHID v2 to zack.

Apr 23 2021, 4:50 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

Apr 19 2021

anlambert closed T2219: Authentication / authorization, a subtask of T2217: Plumbings, as Resolved.

Apr 19 2021, 11:42 AM · meta-task, Roadmap 2020

anlambert closed T2219: Authentication / authorization as Resolved.

Closing this as resolved as swh-web and swh-deposit now use Keycloak based authentication.

Apr 19 2021, 11:42 AM · Roadmap 2020

anlambert closed T2458: Write wiki page about users management with Keycloak as Resolved.

Documentation about Keycloak administration is now available in the swh documentation.

Apr 19 2021, 11:39 AM · System administration, Roadmap 2020

anlambert closed T2458: Write wiki page about users management with Keycloak, a subtask of T2219: Authentication / authorization, as Resolved.

Apr 19 2021, 11:39 AM · Roadmap 2020

Apr 8 2021

vlorentz closed T2540: support the loading of metadata-only deposits in the metadata storage, a subtask of T2537: Extend new deposit endpoint to support metadata-only deposits, as Resolved.

Apr 8 2021, 10:57 AM · Roadmap 2020, SWORD deposit, Scientific Community Building

vlorentz closed T2540: support the loading of metadata-only deposits in the metadata storage as Resolved.

Apr 8 2021, 10:57 AM · Roadmap 2020, SWORD deposit, Scientific Community Building

vlorentz closed T2625: create and publish xml schema for the specific swh-deposit metadata, a subtask of T2540: support the loading of metadata-only deposits in the metadata storage, as Resolved.

Apr 8 2021, 10:53 AM · Roadmap 2020, SWORD deposit, Scientific Community Building

Apr 6 2021

olasd added a comment to T2513: Copy metadata on revisions to the extrinsic metadata storage.

2021-04-06 20:54:44,962 __main__     ERROR    Could not parse revision metadata 00c6e2fe046dee3b5ef629f74f4801345840e70a
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/swh/storage/migrate_extrinsic_metadata.py", line 1161, in main
    handle_row(row, storage, deposit_cur, dry_run)
  File "/usr/lib/python3/dist-packages/swh/storage/migrate_extrinsic_metadata.py", line 843, in handle_row
    assert "id" in actual_metadata or "title" in actual_metadata
AssertionError

Apr 6 2021, 11:12 PM · Metadata workflow, Roadmap 2020

olasd added a comment to T2513: Copy metadata on revisions to the extrinsic metadata storage.

2021-04-06 20:19:19,898 __main__     ERROR    Could not parse revision metadata 00959a167bd98452c98ce73382f4b42179d53d32
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/swh/storage/migrate_extrinsic_metadata.py", line 1161, in main
    handle_row(row, storage, deposit_cur, dry_run)
  File "/usr/lib/python3/dist-packages/swh/storage/migrate_extrinsic_metadata.py", line 979, in handle_row
    storage, row["id"], metadata["original_artifact"][0]["filename"]
  File "/usr/lib/python3/dist-packages/swh/storage/migrate_extrinsic_metadata.py", line 265, in pypi_origin_from_filename
    project_name = pypi_project_from_filename(filename)
  File "/usr/lib/python3/dist-packages/swh/storage/migrate_extrinsic_metadata.py", line 256, in pypi_project_from_filename
    assert match, original_filename
AssertionError: pypops-201408-r4.tar.gz

Apr 6 2021, 10:21 PM · Metadata workflow, Roadmap 2020

olasd added a comment to T2513: Copy metadata on revisions to the extrinsic metadata storage.

(I've also noticed dry_run was = True, so I fixed that as well :P)

Apr 6 2021, 8:12 PM · Metadata workflow, Roadmap 2020

olasd added a comment to T2513: Copy metadata on revisions to the extrinsic metadata storage.

Tail of log:

Apr 6 2021, 7:57 PM · Metadata workflow, Roadmap 2020

olasd claimed T2513: Copy metadata on revisions to the extrinsic metadata storage.

The script is now running on getty.

Apr 6 2021, 7:07 PM · Metadata workflow, Roadmap 2020

vlorentz added a comment to T2513: Copy metadata on revisions to the extrinsic metadata storage.

https://forge.softwareheritage.org/source/swh-storage/browse/master/swh/storage/migrate_extrinsic_metadata.py

Apr 6 2021, 5:14 PM · Metadata workflow, Roadmap 2020

vlorentz merged T3185: Migrate extrinsic metadata from 'revision' to 'raw_extrinsic_metadata' tables into T2513: Copy metadata on revisions to the extrinsic metadata storage.

Apr 6 2021, 5:14 PM · Metadata workflow, Roadmap 2020

Apr 5 2021

rdicosmo added a parent task for T2540: support the loading of metadata-only deposits in the metadata storage: T3128: Improve deposit integration, management and display.

Apr 5 2021, 12:13 PM · Roadmap 2020, SWORD deposit, Scientific Community Building

rdicosmo added a parent task for T2344: Build a connector for software deposit via Zenodo/InvenioRDM: T3128: Improve deposit integration, management and display.

Apr 5 2021, 12:13 PM · meta-task, Roadmap 2022, Roadmap 2020, SWORD deposit, Scientific Community Building

Mar 30 2021

ardumont added a revision to T2235: Clean existing documentation: D5381: docs: Drop unneeded technical specification.

Mar 30 2021, 11:41 AM · Documentation, Roadmap 2020

Mar 23 2021

moranegg edited projects for T2203: Intrinsic metadata, added: Intrinsic metadata; removed Metadata workflow.

Mar 23 2021, 5:32 PM · Intrinsic metadata, Roadmap 2020

vlorentz closed T2703: Use intrinsic identifiers/hashes for RawExtrinsicMetadata objects, a subtask of T2513: Copy metadata on revisions to the extrinsic metadata storage, as Resolved.

Mar 23 2021, 2:33 PM · Metadata workflow, Roadmap 2020

Mar 19 2021

vlorentz added a subtask for T2210: Data Model: T3134: SWHID v2.

Mar 19 2021, 4:23 PM · Data Model, Roadmap 2020

vlorentz added a parent task for T3134: SWHID v2: T2210: Data Model.

Mar 19 2021, 4:23 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

vlorentz triaged T3134: SWHID v2 as Normal priority.

Mar 19 2021, 4:22 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

vlorentz added a project to T3134: SWHID v2: Roadmap 2020.

Mar 19 2021, 4:21 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021

vlorentz merged task T2212: Specification for swh:2+: identifiers into T3134: SWHID v2.

Mar 19 2021, 4:21 PM · Data Model, Roadmap 2020

Mar 16 2021

vlorentz closed T2779: Put information (client, collection and deposit-id) inside metadata for metadata-only deposit, a subtask of T2540: support the loading of metadata-only deposits in the metadata storage, as Resolved.

Mar 16 2021, 10:53 AM · Roadmap 2020, SWORD deposit, Scientific Community Building

Mar 15 2021

vlorentz closed T2942: Update deposit: `swhid` MUST exist in archive for a metadata-only deposit, a subtask of T2540: support the loading of metadata-only deposits in the metadata storage, as Resolved.

Mar 15 2021, 11:59 AM · Roadmap 2020, SWORD deposit, Scientific Community Building

Mar 11 2021

rdicosmo added a parent task for T2234: Write use case-specific documentation: T3118: Documentation for users and ambassadors.

Mar 11 2021, 8:21 PM · Roadmap 2021, meta-task, Documentation

rdicosmo merged task T2209: At least 2 full mirrors up and running into T3116: Roll out at least one operational mirror.

Mar 11 2021, 7:57 PM · Mirror, Roadmap 2020

Mar 10 2021

rdicosmo renamed T2214: Scale-out graph and database storage in production from Scale-out storage (graph) in production to Scale-out graph and database storage in production.

Mar 10 2021, 4:36 PM · meta-task, Roadmap 2022, Roadmap 2021, Storage manager

rdicosmo renamed T2214: Scale-out graph and database storage in production from Scale-out storage in production to Scale-out storage (graph) in production.

Mar 10 2021, 4:35 PM · meta-task, Roadmap 2022, Roadmap 2021, Storage manager

rdicosmo added a subtask for T2193: Add provenance feature: T3112: Provenance index for the full archive.

Mar 10 2021, 4:20 PM · Provenance database, Roadmap 2020