Page MenuHomeSoftware Heritage

model: Add 'id' field to RawExtrinsicMetadata
ClosedPublic

Authored by vlorentz on Jan 29 2021, 3:09 PM.

Details

Summary

So that they can be properly deduplicated and referenced.

Depends on D4935.

Diff Detail

Event Timeline

Build is green

Patch application report for D4970 (id=17747)

Could not rebase; Attempt merge onto cad940dc8c...

Updating cad940d..0bcff7b
Fast-forward
 swh/model/hashutil.py               |   9 ++-
 swh/model/identifiers.py            |  79 ++++++++++++++++++++++++
 swh/model/model.py                  |   8 ++-
 swh/model/tests/test_identifiers.py | 117 ++++++++++++++++++++++++++++++++++++
 swh/model/tests/test_model.py       |   6 +-
 5 files changed, 216 insertions(+), 3 deletions(-)
Changes applied before test
commit 0bcff7b00bd30a7ee7af408bf00229c19785da55
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 29 15:08:49 2021 +0100

    model: Add 'id' field to RawExtrinsicMetadata
    
    So that they can be properly deduplicated and referenced.

commit 2a807789f22fdbf3838684b6b1ce8cf5a599b754
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Jan 25 12:31:12 2021 +0100

    identifiers: Add raw_extrinsic_metadata_identifier
    
    This will be used to compute an intrisic identifier for RawExtrinsicMetadata;
    which can be used for deduplication and refering to it like any other sha1_git
    instead of needed to use a tuple of its fields.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/206/ for more details.

ardumont added a subscriber: ardumont.

lgtm

What's the impact for the production (migration script, code change, ...)?

Can you please reference the task this work is related to? T2703 i guess?

This revision is now accepted and ready to land.Feb 2 2021, 2:00 PM

Build is green

Patch application report for D4970 (id=17867)

Rebasing onto 0c16581283...

Current branch diff-target is up to date.
Changes applied before test
commit 266b88dcaaa0cab48c67e62ebca51f0a4599c435
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 29 15:08:49 2021 +0100

    model: Add 'id' field to RawExtrinsicMetadata
    
    So that they can be properly deduplicated and referenced.

commit 272468f3b5a96c8854a26efe333c32cba4504aff
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Jan 25 12:31:12 2021 +0100

    identifiers: Add raw_extrinsic_metadata_identifier
    
    This will be used to compute an intrisic identifier for RawExtrinsicMetadata;
    which can be used for deduplication and refering to it like any other sha1_git
    instead of needed to use a tuple of its fields.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/211/ for more details.

Build is green

Patch application report for D4970 (id=17973)

Could not rebase; Attempt merge onto 0c16581283...

Updating 0c16581..266b88d
Fast-forward
 swh/model/hashutil.py               |   9 ++-
 swh/model/identifiers.py            |  79 ++++++++++++++++++++++++
 swh/model/model.py                  |   8 ++-
 swh/model/tests/test_identifiers.py | 117 ++++++++++++++++++++++++++++++++++++
 swh/model/tests/test_model.py       |   6 +-
 5 files changed, 216 insertions(+), 3 deletions(-)
Changes applied before test
commit 266b88dcaaa0cab48c67e62ebca51f0a4599c435
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 29 15:08:49 2021 +0100

    model: Add 'id' field to RawExtrinsicMetadata
    
    So that they can be properly deduplicated and referenced.

commit 272468f3b5a96c8854a26efe333c32cba4504aff
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Jan 25 12:31:12 2021 +0100

    identifiers: Add raw_extrinsic_metadata_identifier
    
    This will be used to compute an intrisic identifier for RawExtrinsicMetadata;
    which can be used for deduplication and refering to it like any other sha1_git
    instead of needed to use a tuple of its fields.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/218/ for more details.

big rebase, plz review again

Build is green

Patch application report for D4970 (id=18553)

Could not rebase; Attempt merge onto 8e0119962b...

Updating 8e01199..8cbe59e
Fast-forward
 swh/model/cli.py                    |  92 +++++---
 swh/model/hashutil.py               |   9 +-
 swh/model/identifiers.py            | 263 +++++++---------------
 swh/model/model.py                  |   8 +-
 swh/model/tests/test_cli.py         |   6 +-
 swh/model/tests/test_identifiers.py | 432 ++++++++++--------------------------
 swh/model/tests/test_model.py       |   7 +-
 7 files changed, 271 insertions(+), 546 deletions(-)
Changes applied before test
commit 8cbe59ee6251ac834a3d98b04146d2dfeb8da075
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 29 15:08:49 2021 +0100

    model: Add 'id' field to RawExtrinsicMetadata
    
    So that they can be properly deduplicated and referenced.

commit d88a5e13f2ffea5c0ebfad24e29ba41e86af20c0
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Jan 25 12:31:12 2021 +0100

    identifiers: Add raw_extrinsic_metadata_identifier
    
    This will be used to compute an intrisic identifier for RawExtrinsicMetadata;
    which can be used for deduplication and refering to it like any other sha1_git
    instead of needed to use a tuple of its fields.

commit bf4ab4336f7b43d442988c47d3dd70bb82b595c5
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 3 10:44:48 2021 +0100

    identifiers: Remove the deprecated SWHID class
    
    Other packages don't use it anymore.

commit 1e924e84198a895003d6f649b8e3471cd93a7c7b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 3 10:44:27 2021 +0100

    cli: stop using the deprecated SWHID class

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/274/ for more details.

This revision is now accepted and ready to land.Mar 4 2021, 11:06 AM

rebase, fix format of discovery_date

Build is green

Patch application report for D4970 (id=18587)

Could not rebase; Attempt merge onto bf4ab4336f...

Updating bf4ab43..fc808e1
Fast-forward
 swh/model/hashutil.py               |   9 ++-
 swh/model/identifiers.py            |  79 +++++++++++++++++++++++++
 swh/model/model.py                  |   8 ++-
 swh/model/tests/test_identifiers.py | 115 ++++++++++++++++++++++++++++++++++++
 swh/model/tests/test_model.py       |   7 ++-
 5 files changed, 215 insertions(+), 3 deletions(-)
Changes applied before test
commit fc808e1fc9e59ffae1e82ea483d529025d7d1436
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 29 15:08:49 2021 +0100

    model: Add 'id' field to RawExtrinsicMetadata
    
    So that they can be properly deduplicated and referenced.

commit f6eab95253f13f28fe4d4652fc471e3e8a0b5565
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Jan 25 12:31:12 2021 +0100

    identifiers: Add raw_extrinsic_metadata_identifier
    
    This will be used to compute an intrisic identifier for RawExtrinsicMetadata;
    which can be used for deduplication and refering to it like any other sha1_git
    instead of needed to use a tuple of its fields.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/278/ for more details.