Page MenuHomeSoftware Heritage
Differential D3417

Add a new ImmutableDict class, and use it in model objects.
ClosedPublic
Actions

Authored by vlorentz on Jul 3 2020, 5:08 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Commits
rDMODaf0dd1aaee49: Add a new ImmutableDict class, and use it in model objects.
Summary

So they are truly immutable now.

Diff Detail

Repository
rDMOD Data model
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Jul 3 2020, 5:08 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptJul 3 2020, 5:08 PM
swh-public-ci added a comment.Jul 3 2020, 5:10 PM

Build is green

Patch application report for D3417 (id=12114)

Could not rebase; Attempt merge onto 1ff05161e7...

Merge made by the 'recursive' strategy.
 swh/model/collections.py            |  36 ++++
 swh/model/model.py                  | 263 ++++++++++++++++++++++-
 swh/model/tests/test_collections.py |  42 ++++
 swh/model/tests/test_model.py       | 409 ++++++++++++++++++++++++++++++++++++
 4 files changed, 740 insertions(+), 10 deletions(-)
 create mode 100644 swh/model/collections.py
 create mode 100644 swh/model/tests/test_collections.py
Changes applied before test
commit e3fb916401226ce8f46c959894c37ff27a95d76d
Merge: 1ff0516 5b7ac4f
Author: Jenkins user <jenkins@localhost>
Date:   Fri Jul 3 15:08:49 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 5b7ac4fdf7c465def8303f2184ae9e0097983eca
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 3 17:08:19 2020 +0200

    Add a new ImmutableDict class, and use it in model objects.
    
    So they are truly immutable now.

commit 5e3ae76689895f5bccb62cccca813f3810231a85
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Jul 2 17:53:13 2020 +0200

    Add raw metadata to the model.
    
    This will allow swh-storage to have a signature for *_metadata_add
    that is consistent with other *_add endpoints.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/101/ for more details.

Harbormaster completed remote builds in B13335: Diff 12114.Jul 3 2020, 5:10 PM
vlorentz added a child revision: D3418: Implement to_dict and from_dict for metadata-related classes..Jul 3 2020, 5:10 PM
ardumont added a subscriber: ardumont.Jul 3 2020, 6:06 PM
ardumont added inline comments.
swh/model/collections.py
10

What does KT and VT stands for?

KeyType and ValueType?

ardumont added a comment.Jul 3 2020, 6:08 PM

Seems right ;)

olasd added a subscriber: olasd.Jul 4 2020, 1:54 PM

That's great, thanks!

Does the serialization/deserialization of these data structures work ok? I think we'll need to do a "deep" (well, 2D really) tuple conversion, instead of just top level tuplification, as msgpack turns all tuples into lists.

I think ImmutableDict[str, object] should really be spelled ImmutableDict[str, Any]. Both versions are used currently.

vlorentz added a comment.Jul 6 2020, 11:54 AM
In D3417#84040, @olasd wrote:

Does the serialization/deserialization of these data structures work ok? I think we'll need to do a "deep" (well, 2D really) tuple conversion, instead of just top level tuplification, as msgpack turns all tuples into lists.

Probably not. I assumed we had tests for it, but it looks like we don't.

swh/model/collections.py
10

Yes. I'm reusing the names from typing.py.

vlorentz updated this revision to Diff 12132.Jul 6 2020, 11:58 AM

rebase

vlorentz updated this revision to Diff 12134.Jul 6 2020, 11:59 AM

fix base commit

swh-public-ci added a comment.Jul 6 2020, 12:00 PM

Build is green

Patch application report for D3417 (id=12132)

Rebasing onto 1ff05161e7...

Current branch diff-target is up to date.
Changes applied before test
commit 30e0967fbdb38701eb9f4fa16f31a5c0b74b4a79
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 3 17:08:19 2020 +0200

    Add a new ImmutableDict class, and use it in model objects.
    
    So they are truly immutable now.

commit da01882e0d68bfd7797a8e4e7502ab90906358ee
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Jul 2 17:53:13 2020 +0200

    Add raw metadata to the model.
    
    This will allow swh-storage to have a signature for *_metadata_add
    that is consistent with other *_add endpoints.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/104/ for more details.

Harbormaster completed remote builds in B13354: Diff 12132.Jul 6 2020, 12:00 PM
swh-public-ci added a comment.Jul 6 2020, 12:03 PM

Build is green

Patch application report for D3417 (id=12134)

Could not rebase; Attempt merge onto 1ff05161e7...

Updating 1ff0516..408e638
Fast-forward
 swh/model/collections.py            |  36 ++++
 swh/model/model.py                  | 255 +++++++++++++++++++++-
 swh/model/tests/test_collections.py |  42 ++++
 swh/model/tests/test_model.py       | 407 ++++++++++++++++++++++++++++++++++++
 4 files changed, 730 insertions(+), 10 deletions(-)
 create mode 100644 swh/model/collections.py
 create mode 100644 swh/model/tests/test_collections.py
Changes applied before test
commit 408e638b1dffe99e1afb1a3c77dbe0e14cb56042
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 3 17:08:19 2020 +0200

    Add a new ImmutableDict class, and use it in model objects.
    
    So they are truly immutable now.

commit d4ea9a56aca02b8aed15186372749bfbe639d69f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Jul 2 17:53:13 2020 +0200

    Add raw metadata to the model.
    
    This will allow swh-storage to have a signature for *_metadata_add
    that is consistent with other *_add endpoints.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/106/ for more details.

Harbormaster completed remote builds in B13356: Diff 12134.Jul 6 2020, 12:03 PM
vlorentz added a comment.Jul 6 2020, 12:34 PM
In D3417#84145, @vlorentz wrote:
In D3417#84040, @olasd wrote:

Does the serialization/deserialization of these data structures work ok? I think we'll need to do a "deep" (well, 2D really) tuple conversion, instead of just top level tuplification, as msgpack turns all tuples into lists.

Probably not. I assumed we had tests for it, but it looks like we don't.

Actually, yes it does, because dictify converts ImmutableDict to dict

ardumont accepted this revision.Jul 6 2020, 5:18 PM
This revision is now accepted and ready to land.Jul 6 2020, 5:18 PM
vlorentz added a parent revision: D3396: Add raw metadata to the model..Jul 7 2020, 11:07 AM
douardda mentioned this in D3396: Add raw metadata to the model..Jul 7 2020, 11:26 AM
vlorentz updated this revision to Diff 12184.Jul 7 2020, 11:49 AM

rebase

swh-public-ci added a comment.Jul 7 2020, 11:51 AM

Build has FAILED

Patch application report for D3417 (id=12184)

Could not rebase; Attempt merge onto a7d9aca2b0...

Updating a7d9aca..7e870da
Fast-forward
 swh/model/collections.py            |  36 ++++
 swh/model/model.py                  | 255 +++++++++++++++++++++-
 swh/model/tests/test_collections.py |  42 ++++
 swh/model/tests/test_model.py       | 407 ++++++++++++++++++++++++++++++++++++
 4 files changed, 730 insertions(+), 10 deletions(-)
 create mode 100644 swh/model/collections.py
 create mode 100644 swh/model/tests/test_collections.py
Changes applied before test
commit 7e870da44f87655ce027876a3b3885d60b8ffc57
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 3 17:08:19 2020 +0200

    Add a new ImmutableDict class, and use it in model objects.
    
    So they are truly immutable now.

commit 78fc5f7cbc500bde5d917b8f55fc6c9e79608479
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Jul 2 17:53:13 2020 +0200

    Add raw metadata to the model.
    
    This will allow swh-storage to have a signature for *_metadata_add
    that is consistent with other *_add endpoints.

Link to build: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/110/
See console output for more information: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/110/console

Harbormaster failed remote builds in B13404: Diff 12184!Jul 7 2020, 11:51 AM
vlorentz updated this revision to Diff 12198.Jul 7 2020, 1:20 PM

rebase + update for extra_headers / copy_pop

swh-public-ci added a comment.Jul 7 2020, 1:20 PM

Build has FAILED

Patch application report for D3417 (id=12198)

Rebasing onto a7d9aca2b0...

Current branch diff-target is up to date.
Changes applied before test
commit 2628ba1ab3e2163a043c2f1571c7a2a640d12cb0
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 3 17:08:19 2020 +0200

    Add a new ImmutableDict class, and use it in model objects.
    
    So they are truly immutable now.

commit 78fc5f7cbc500bde5d917b8f55fc6c9e79608479
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Jul 2 17:53:13 2020 +0200

    Add raw metadata to the model.
    
    This will allow swh-storage to have a signature for *_metadata_add
    that is consistent with other *_add endpoints.

Link to build: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/112/
See console output for more information: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/112/console

Harbormaster failed remote builds in B13418: Diff 12198!Jul 7 2020, 1:20 PM
vlorentz updated this revision to Diff 12201.Jul 7 2020, 1:31 PM

rebase + add test + fix type annotation

swh-public-ci added a comment.Jul 7 2020, 1:34 PM

Build is green

Patch application report for D3417 (id=12201)

Could not rebase; Attempt merge onto a7d9aca2b0...

Updating a7d9aca..af0dd1a
Fast-forward
 swh/model/collections.py            |  49 +++++
 swh/model/model.py                  | 263 +++++++++++++++++++++--
 swh/model/tests/test_collections.py |  50 +++++
 swh/model/tests/test_model.py       | 407 ++++++++++++++++++++++++++++++++++++
 4 files changed, 754 insertions(+), 15 deletions(-)
 create mode 100644 swh/model/collections.py
 create mode 100644 swh/model/tests/test_collections.py
Changes applied before test
commit af0dd1aaee4943d6cec0f50f77a14bcd4d8733b5
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jul 3 17:08:19 2020 +0200

    Add a new ImmutableDict class, and use it in model objects.
    
    So they are truly immutable now.

commit 78fc5f7cbc500bde5d917b8f55fc6c9e79608479
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Jul 2 17:53:13 2020 +0200

    Add raw metadata to the model.
    
    This will allow swh-storage to have a signature for *_metadata_add
    that is consistent with other *_add endpoints.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/115/ for more details.

Harbormaster completed remote builds in B13421: Diff 12201.Jul 7 2020, 1:34 PM
ardumont accepted this revision.Jul 7 2020, 1:40 PM
Closed by commit rDMODaf0dd1aaee49: Add a new ImmutableDict class, and use it in model objects. (authored by vlorentz). · Explain WhyJul 7 2020, 2:27 PM
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDMODaf0dd1aaee49: Add a new ImmutableDict class, and use it in model objects..

Revision Contents
Changeset List

PathSize
swh/
model/
49 lines
61 lines
tests/
50 lines

Diff 12208

View Options

swh/model/collections.py

Loading...
View Options

swh/model/model.py

Loading...
View Options

swh/model/tests/test_collections.py

Loading...
About · Developer information · Legal