Page MenuHomeSoftware Heritage

Add an ExtID object
ClosedPublic

Authored by douardda on Jan 5 2021, 5:16 PM.

Diff Detail

Repository
rDMOD Data model
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D4807 (id=17038)

Rebasing onto 18fde50472...

Current branch diff-target is up to date.
Changes applied before test
commit 0366dc68e38b20933809142638db1cf73992cf2f
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jan 4 12:33:04 2021 +0100

    Add an ExtID object

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/191/ for more details.

vlorentz added inline comments.
swh/model/model.py
591

Not an enum?

swh/model/model.py
592–593

What about a SWHID instead?

add a unique_key method to ExtID and some tests

Build has FAILED

Patch application report for D4807 (id=17091)

Rebasing onto 18fde50472...

Current branch diff-target is up to date.
Changes applied before test
commit 0268627e43054049f8363ecb6497ee7516233416
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jan 4 12:33:04 2021 +0100

    Add an ExtID object

Link to build: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/192/
See console output for more information: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/192/console

swh/model/model.py
591

Not sure yet if we want an enum for this. So I started simple...

592–593

we do not (want) to support generic SWHID (with context), plus swhid is a textual representation that will need string manipulations / (en|de)coding, etc. so the idea was to stick, in the data model, to stricly what we need.

swh/model/model.py
592–593

we do not (want) to support generic SWHID (with context)

neither does raw_extrinsic_metadata, there's a validator to check it has no context.

fix hypothesis strategies tests

Build has FAILED

Patch application report for D4807 (id=17095)

Rebasing onto 18fde50472...

Current branch diff-target is up to date.
Changes applied before test
commit 0268627e43054049f8363ecb6497ee7516233416
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jan 4 12:33:04 2021 +0100

    Add an ExtID object

Link to build: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/193/
See console output for more information: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/193/console

same but with all files commited

Build is green

Patch application report for D4807 (id=17096)

Rebasing onto 18fde50472...

Current branch diff-target is up to date.
Changes applied before test
commit 10ade43f14328c09d931e0ce10df25f6e9231f6a
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jan 4 12:33:04 2021 +0100

    Add an ExtID object

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/194/ for more details.

adapted to nes SWHID classes

Build is green

Patch application report for D4807 (id=18625)

Rebasing onto 2185f9308e...

Current branch diff-target is up to date.
Changes applied before test
commit 1735c1a1f865c178b42991685b0c486cfd8a68c9
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jan 4 12:33:04 2021 +0100

    Add an ExtID object
    
    this object aims at being able to keep in the SWH Archive an
      SWHID <-> External object ID
    map, e.g. to be able to keep track of Mercurial ids so the Mercurial
    loader can be made more efficient.
    
    Related to T2849.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/282/ for more details.

douardda retitled this revision from [WIP] Add an ExtID object to Add an ExtID object.Mar 8 2021, 11:27 AM

Rebase, make ExtID a HashableObject, and remove hypothesis strategies for ExtID

the mood is to get rid of those nowadays.

Build is green

Patch application report for D4807 (id=18686)

Rebasing onto fca36585a3...

Current branch diff-target is up to date.
Changes applied before test
commit b2a0d79d08a6a4d531b84a3d034a40ffdb91cfa4
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jan 4 12:33:04 2021 +0100

    Add an ExtID object
    
    this object aims at being able to keep in the SWH Archive an
      SWHID <-> External object ID
    map, e.g. to be able to keep track of Mercurial ids so the Mercurial
    loader can be made more efficient.
    
    Related to T2849.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/284/ for more details.

swh/model/identifiers.py
832–833

and should mention $StrWithoutSpaces is ASCII-encoded and something like this quote from the raw-ext-meta:

Newlines in $Bytes are escaped as with other git fields,
ie. by adding a space after them.
843
845

probably

swh/model/tests/swh_model_data.py
134–145

;)

Build is green

Patch application report for D4807 (id=18706)

Rebasing onto fca36585a3...

Current branch diff-target is up to date.
Changes applied before test
commit c4ffa76fdf0ec974d446d0fe5347648d9cd5644c
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jan 4 12:33:04 2021 +0100

    Add an ExtID object
    
    this object aims at being able to keep in the SWH Archive an
      SWHID <-> External object ID
    map, e.g. to be able to keep track of Mercurial ids so the Mercurial
    loader can be made more efficient.
    
    Related to T2849.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/285/ for more details.

This revision is now accepted and ready to land.Mar 10 2021, 10:15 AM
olasd added a subscriber: olasd.

I wish we dropped the $object_identifier(dict) pattern in favor of a method of the model objects, but I guess it's consistent.

I wonder if the TEST_OBJECTS addition will break the tests of swh.storage until D4698 lands.

swh/model/model.py
1113–1119

You're manually handling all the attributes, why not just call the class initializer directly?

swh/model/model.py
1113–1119

possibly "just" the result of several refactorings... lemme check

Apply vlorentz and olasd's suggestions

Build is green

Patch application report for D4807 (id=18720)

Rebasing onto fca36585a3...

Current branch diff-target is up to date.
Changes applied before test
commit 71be46174b87d9658259517622fa689c68a003db
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jan 4 12:33:04 2021 +0100

    Add an ExtID object
    
    this object aims at being able to keep in the SWH Archive an
      SWHID <-> External object ID
    map, e.g. to be able to keep track of Mercurial ids so the Mercurial
    loader can be made more efficient.
    
    Related to T2849.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/286/ for more details.

In D4807#132570, @olasd wrote:

I wonder if the TEST_OBJECTS addition will break the tests of swh.storage until D4698 lands.

yes it will break tests there...

This revision was automatically updated to reflect the committed changes.