Page MenuHomeSoftware Heritage

Add test for all qualifiers' converters and validators
ClosedPublic

Authored by vlorentz on Feb 23 2021, 1:19 PM.

Diff Detail

Event Timeline

Build is green

Patch application report for D5128 (id=18340)

Could not rebase; Attempt merge onto 758eb885d3...

Updating 758eb88..255d913
Fast-forward
 swh/model/identifiers.py            | 450 ++++++++++++++++++++++++--
 swh/model/tests/test_identifiers.py | 625 ++++++++++++++++++++++++++++++++++--
 tox.ini                             |   1 +
 3 files changed, 1037 insertions(+), 39 deletions(-)
Changes applied before test
commit 255d9132023a85ab0b980008818d9e44feab86ec
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 23 13:18:40 2021 +0100

    Add test for all qualifiers' converters and validators

commit afb64ffd4414d22745da904a2f1d2a6732a24916
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 23 12:04:20 2021 +0100

    Escape semicolon in origin qualifiers.

commit 023693d6891821cede2ef20318f444b75789080c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 23 11:32:01 2021 +0100

    Add test checking SWHID_QUALIFIERS matches the attributes of QualifiedSWHID.

commit 4a31540cec642cc0cfe11ca10f23cf19d362198d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 23 11:19:21 2021 +0100

    Fix qualifier parsing and add tests
    
    * Quote/unquote path
    * Fix line parsing and serializing to properly handle None
    * Fix error raised by check_visit/check_anchor

commit b55a60a830be4ab0b0fbc72d88dac978ce124739
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 16:37:44 2021 +0100

    Deduplicate parsing/unparsing tests of the new SWHID classes
    
    They were all very similar and only differ in what 'edge' cases they accept

commit bd84ec2860e2c8a16f804259c3fc9daa9cc6bd09
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 16:06:47 2021 +0100

    Deduplicate code between CoreSWHID, QualifiedSWHID, and ExtendedSWHID
    
    by making them all derive from an abstract class.

commit 36bfa7835736a9e3352657435e83234ef0ed6387
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 14:18:45 2021 +0100

    Add new class ExtendedSWHID as an alternative to SWHID/QualifiedSWHID
    
    Following the discussion on T3034, we decided to replace SWHID with
    two or three classes:
    
    * QualifiedSWHID to replace the existing SWHID (standard types + qualifiers)
    * CoreSWHID, for "core SWHID" only (standard types + no qualifiers)
    * ExtendedSWHID for internal use in Software Heritage (extra types + no qualifiers)
    
    This commit adds the last one.
    
    It also removes "ori" as a valid object type for CoreSWHID and
    QualifiedSWHID, as it now only belongs in ExtendedSWHID.

commit 992376597d6f44dfe78c4eef40bf651fe355f0fd
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 13:58:03 2021 +0100

    Use dict instead of temporary SWHID when parsing {Core,Qualified}SWHID.
    
    It is cleaner, avoids warnings, and will be needed when introducing
    ExtendedSWHID in a future commit.

commit 8e917597dbb22054ada06387a0501452d8862d34
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 10:56:29 2021 +0100

    QualifiedSWHID: Replace the 'qualifiers' dict with statically defined attributes
    
    And store their parsed values (CoreSWHID, tuple of ints, etc.) instead of string.

commit eba8d84de660e2b3d7df304fa32da7404fb9f6bb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 18 13:09:21 2021 +0100

    Add new class CoreSWHID as an alternative to SWHID/QualifiedSWHID
    
    Following the discussion on T3034, we decided to replace SWHID with
    two or three classes:
    
    * QualifiedSWHID to replace the existing SWHID (standard types + qualifiers)
    * CoreSWHID, for "core SWHID" only (standard types + no qualifiers)
    * ExtendedSWHID for internal use in Software Heritage (extra types + no qualifiers)
    
    This commit adds the second one

commit 690b7f824f55bc65fbec21b29714926476038abb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 16 13:33:13 2021 +0100

    Add new class QualifiedSWHID to replace SWHID, and deprecate the latter.
    
    Following the discussion on T3034, we decided to replace SWHID with
    two or three classes:
    
    * QualifiedSWHID to replace the existing SWHID (standard types + qualifiers)
    * CoreSWHID, for "core SWHID" only (standard types + no qualifiers)
    * ExtendedSWHID for internal use in Software Heritage (extra types + no qualifiers)
    
    Since migrating from SWHID will break existing code, this commit uses
    the opportunity to modernize it a little, ie.:
    
    * `keyword`-only constructor, to get rid of the hacky default values for
      `object_type` and `object_id`
    * enum instead of strings for the object type
    * `bytes` instead of an hex string for the object id
    * rename `metadata` to `qualifiers`

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/258/ for more details.

Build is green

Patch application report for D5128 (id=18345)

Could not rebase; Attempt merge onto 758eb885d3...

Updating 758eb88..24b653e
Fast-forward
 swh/model/identifiers.py            | 450 +++++++++++++++++++++++--
 swh/model/tests/test_identifiers.py | 637 ++++++++++++++++++++++++++++++++++--
 tox.ini                             |   1 +
 3 files changed, 1049 insertions(+), 39 deletions(-)
Changes applied before test
commit 24b653e4c09ddf7831db23f48e3b90e66bb361d6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 23 13:18:40 2021 +0100

    Add test for all qualifiers' converters and validators

commit 069b56af07e2fdb300a09164b96975e2b78c0f2d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 23 12:04:20 2021 +0100

    Escape semicolon in origin qualifiers.

commit 710fb4249e4b071aa77324da8a7f89ba0a0bad5c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 23 11:32:01 2021 +0100

    Add test checking SWHID_QUALIFIERS matches the attributes of QualifiedSWHID.

commit 7dead5df36cf445f7b03e1ed8743d9d9d461f301
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 23 11:19:21 2021 +0100

    Fix qualifier parsing and add tests
    
    * Quote/unquote path
    * Fix line parsing and serializing to properly handle None
    * Fix error raised by check_visit/check_anchor

commit 172eadb86cae99ccd65a98f4953ad5b422d8be27
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 16:37:44 2021 +0100

    Deduplicate parsing/unparsing tests of the new SWHID classes
    
    They were all very similar and only differ in what 'edge' cases they accept

commit 9bcc88469c1dd3080014dc4d05177724c2c9332d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 16:06:47 2021 +0100

    Deduplicate code between CoreSWHID, QualifiedSWHID, and ExtendedSWHID
    
    by making them all derive from an abstract class.

commit d4b20dcdc2795d544461ea469e4156968f9e2cda
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 14:18:45 2021 +0100

    Add new class ExtendedSWHID as an alternative to SWHID/QualifiedSWHID
    
    Following the discussion on T3034, we decided to replace SWHID with
    two or three classes:
    
    * QualifiedSWHID to replace the existing SWHID (standard types + qualifiers)
    * CoreSWHID, for "core SWHID" only (standard types + no qualifiers)
    * ExtendedSWHID for internal use in Software Heritage (extra types + no qualifiers)
    
    This commit adds the last one.
    
    It also removes "ori" as a valid object type for CoreSWHID and
    QualifiedSWHID, as it now only belongs in ExtendedSWHID.

commit 992376597d6f44dfe78c4eef40bf651fe355f0fd
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 13:58:03 2021 +0100

    Use dict instead of temporary SWHID when parsing {Core,Qualified}SWHID.
    
    It is cleaner, avoids warnings, and will be needed when introducing
    ExtendedSWHID in a future commit.

commit 8e917597dbb22054ada06387a0501452d8862d34
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Feb 19 10:56:29 2021 +0100

    QualifiedSWHID: Replace the 'qualifiers' dict with statically defined attributes
    
    And store their parsed values (CoreSWHID, tuple of ints, etc.) instead of string.

commit eba8d84de660e2b3d7df304fa32da7404fb9f6bb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 18 13:09:21 2021 +0100

    Add new class CoreSWHID as an alternative to SWHID/QualifiedSWHID
    
    Following the discussion on T3034, we decided to replace SWHID with
    two or three classes:
    
    * QualifiedSWHID to replace the existing SWHID (standard types + qualifiers)
    * CoreSWHID, for "core SWHID" only (standard types + no qualifiers)
    * ExtendedSWHID for internal use in Software Heritage (extra types + no qualifiers)
    
    This commit adds the second one

commit 690b7f824f55bc65fbec21b29714926476038abb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Feb 16 13:33:13 2021 +0100

    Add new class QualifiedSWHID to replace SWHID, and deprecate the latter.
    
    Following the discussion on T3034, we decided to replace SWHID with
    two or three classes:
    
    * QualifiedSWHID to replace the existing SWHID (standard types + qualifiers)
    * CoreSWHID, for "core SWHID" only (standard types + no qualifiers)
    * ExtendedSWHID for internal use in Software Heritage (extra types + no qualifiers)
    
    Since migrating from SWHID will break existing code, this commit uses
    the opportunity to modernize it a little, ie.:
    
    * `keyword`-only constructor, to get rid of the hacky default values for
      `object_type` and `object_id`
    * enum instead of strings for the object type
    * `bytes` instead of an hex string for the object id
    * rename `metadata` to `qualifiers`

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/263/ for more details.

This revision is now accepted and ready to land.Feb 24 2021, 5:28 PM