Page MenuHomeSoftware Heritage
Differential D7253

Make codemeta:name mandatory in codemeta:author and codemeta:contributor
ClosedPublic
Actions

Authored by vlorentz on Feb 24 2022, 4:45 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T3965: deposit: Validate Atom documents are well-formed
Commits
rDDEP7b3fcb933004: Make codemeta:name mandatory in codemeta:author and codemeta:contributor
Summary

Depends on D7252.

Diff Detail

Repository
rDDEP Push deposit
Branch
validate-metadata
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 27150
Build 42469: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 42468: arc lint + arc unit

Unit TestsFailed
View All

TimeTest
1,069 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.deposit.tests.api.test_checks::test_api_checks_check_metadata_ko_schema[<entry xmlns="http://www.w3.org/2005/Atom"\n xmlns:swh="https://www.softwareheritage.org/schema/2018/deposit"\n xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"\n xmlns:schema="http://schema.org/" >\n <url>some url</url>\n <codemeta:name>bar</codemeta:name>\n <codemeta:author>\n <schema:unknown-tag>should allow anything here</schema:unknown-tag>\n </codemeta:author>\n\n <swh:deposit>\n <swh:metadata-provenance>\n <schema:url>some-metadata-provenance-url</schema:url>\n </swh:metadata-provenance>\n </swh:deposit>\n</entry>\n-expected_summaries2]
metadata_ko = '<entry xmlns="http://www.w3.org/2005/Atom"\n xmlns:swh="https://www.softwareheritage.org/schema/2018/deposit"\n...hema:url>some-metadata-provenance-url</schema:url>\n </swh:metadata-provenance>\n </swh:deposit>\n</entry>\n' expected_summaries = [{'fields': ['codemeta:author'], 'summary': '.*codemeta:name expected.*'}] swh_checks_deposit = <swh.deposit.api.private.deposit_check.APIChecks object at 0x7fd9f6f3e6a0>
1,297 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.deposit.tests.api.test_basic_auth::test_service_document_basic
980 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.deposit.tests.api.test_checks::test_api_checks_check_metadata_ko[<entry xmlns:atom="http://www.w3.org/2005/Atom"\n xmlns:swh="https://www.softwareheritage.org/schema/2018/deposit"\n xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"\n xmlns:schema="http://schema.org/">\n <atom:url>something</atom:url>\n <atom:external_identifier>something-else</atom:external_identifier>\n <atom:title>foobar</atom:title>\n <author>foo</author>\n\n <swh:deposit>\n <swh:metadata-provenance>\n <schema:url>some-metadata-provenance-url</schema:url>\n </swh:metadata-provenance>\n </swh:deposit>\n</entry>\n-expected_summary3]
773 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.deposit.tests.api.test_checks::test_api_checks_check_metadata_ko[<entry xmlns="http://www.w3.org/2005/Atom"\n xmlns:swh="https://www.softwareheritage.org/schema/2018/deposit"\n xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"\n xmlns:schema="http://schema.org/" >\n <url>something</url>\n <external_identifier>something-else</external_identifier>\n <author>someone</author>\n\n <swh:deposit>\n <swh:metadata-provenance>\n <schema:url>some-metadata-provenance-url</schema:url>\n </swh:metadata-provenance>\n </swh:deposit>\n</entry>\n-expected_summary0]
781 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.deposit.tests.api.test_checks::test_api_checks_check_metadata_ko[<entry xmlns="http://www.w3.org/2005/Atom"\n xmlns:swh="https://www.softwareheritage.org/schema/2018/deposit"\n xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"\n xmlns:schema="http://schema.org/" >\n <url>something</url>\n <external_identifier>something-else</external_identifier>\n <codemeta:title>bar</codemeta:title>\n <author>someone</author>\n\n <swh:deposit>\n <swh:metadata-provenance>\n <schema:url>some-metadata-provenance-url</schema:url>\n </swh:metadata-provenance>\n </swh:deposit>\n</entry>\n-expected_summary2]
View Full Test Results (1 Failed · 307 Passed)

Event Timeline

vlorentz created this revision.Feb 24 2022, 4:45 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptFeb 24 2022, 4:45 PM
swh-public-ci added a comment.Feb 24 2022, 4:52 PM

Build has FAILED

Patch application report for D7253 (id=26280)

Could not rebase; Attempt merge onto ccff46762b...

Removing docs/specs/swh.xsd
Merge made by the 'recursive' strategy.
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   4 +-
 docs/specs/swh.xsd                                 |  60 ---
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  59 ++-
 swh/deposit/tests/api/test_checks.py               | 501 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/codemeta.xsd                       |  36 ++
 swh/deposit/xsd/swh.xsd                            |  81 ++++
 10 files changed, 622 insertions(+), 124 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/codemeta.xsd
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit 03e32bd94b3a5a7172eacf4161e0fed96d7d927e
Merge: ccff4676 4bb9ba86
Author: Jenkins user <jenkins@localhost>
Date:   Thu Feb 24 15:45:23 2022 +0000

    Merge branch 'diff-target' into HEAD

commit 4bb9ba86faa921283be3e6f8a89aba75356a1074
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:44:53 2022 +0100

    Make codemeta:name mandatory in codemeta:author and codemeta:contributor

commit 6a8e0112df12208a19007adac8df651e935b7688
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:23:43 2022 +0100

    Add basic validation for codemeta:author and codemeta:contributor
    
    For now, this only checks they aren't just a string

commit a58ea3bc9a0419becb307603ffc783668e95325e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:15:40 2022 +0100

    Validate codemeta:name
    
    This only checks the name is a string.

commit 52bf22e278f97f153f785634c4ca47be4637c33c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:04:43 2022 +0100

    Validate codemeta:dateModified.

commit d9b748d7d4779959e8f397526016ecd496dbcc44
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 15:57:40 2022 +0100

    Use xmlschema to validate dates, instead of custom code.
    
    For now this increases code complexity, but this will allow addition
    of other check more easily.

commit fb205f30fc92eb35631705cdb6ec9b639f6fe6b1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:35:42 2022 +0100

    Enforce create_origin/add_to_origin/reference are mutually exclusive in the schema

commit a0d5f3a6c4800f6ee460b30ad6eee04caf523e89
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 89c086661371c7dad7b661fb967b451c41b5d955
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit d2d904a283a1434d30cf8c68bfc9e708753f705f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit d56f08cb83bd25c70a8cb4bafc6df71b6844f5d2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

Link to build: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/775/
See console output for more information: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/775/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 24 2022, 4:52 PM
Harbormaster failed remote builds in B27150: Diff 26280!
vlorentz updated this revision to Diff 26282.Feb 24 2022, 4:54 PM

fix support of newer xmlschema versions

swh-public-ci added a comment.Feb 24 2022, 5:02 PM

Build is green

Patch application report for D7253 (id=26282)

Could not rebase; Attempt merge onto ccff46762b...

Removing docs/specs/swh.xsd
Merge made by the 'recursive' strategy.
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   4 +-
 docs/specs/swh.xsd                                 |  60 ---
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  59 ++-
 swh/deposit/tests/api/test_checks.py               | 501 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/codemeta.xsd                       |  36 ++
 swh/deposit/xsd/swh.xsd                            |  81 ++++
 10 files changed, 622 insertions(+), 124 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/codemeta.xsd
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit 4cd4dd6ed20d5ddce05e938ce20608f9d416abd8
Merge: ccff4676 3c723c5b
Author: Jenkins user <jenkins@localhost>
Date:   Thu Feb 24 15:54:52 2022 +0000

    Merge branch 'diff-target' into HEAD

commit 3c723c5bbc39a66a0ad09da8aad0b5227c837c5b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:44:53 2022 +0100

    Make codemeta:name mandatory in codemeta:author and codemeta:contributor

commit 6a8e0112df12208a19007adac8df651e935b7688
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:23:43 2022 +0100

    Add basic validation for codemeta:author and codemeta:contributor
    
    For now, this only checks they aren't just a string

commit a58ea3bc9a0419becb307603ffc783668e95325e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:15:40 2022 +0100

    Validate codemeta:name
    
    This only checks the name is a string.

commit 52bf22e278f97f153f785634c4ca47be4637c33c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:04:43 2022 +0100

    Validate codemeta:dateModified.

commit d9b748d7d4779959e8f397526016ecd496dbcc44
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 15:57:40 2022 +0100

    Use xmlschema to validate dates, instead of custom code.
    
    For now this increases code complexity, but this will allow addition
    of other check more easily.

commit fb205f30fc92eb35631705cdb6ec9b639f6fe6b1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:35:42 2022 +0100

    Enforce create_origin/add_to_origin/reference are mutually exclusive in the schema

commit a0d5f3a6c4800f6ee460b30ad6eee04caf523e89
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 89c086661371c7dad7b661fb967b451c41b5d955
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit d2d904a283a1434d30cf8c68bfc9e708753f705f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit d56f08cb83bd25c70a8cb4bafc6df71b6844f5d2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/776/ for more details.

Harbormaster completed remote builds in B27153: Diff 26282.Feb 24 2022, 5:02 PM
vlorentz requested review of this revision.Feb 24 2022, 5:02 PM
ardumont accepted this revision.Feb 24 2022, 5:50 PM
This revision is now accepted and ready to land.Feb 24 2022, 5:50 PM
vlorentz updated this revision to Diff 26308.Feb 28 2022, 10:24 AM

rebase

vlorentz added a task: T3965: deposit: Validate Atom documents are well-formed.Feb 28 2022, 10:24 AM
swh-public-ci added a comment.Feb 28 2022, 10:33 AM

Build is green

Patch application report for D7253 (id=26308)

Could not rebase; Attempt merge onto ccff46762b...

Updating ccff4676..7b3fcb93
Fast-forward
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   4 +-
 docs/specs/swh.xsd                                 |  60 ---
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  59 ++-
 swh/deposit/tests/api/test_checks.py               | 501 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/codemeta.xsd                       |  36 ++
 swh/deposit/xsd/swh.xsd                            |  81 ++++
 10 files changed, 622 insertions(+), 124 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/codemeta.xsd
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit 7b3fcb9330040865fb3b7b8116ad9b59edce063a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:44:53 2022 +0100

    Make codemeta:name mandatory in codemeta:author and codemeta:contributor

commit d72933b01a11d05ac806baf904353b743b048999
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:23:43 2022 +0100

    Add basic validation for codemeta:author and codemeta:contributor
    
    For now, this only checks they aren't just a string

commit 25d75f1cb9c67dd372a2618b51b853cf2a04d2dd
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:15:40 2022 +0100

    Validate codemeta:name
    
    This only checks the name is a string.

commit d8cf768a0ea2e7b078406d4d7f8fb90efbb37c82
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:04:43 2022 +0100

    Validate codemeta:dateModified.

commit 74d5567b31f4e72d37d33b424ff7a8dc57d06b02
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 15:57:40 2022 +0100

    Use xmlschema to validate dates, instead of custom code.
    
    For now this increases code complexity, but this will allow addition
    of other check more easily.

commit b49d5e6ab16bdc53bfcd7b942e4feebac7b19cf8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:35:42 2022 +0100

    Enforce create_origin/add_to_origin/reference are mutually exclusive in the schema

commit e087956040e35eee914135b4ec2871b04dd192c4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 0a6c03965a7a5446884ca07d21b7e03d6ab145c4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit 2e95623c6c7783ed407fa721f09289742ecccab8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit 8a7e0832641bda3b4c445793cfb92e3de6c5947e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/789/ for more details.

Harbormaster completed remote builds in B27181: Diff 26308.Feb 28 2022, 10:33 AM
Closed by commit rDDEP7b3fcb933004: Make codemeta:name mandatory in codemeta:author and codemeta:contributor (authored by vlorentz). · Explain WhyFeb 28 2022, 11:01 AM
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDDEP7b3fcb933004: Make codemeta:name mandatory in codemeta:author and codemeta:contributor.

Revision Contents
Changeset List

PathSize
swh/
deposit/
tests/
api/
46 lines
xsd/
10 lines

Diff 26280

View Options

swh/deposit/tests/api/test_checks.py

Loading...
View Options

swh/deposit/xsd/codemeta.xsd

Loading...
About · Developer information · Legal