Page MenuHomeSoftware Heritage
Differential D7252

Add basic validation for codemeta:author and codemeta:contributor
ClosedPublic
Actions

Authored by vlorentz on Feb 24 2022, 4:24 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T3965: deposit: Validate Atom documents are well-formed
Commits
rDDEPd72933b01a11: Add basic validation for codemeta:author and codemeta:contributor
Summary

For now, this only checks they aren't just a string

Depends on D7251.

Diff Detail

Repository
rDDEP Push deposit
Branch
validate-metadata
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 27146
Build 42462: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 42461: arc lint + arc unit

Event Timeline

vlorentz created this revision.Feb 24 2022, 4:24 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptFeb 24 2022, 4:24 PM
swh-public-ci added a comment.Feb 24 2022, 4:31 PM

Build is green

Patch application report for D7252 (id=26275)

Could not rebase; Attempt merge onto ccff46762b...

Removing docs/specs/swh.xsd
Merge made by the 'recursive' strategy.
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   4 +-
 docs/specs/swh.xsd                                 |  60 ---
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  59 ++-
 swh/deposit/tests/api/test_checks.py               | 455 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/codemeta.xsd                       |  34 ++
 swh/deposit/xsd/swh.xsd                            |  81 ++++
 10 files changed, 574 insertions(+), 124 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/codemeta.xsd
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit 7a52714a6dc7af074257ff7db603bdf2f22d7721
Merge: ccff4676 6a8e0112
Author: Jenkins user <jenkins@localhost>
Date:   Thu Feb 24 15:24:13 2022 +0000

    Merge branch 'diff-target' into HEAD

commit 6a8e0112df12208a19007adac8df651e935b7688
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:23:43 2022 +0100

    Add basic validation for codemeta:author and codemeta:contributor
    
    For now, this only checks they aren't just a string

commit a58ea3bc9a0419becb307603ffc783668e95325e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:15:40 2022 +0100

    Validate codemeta:name
    
    This only checks the name is a string.

commit 52bf22e278f97f153f785634c4ca47be4637c33c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:04:43 2022 +0100

    Validate codemeta:dateModified.

commit d9b748d7d4779959e8f397526016ecd496dbcc44
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 15:57:40 2022 +0100

    Use xmlschema to validate dates, instead of custom code.
    
    For now this increases code complexity, but this will allow addition
    of other check more easily.

commit fb205f30fc92eb35631705cdb6ec9b639f6fe6b1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:35:42 2022 +0100

    Enforce create_origin/add_to_origin/reference are mutually exclusive in the schema

commit a0d5f3a6c4800f6ee460b30ad6eee04caf523e89
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 89c086661371c7dad7b661fb967b451c41b5d955
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit d2d904a283a1434d30cf8c68bfc9e708753f705f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit d56f08cb83bd25c70a8cb4bafc6df71b6844f5d2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/774/ for more details.

Harbormaster completed remote builds in B27146: Diff 26275.Feb 24 2022, 4:31 PM
vlorentz requested review of this revision.Feb 24 2022, 4:31 PM
vlorentz added a task: T3965: deposit: Validate Atom documents are well-formed.Feb 24 2022, 4:32 PM
vlorentz added a child revision: D7253: Make codemeta:name mandatory in codemeta:author and codemeta:contributor.Feb 24 2022, 4:45 PM
ardumont accepted this revision.Feb 24 2022, 5:48 PM
This revision is now accepted and ready to land.Feb 24 2022, 5:48 PM
vlorentz updated this revision to Diff 26307.Feb 28 2022, 10:24 AM

rebase

swh-public-ci added a comment.Feb 28 2022, 10:33 AM

Build is green

Patch application report for D7252 (id=26307)

Could not rebase; Attempt merge onto ccff46762b...

Updating ccff4676..d72933b0
Fast-forward
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   4 +-
 docs/specs/swh.xsd                                 |  60 ---
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  59 ++-
 swh/deposit/tests/api/test_checks.py               | 455 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/codemeta.xsd                       |  34 ++
 swh/deposit/xsd/swh.xsd                            |  81 ++++
 10 files changed, 574 insertions(+), 124 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/codemeta.xsd
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit d72933b01a11d05ac806baf904353b743b048999
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:23:43 2022 +0100

    Add basic validation for codemeta:author and codemeta:contributor
    
    For now, this only checks they aren't just a string

commit 25d75f1cb9c67dd372a2618b51b853cf2a04d2dd
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:15:40 2022 +0100

    Validate codemeta:name
    
    This only checks the name is a string.

commit d8cf768a0ea2e7b078406d4d7f8fb90efbb37c82
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 16:04:43 2022 +0100

    Validate codemeta:dateModified.

commit 74d5567b31f4e72d37d33b424ff7a8dc57d06b02
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 15:57:40 2022 +0100

    Use xmlschema to validate dates, instead of custom code.
    
    For now this increases code complexity, but this will allow addition
    of other check more easily.

commit b49d5e6ab16bdc53bfcd7b942e4feebac7b19cf8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:35:42 2022 +0100

    Enforce create_origin/add_to_origin/reference are mutually exclusive in the schema

commit e087956040e35eee914135b4ec2871b04dd192c4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 0a6c03965a7a5446884ca07d21b7e03d6ab145c4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit 2e95623c6c7783ed407fa721f09289742ecccab8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit 8a7e0832641bda3b4c445793cfb92e3de6c5947e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/788/ for more details.

Harbormaster completed remote builds in B27180: Diff 26307.Feb 28 2022, 10:33 AM
Closed by commit rDDEPd72933b01a11: Add basic validation for codemeta:author and codemeta:contributor (authored by vlorentz). · Explain WhyFeb 28 2022, 11:01 AM
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDDEPd72933b01a11: Add basic validation for codemeta:author and codemeta:contributor.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
deposit/
tests/
api/
32 lines100%
Loading...
xsd/
13 lines--

Diff 26275

View Options

swh/deposit/tests/api/test_checks.py

Loading...
View Options

swh/deposit/xsd/codemeta.xsd

Loading...
About · Developer information · Legal