Page MenuHomeSoftware Heritage
Differential D7243

Add schema validation of <swh:deposit> using swh.xsd
ClosedPublic
Actions

Authored by vlorentz on Feb 24 2022, 1:31 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T3965: deposit: Validate Atom documents are well-formed
Commits
rDDEPe087956040e3: Add schema validation of <swh:deposit> using swh.xsd

Diff Detail

Repository
rDDEP Push deposit
Branch
validate-metadata
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 27125
Build 42423: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 42422: arc lint + arc unit

Event Timeline

vlorentz created this revision.Feb 24 2022, 1:31 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptFeb 24 2022, 1:31 PM
swh-public-ci added a comment.Feb 24 2022, 1:33 PM

Build has FAILED

Patch application report for D7243 (id=26253)

Could not rebase; Attempt merge onto ccff46762b...

Removing docs/specs/swh.xsd
Merge made by the 'recursive' strategy.
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   2 +-
 docs/specs/swh.xsd                                 |  60 ------
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  27 +++
 swh/deposit/tests/api/test_checks.py               | 224 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/swh.xsd                            |  66 ++++++
 9 files changed, 308 insertions(+), 76 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit e621a8598dff0045c6112554fdb7aedffc61e1f9
Merge: ccff4676 18dea765
Author: Jenkins user <jenkins@localhost>
Date:   Thu Feb 24 12:32:09 2022 +0000

    Merge branch 'diff-target' into HEAD

commit 18dea765d6aea395d319c0ebffed282d7b8b35e0
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 89c086661371c7dad7b661fb967b451c41b5d955
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit d2d904a283a1434d30cf8c68bfc9e708753f705f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit d56f08cb83bd25c70a8cb4bafc6df71b6844f5d2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

Link to build: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/761/
See console output for more information: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/761/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 24 2022, 1:33 PM
Harbormaster failed remote builds in B27125: Diff 26253!
vlorentz updated this revision to Diff 26256.Feb 24 2022, 1:37 PM

fix typing

vlorentz updated this revision to Diff 26258.Feb 24 2022, 1:38 PM

wrong commit

swh-public-ci added a comment.Feb 24 2022, 1:39 PM

Build has FAILED

Patch application report for D7243 (id=26258)

Could not rebase; Attempt merge onto ccff46762b...

Removing docs/specs/swh.xsd
Merge made by the 'recursive' strategy.
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   2 +-
 docs/specs/swh.xsd                                 |  60 ------
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  27 +++
 swh/deposit/tests/api/test_checks.py               | 224 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/swh.xsd                            |  66 ++++++
 9 files changed, 308 insertions(+), 76 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit 89ca2bd1247d0c03eaba19ebc53f8ad895a32e3a
Merge: ccff4676 51f4f659
Author: Jenkins user <jenkins@localhost>
Date:   Thu Feb 24 12:38:46 2022 +0000

    Merge branch 'diff-target' into HEAD

commit 51f4f659e2eece91ff0513d5d85c07208b8d269c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 89c086661371c7dad7b661fb967b451c41b5d955
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit d2d904a283a1434d30cf8c68bfc9e708753f705f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit d56f08cb83bd25c70a8cb4bafc6df71b6844f5d2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

Link to build: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/765/
See console output for more information: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/765/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 24 2022, 1:39 PM
Harbormaster failed remote builds in B27129: Diff 26258!
vlorentz updated this revision to Diff 26259.Feb 24 2022, 1:40 PM

fix typing again

swh-public-ci added a comment.Feb 24 2022, 1:43 PM

Build is green

Patch application report for D7243 (id=26256)

Could not rebase; Attempt merge onto ccff46762b...

Merge made by the 'recursive' strategy.
 docs/specs/swh.xsd                                 | 66 ++++++++++++----------
 .../{tests_migration => tests}/test_migrations.py  |  0
 2 files changed, 36 insertions(+), 30 deletions(-)
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
Changes applied before test
commit 44ab6ba4529a01bf69bfde49df6d05e45ae7327e
Merge: ccff4676 d2d904a2
Author: Jenkins user <jenkins@localhost>
Date:   Thu Feb 24 12:37:43 2022 +0000

    Merge branch 'diff-target' into HEAD

commit d2d904a283a1434d30cf8c68bfc9e708753f705f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit d56f08cb83bd25c70a8cb4bafc6df71b6844f5d2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/763/ for more details.

Harbormaster completed remote builds in B27127: Diff 26256.Feb 24 2022, 1:44 PM
swh-public-ci added a comment.Feb 24 2022, 1:47 PM

Build has FAILED

Patch application report for D7243 (id=26259)

Could not rebase; Attempt merge onto ccff46762b...

Removing docs/specs/swh.xsd
Merge made by the 'recursive' strategy.
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   2 +-
 docs/specs/swh.xsd                                 |  60 ------
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  27 +++
 swh/deposit/tests/api/test_checks.py               | 224 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/swh.xsd                            |  66 ++++++
 9 files changed, 308 insertions(+), 76 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit aee23926031ee727885bf74f42b0572bbb843dbc
Merge: ccff4676 db145054
Author: Jenkins user <jenkins@localhost>
Date:   Thu Feb 24 12:41:02 2022 +0000

    Merge branch 'diff-target' into HEAD

commit db145054e2b29b3a24b688cc38bf2419ccf1ffcb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 89c086661371c7dad7b661fb967b451c41b5d955
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit d2d904a283a1434d30cf8c68bfc9e708753f705f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit d56f08cb83bd25c70a8cb4bafc6df71b6844f5d2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

Link to build: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/766/
See console output for more information: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/766/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 24 2022, 1:47 PM
Harbormaster failed remote builds in B27130: Diff 26259!
vlorentz updated this revision to Diff 26261.Feb 24 2022, 1:58 PM

remove file inclusion; does not work when doc is installed from a package

swh-public-ci added a comment.Feb 24 2022, 2:04 PM

Build is green

Patch application report for D7243 (id=26261)

Could not rebase; Attempt merge onto ccff46762b...

Removing docs/specs/swh.xsd
Merge made by the 'recursive' strategy.
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   4 +-
 docs/specs/swh.xsd                                 |  60 ------
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  27 +++
 swh/deposit/tests/api/test_checks.py               | 224 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/swh.xsd                            |  66 ++++++
 9 files changed, 308 insertions(+), 78 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit 825c1649ec25a70ecaed6abd695197b58c396655
Merge: ccff4676 a0d5f3a6
Author: Jenkins user <jenkins@localhost>
Date:   Thu Feb 24 12:58:17 2022 +0000

    Merge branch 'diff-target' into HEAD

commit a0d5f3a6c4800f6ee460b30ad6eee04caf523e89
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 89c086661371c7dad7b661fb967b451c41b5d955
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit d2d904a283a1434d30cf8c68bfc9e708753f705f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit d56f08cb83bd25c70a8cb4bafc6df71b6844f5d2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/768/ for more details.

Harbormaster completed remote builds in B27132: Diff 26261.Feb 24 2022, 2:04 PM
vlorentz requested review of this revision.Feb 24 2022, 2:04 PM
vlorentz added a child revision: D7244: Enforce create_origin/add_to_origin/reference are mutually exclusive in the schema.Feb 24 2022, 4:05 PM
vlorentz added a parent revision: D7242: Improve readability of pytest errors by removing unnecessary whitespaces.
vlorentz added a task: T3965: deposit: Validate Atom documents are well-formed.Feb 24 2022, 4:32 PM
ardumont accepted this revision.Feb 24 2022, 4:53 PM
ardumont added a subscriber: ardumont.

lgtm

This revision is now accepted and ready to land.Feb 24 2022, 4:53 PM
vlorentz updated this revision to Diff 26302.Feb 28 2022, 10:20 AM

rebase

swh-public-ci added a comment.Feb 28 2022, 10:27 AM

Build is green

Patch application report for D7243 (id=26302)

Could not rebase; Attempt merge onto ccff46762b...

Updating ccff4676..e0879560
Fast-forward
 MANIFEST.in                                        |   1 +
 docs/specs/protocol-reference.rst                  |   4 +-
 docs/specs/swh.xsd                                 |  60 ------
 mypy.ini                                           |   3 +
 requirements-server.txt                            |   1 +
 swh/deposit/api/checks.py                          |  27 +++
 swh/deposit/tests/api/test_checks.py               | 224 +++++++++++++++++++--
 .../{tests_migration => tests}/test_migrations.py  |   0
 swh/deposit/xsd/swh.xsd                            |  66 ++++++
 9 files changed, 308 insertions(+), 78 deletions(-)
 delete mode 100644 docs/specs/swh.xsd
 rename swh/deposit/{tests_migration => tests}/test_migrations.py (100%)
 create mode 100644 swh/deposit/xsd/swh.xsd
Changes applied before test
commit e087956040e35eee914135b4ec2871b04dd192c4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:30:33 2022 +0100

    Add schema validation of <swh:deposit> using swh.xsd

commit 0a6c03965a7a5446884ca07d21b7e03d6ab145c4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 13:24:10 2022 +0100

    Improve readability of pytest errors by removing unnecessary whitespaces
    
    The leading newline prevented textwrap.dedent from removing them.

commit 2e95623c6c7783ed407fa721f09289742ecccab8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:13:50 2022 +0100

    Fix swh.xsd
    
    It was never actually tested...

commit 8a7e0832641bda3b4c445793cfb92e3de6c5947e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Feb 24 12:04:03 2022 +0100

    Move tests_migration/test_migrations.py to tests/test_migrations.py
    
    This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
    lacks a conftest to initialize the database.

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/783/ for more details.

Harbormaster completed remote builds in B27175: Diff 26302.Feb 28 2022, 10:27 AM
Closed by commit rDDEPe087956040e3: Add schema validation of <swh:deposit> using swh.xsd (authored by vlorentz). · Explain WhyFeb 28 2022, 11:01 AM
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDDEPe087956040e3: Add schema validation of <swh:deposit> using swh.xsd.

Revision Contents
Changeset List

PathSize
1 line
docs/
specs/
2 lines
3 lines
1 line
swh/
deposit/
api/
27 lines
tests/
api/
196 lines
swh/
deposit/
xsd/
docs/
specs/
66 lines

Diff 26253

View Options

MANIFEST.in

Loading...
View Options

docs/specs/protocol-reference.rst

Loading...
View Options

docs/specs/swh.xsd

Loading...
View Options

mypy.ini

Loading...
View Options

requirements-server.txt

Loading...
View Options

swh/deposit/api/checks.py

Loading...
View Options

swh/deposit/tests/api/test_checks.py

Loading...
View Options

swh/deposit/xsd/swh.xsd

Loading...
About · Developer information · Legal