Differential D4065

deposit.parsers: Process namespace when using xmltodict.parse
ClosedPublic
Actions

Authored by ardumont on Sep 28 2020, 4:13 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Commits

rDDEPf05480888a31: deposit.parsers: Process namespace when using xmltodict.parse

Summary

This keeps the existing behavior though by mapping the expanded namespace (full url in keys)
to simpler namespaces (again as it is currently).

This also fixes some test data sample which were not properly defining their
namespace (and which now expectedly failed).

This will allow to deal properly with all empty body cases (only 1 so far). D4067 fixes that.

Test Plan

tox

Diff Detail

Repository

rDDEP Push deposit

Branch

master

Lint

Lint Skipped

Unit

Unit Tests Skipped

Build Status

Buildable 15658
Build 24108: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 24107: arc lint + arc unit

Event Timeline

ardumont created this revision.Sep 28 2020, 4:13 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptSep 28 2020, 4:13 PM

Build is green

Patch application report for D4065 (id=14357)

Could not rebase; Attempt merge onto 87b39c10ee...

Updating 87b39c10..d86c1922
Fast-forward
 swh/deposit/api/checks.py                                   |  2 ++
 swh/deposit/parsers.py                                      | 11 ++++++++++-
 swh/deposit/tests/api/test_deposit_private_read_metadata.py |  8 --------
 swh/deposit/tests/api/test_parser.py                        |  4 ----
 swh/deposit/tests/data/atom/entry-data3.xml                 |  2 +-
 5 files changed, 13 insertions(+), 14 deletions(-)

Changes applied before test

commit d86c192234a025a027202379789e074441dadbc3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Sep 28 16:09:59 2020 +0200

    deposit.parsers: Process namespace when using xmltodict.parse
    
    This keeps the existing behavior though by mapping the expanded name to simpler
    namespaces.
    
    This also fixes some test data sample which were not properly defining their
    namespace.
    
    This will allow to deal more properly the empty body case which is incomplete
    right now.

commit cc8d12524845811345735a00e836f29dea8fac8c
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Sep 28 14:15:48 2020 +0200

    deposit.api.checks: Add docstring to module

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/154/ for more details.

Harbormaster completed remote builds in B15653: Diff 14357.Sep 28 2020, 4:15 PM

ardumont edited the summary of this revision. (Show Details)Sep 28 2020, 4:18 PM

ardumont edited the summary of this revision. (Show Details)Sep 28 2020, 5:09 PM

nice!

Can you also remove the xmlns declaration in the "empty" XML data file?

vlorentz accepted this revision.Sep 28 2020, 5:34 PM

This revision is now accepted and ready to land.Sep 28 2020, 5:34 PM

Rebase

Build is green

Patch application report for D4065 (id=14362)

Rebasing onto 87b39c10ee...

Current branch diff-target is up to date.

Changes applied before test

commit f05480888a317c5cbc181ebdaa8d98e58914cc06
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Sep 28 16:09:59 2020 +0200

    deposit.parsers: Process namespace when using xmltodict.parse
    
    This keeps the existing behavior though by mapping the expanded name to simpler
    namespaces.
    
    This also fixes some test data sample which were not properly defining their
    namespace.
    
    This will allow to deal more properly the empty body case which is incomplete
    right now.

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/157/ for more details.

Harbormaster completed remote builds in B15658: Diff 14362.Sep 28 2020, 5:59 PM

Closed by commit rDDEPf05480888a31: deposit.parsers: Process namespace when using xmltodict.parse (authored by ardumont). · Explain WhySep 28 2020, 7:14 PM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDDEPf05480888a31: deposit.parsers: Process namespace when using xmltodict.parse.

vlorentz mentioned this in D4233: migrate_extrinsic_metadata: add support for the new deposit metadata formats introduced in late september..Oct 12 2020, 2:57 PM

vlorentz mentioned this in rDSTOb425b5ca48b8: migrate_extrinsic_metadata: add support for the new deposit metadata formats….Oct 12 2020, 3:57 PM

ardumont mentioned this in D4267: deposit.loader: Avoid unnecessary metadata json transformation.Oct 15 2020, 11:59 AM