Page MenuHomeSoftware Heritage

cli.deposit: Add mandatory flags to generate the metadata xml
ClosedPublic

Authored by ardumont on Apr 16 2019, 12:01 PM.

Details

Summary

Metadata file is now generated if the user provides the authors (multiple --author flags) and the name.

Related T1650

Test Plan

shell 1:

$ workon swh
$ doco up -d swh-deposit
$ doco logs -f swh-deposit

shell 2:

$ workon swh
$ swh-deposit deposit --verbose --username test --password test \
  --archive ../swh-docker-dev.tgz \
  --name swh-docker-dev \
  --author 'Antoine r. Dumont (@ardumont)' \
  --author 'Morane Gruenpeter'

INFO:swh.deposit.cli.deposit:{'deposit_id': '4', 'deposit_status': 'deposited', 'deposit_status_detail': None, 'deposit_date': 'April 16, 2019, 10:25 a.m.'}

We could still use the old way (mentioning the metadata file):

$ swh-deposit deposit --url http://localhost:5006/1 \
  --verbose \
  --username test \
  --password test \
  --collection test \
  --archive ../swh-docker-dev.tgz \
  --metadata ../swh-docker-dev.tgz.metadata.xml

INFO:swh.deposit.cli.deposit:{'deposit_id': '5', 'deposit_status': 'deposited', 'deposit_status_detail': None, 'deposit_date': 'April 16, 2019, 10:28 a.m.'}

You got rejected if you mention neither --metadata nor (--author and --name).

$ swh-deposit deposit --url http://localhost:5006/1 \
  --verbose \
  --username test \ 
  --password test \
  --collection test \
  --archive ../swh-docker-dev.tgz

INFO:swh.deposit.cli.deposit:{'error': 'Problem during parsing options: Either metadata deposit file or (`--name`  and `--author`) fields must be provided'}

Note: This assumes the use of the virtualenv and the swh-docker-dev as per swh's getting-started.

Diff Detail

Repository
rDDEP Push deposit
Branch
add-mandatory-flags
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 5486
Build 7445: tox-on-jenkinsJenkins
Build 7444: arc lint + arc unit

Event Timeline

Make it work using tempfile.mkstemp instead of NamedTemporaryFile

swh-deposit deposit --url http://localhost:5006/1 \
  --verbose \
  --username test \
  --password test \
  --collection test \
  --archive ../swh-docker-dev.tgz \
  --name swh-docker-dev \
  --author 'Antoine r. Dumont (@ardumont)' \
  --author 'Morane Gruenpeter'

INFO:swh.deposit.cli.deposit:{'deposit_id': '4', 'deposit_status': 'deposited', 'deposit_status_detail': None, 'deposit_date': 'April 16, 2019, 10:25 a.m.'}
ardumont added a subscriber: moranegg.
ardumont edited the test plan for this revision. (Show Details)

ardumont: could you do a paste with the generated xml? I can't seem to be able to run the client today (i'll have more time to reconstruct that part locally tomorrow)

Sure.

$ ipython
Python 3.7.2+ (default, Feb  2 2019, 14:31:48)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import logging

In [2]: logging.basicConfig(level=logging.DEBUG)

In [3]: from swh.deposit.cli.deposit import generate_metadata_file; name = 'some-awesome-tool'; author0 = 'Antoine R. Dumont (@ardumont)'; author1 = 'Morane G.'; authors = [author1, author0]; path = generate_metadata_file(name, authors)
DEBUG:root:Temporary file: /tmp/swh.deposit.cli.m79wstvz
DEBUG:root:Metadata dict to generate as xml: {'entry': {'@xmlns': 'http://www.w3.org/2005/Atom', '@xmlns:codemeta': 'https://doi.org/10.5063/SCHEMA/CODEMETA-2.0', 'codemeta:name': 'some-awesome-tool', 'codemeta:author': [{'codemeta:name':
'Morane G.'}, {'codemeta:name': 'Antoine R. Dumont (@ardumont)'}]}}
DEBUG:root:Metadata dict as xml generated: <?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0">
        <codemeta:name>some-awesome-tool</codemeta:name>
        <codemeta:author>
                <codemeta:name>Morane G.</codemeta:name>
        </codemeta:author>
        <codemeta:author>
                <codemeta:name>Antoine R. Dumont (@ardumont)</codemeta:name>
        </codemeta:author>
</entry>

Related P381

ardumont edited the test plan for this revision. (Show Details)

I'm going to try and run it, but am a bit blocked with swh-deposit command................
Maybe @douardda could validate he can do a deposit?

swh/deposit/cli/deposit.py
56

I would add the slug or the made-up slug to this file (if possible) with:
<codemeta:identifier> slug <codemeta:identifier>

301

Is this url used? or was it for the manadatory url entry?

I'm going to try and run it, but am a bit blocked with swh-deposit command................

I don't know what you mean by blocked. What's blocking?

In the test plan, i mention

Note: This assumes the use of the virtualenv and the swh-docker-dev as per swh's getting-started.

You need to follow the getting started [1] which uses both the virtualenv and the docker-env...

You might have to either use (still from the virtualenv):

  • python3 -m swh.deposit.cli.__init__ (instead of swh-deposit) as in:
$ workon swh
$ git checkout <the-right-branch-in-regards-to-this-diff>  # should be arcpatch-D1419
$ python3 -m swh.deposit.cli.__init__ --help  # urg!
  • or update your local virtualenv:
$ workon swh
$ cd swh-deposit
$ git checkout <the-right-branch>  # should be arcpatch-D1419
$ pip install -e .            # that updates your local install
$ swh-deposit --help  # <- should be in the right version, providing the right branch that is

[1] https://docs.softwareheritage.org/devel/getting-started.html#getting-started

Maybe @douardda could validate he can do a deposit?

Or you trust in the test plan? ;)

swh/deposit/cli/deposit.py
301

Is this url used?

yes, it is.
It is the real deposit server uri (that is not something new in that diff, that is the current code).

Thus the need to override it in the sample i provided.

or was it for the manadatory url entry?

It is not.

Blocked because I thought I can call swh-deposit like old times:-)
Now it is good for me (saw the slug changes live).

swh/deposit/cli/deposit.py
301

oh right, I thought it was the url of the deposit artifact.

This revision is now accepted and ready to land.Apr 17 2019, 2:40 PM
  • Remove unnecessary leftover check on no longer existing url fields

After pairing with Ardumont, the updates are validated!

This revision was automatically updated to reflect the committed changes.