Page MenuHomeSoftware Heritage

cli.deposit: Add mandatory flags to generate the metadata xml
ClosedPublic

Authored by ardumont on Apr 16 2019, 12:01 PM.

Details

Summary

Metadata file is now generated if the user provides the authors (multiple --author flags) and the name.

Related T1650

Test Plan

shell 1:

$ workon swh
$ doco up -d swh-deposit
$ doco logs -f swh-deposit

shell 2:

$ workon swh
$ swh-deposit deposit --verbose --username test --password test \
  --archive ../swh-docker-dev.tgz \
  --name swh-docker-dev \
  --author 'Antoine r. Dumont (@ardumont)' \
  --author 'Morane Gruenpeter'

INFO:swh.deposit.cli.deposit:{'deposit_id': '4', 'deposit_status': 'deposited', 'deposit_status_detail': None, 'deposit_date': 'April 16, 2019, 10:25 a.m.'}

We could still use the old way (mentioning the metadata file):

$ swh-deposit deposit --url http://localhost:5006/1 \
  --verbose \
  --username test \
  --password test \
  --collection test \
  --archive ../swh-docker-dev.tgz \
  --metadata ../swh-docker-dev.tgz.metadata.xml

INFO:swh.deposit.cli.deposit:{'deposit_id': '5', 'deposit_status': 'deposited', 'deposit_status_detail': None, 'deposit_date': 'April 16, 2019, 10:28 a.m.'}

You got rejected if you mention neither --metadata nor (--author and --name).

$ swh-deposit deposit --url http://localhost:5006/1 \
  --verbose \
  --username test \ 
  --password test \
  --collection test \
  --archive ../swh-docker-dev.tgz

INFO:swh.deposit.cli.deposit:{'error': 'Problem during parsing options: Either metadata deposit file or (`--name`  and `--author`) fields must be provided'}

Note: This assumes the use of the virtualenv and the swh-docker-dev as per swh's getting-started.

Diff Detail

Repository
rDDEP swh-deposit
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Apr 16 2019, 12:01 PM
ardumont planned changes to this revision.Apr 16 2019, 12:02 PM
ardumont updated this revision to Diff 4591.Apr 16 2019, 12:27 PM

Make it work using tempfile.mkstemp instead of NamedTemporaryFile

swh-deposit deposit --url http://localhost:5006/1 \
  --verbose \
  --username test \
  --password test \
  --collection test \
  --archive ../swh-docker-dev.tgz \
  --name swh-docker-dev \
  --author 'Antoine r. Dumont (@ardumont)' \
  --author 'Morane Gruenpeter'

INFO:swh.deposit.cli.deposit:{'deposit_id': '4', 'deposit_status': 'deposited', 'deposit_status_detail': None, 'deposit_date': 'April 16, 2019, 10:25 a.m.'}
ardumont edited the summary of this revision. (Show Details)Apr 16 2019, 12:28 PM
ardumont added a subscriber: moranegg.
ardumont edited the test plan for this revision. (Show Details)Apr 16 2019, 12:31 PM
ardumont edited the test plan for this revision. (Show Details)
ardumont edited the summary of this revision. (Show Details)Apr 16 2019, 1:39 PM

ardumont: could you do a paste with the generated xml? I can't seem to be able to run the client today (i'll have more time to reconstruct that part locally tomorrow)

Sure.

$ ipython
Python 3.7.2+ (default, Feb  2 2019, 14:31:48)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import logging

In [2]: logging.basicConfig(level=logging.DEBUG)

In [3]: from swh.deposit.cli.deposit import generate_metadata_file; name = 'some-awesome-tool'; author0 = 'Antoine R. Dumont (@ardumont)'; author1 = 'Morane G.'; authors = [author1, author0]; path = generate_metadata_file(name, authors)
DEBUG:root:Temporary file: /tmp/swh.deposit.cli.m79wstvz
DEBUG:root:Metadata dict to generate as xml: {'entry': {'@xmlns': 'http://www.w3.org/2005/Atom', '@xmlns:codemeta': 'https://doi.org/10.5063/SCHEMA/CODEMETA-2.0', 'codemeta:name': 'some-awesome-tool', 'codemeta:author': [{'codemeta:name':
'Morane G.'}, {'codemeta:name': 'Antoine R. Dumont (@ardumont)'}]}}
DEBUG:root:Metadata dict as xml generated: <?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0">
        <codemeta:name>some-awesome-tool</codemeta:name>
        <codemeta:author>
                <codemeta:name>Morane G.</codemeta:name>
        </codemeta:author>
        <codemeta:author>
                <codemeta:name>Antoine R. Dumont (@ardumont)</codemeta:name>
        </codemeta:author>
</entry>

Related P381

ardumont edited the test plan for this revision. (Show Details)Apr 16 2019, 5:12 PM
ardumont edited the test plan for this revision. (Show Details)
ardumont edited the summary of this revision. (Show Details)Apr 16 2019, 7:31 PM
ardumont added reviewers: moranegg, douardda.
ardumont updated this revision to Diff 4608.Apr 17 2019, 10:52 AM

Rebase on latest master

ardumont updated this revision to Diff 4614.Apr 17 2019, 11:58 AM

Rebase on latest master

I'm going to try and run it, but am a bit blocked with swh-deposit command................
Maybe @douardda could validate he can do a deposit?

swh/deposit/cli/deposit.py
56

I would add the slug or the made-up slug to this file (if possible) with:
<codemeta:identifier> slug <codemeta:identifier>

301

Is this url used? or was it for the manadatory url entry?

I'm going to try and run it, but am a bit blocked with swh-deposit command................

I don't know what you mean by blocked. What's blocking?

In the test plan, i mention

Note: This assumes the use of the virtualenv and the swh-docker-dev as per swh's getting-started.

You need to follow the getting started [1] which uses both the virtualenv and the docker-env...

You might have to either use (still from the virtualenv):

  • python3 -m swh.deposit.cli.__init__ (instead of swh-deposit) as in:
$ workon swh
$ git checkout <the-right-branch-in-regards-to-this-diff>  # should be arcpatch-D1419
$ python3 -m swh.deposit.cli.__init__ --help  # urg!
  • or update your local virtualenv:
$ workon swh
$ cd swh-deposit
$ git checkout <the-right-branch>  # should be arcpatch-D1419
$ pip install -e .            # that updates your local install
$ swh-deposit --help  # <- should be in the right version, providing the right branch that is

[1] https://docs.softwareheritage.org/devel/getting-started.html#getting-started

Maybe @douardda could validate he can do a deposit?

Or you trust in the test plan? ;)

swh/deposit/cli/deposit.py
301

Is this url used?

yes, it is.
It is the real deposit server uri (that is not something new in that diff, that is the current code).

Thus the need to override it in the sample i provided.

or was it for the manadatory url entry?

It is not.

moranegg accepted this revision.Apr 17 2019, 2:40 PM

Blocked because I thought I can call swh-deposit like old times:-)
Now it is good for me (saw the slug changes live).

swh/deposit/cli/deposit.py
301

oh right, I thought it was the url of the deposit artifact.

This revision is now accepted and ready to land.Apr 17 2019, 2:40 PM
ardumont updated this revision to Diff 4617.Apr 17 2019, 3:25 PM
  • Remove unnecessary leftover check on no longer existing url fields
moranegg accepted this revision.Apr 17 2019, 3:28 PM

After pairing with Ardumont, the updates are validated!

ardumont updated this revision to Diff 4620.Apr 17 2019, 3:32 PM

Rebase on latest master

ardumont marked an inline comment as done.Apr 17 2019, 3:33 PM
ardumont updated this revision to Diff 4621.Apr 17 2019, 3:35 PM

Plug on master

This revision was automatically updated to reflect the committed changes.